Ideas on how to deal with bad data

Hi everyone,
PLC: Micrologix 1400

I have a application that is looking at 2 different flow meters and comparing there totalizer for leak detection. I get the data from both meters via Modbus.

What I do is take in both reading compare them, compute a % difference and average the difference. I also keep track of the highest % difference.

Most of the time there is very little if any differences between the 2 totalizers. less than 1% most of the time But every now and then I get a difference that is way off the chart, like 135%. I never catch it happening but I see it in the highest % register. When I finally get to look at it live the difference is back to normal.

I believe it is caused by one meter not responding to a Modbus poll in time or at all. It doesn't happen a lot but just enough to make things a problem. I need to set an alarm bit if the % difference is above 5%.

So I am looking for some ideas on how I might "filter" out those times when the data is bad and only alarm when it is consistently off.

What I am thinking is that every time the difference is above 5% I would throw away that value and add 1 to a counter. When the counter reaches a number say 5 I would then let the value stand and set the alarm bit. If I get a difference less than 5% I would reset the counter.

I would like to solicit other ideas here. You guys are always good at coming up with that ah ha idea. The product is water so there is no big environmental issue in waiting a few minutes before sounding the alarm.

Thanks for the help.

If it were me, I would move both the values into an array when the difference is > 5% every 30 seconds with a time stamp from the wallclock and look for any patterns that may occur.

Depending on the results, if it very infrequent, I would just ignore a difference of >5% if it occurs for less than say 30 seconds.

If it turns out to be very frequent, then you will need to deal with the cause rather than program around it IMO

don't do the comparison unless all of the modbus polls have been successful for the last X seconds?

Trend is your friend. Trend you flow rates/totals too if realistic.

Also, make a couple of tags called something like DataAgeNode_1, DataAgeNade_2. Tie them to timer accumulators that get reset on the rising edge of the corresponding message DN bits. Trend those too. If Data age follows a consistent sawtooth and then occasionally spikes, you know you missed a message and when, and for how long you went between successful polls.

Like V0N_hydro said, is there a way to determine you had a successful data transfer?

If not, before processing any logic, check the outlandishness of the values and disregard it. If the value is above 100% it is clearly wrong and instead of using that value you use the previous one.
To prevent this from going unnoticed, count the number of times you get this error in a time interval and if X amount of times within Y amount of minutes, raise a different alarm.

Ideally though, I would look at the wiring and poll rate first.