Loops in twincat 3

Vijolica555 · Jan 20, 2021

Hi guys,

I am using TwinCAT 3 and I have a laser sensor that sends me data in Array. At the beginning of the measurement I run reference scan where I calculate mean value of 100 received arrays. In the moment we have some weird surface and received values in the array are sometimes very wrong. Idea is to change those values with next good value. So we set some limits what is bad and what is good. Unfortunately what i wrote for conditions, bad values are not changed with the NEXT good one but with the LAST good one. If i use WHILE loop instead the inner IF is not good. Any ideas what could be changed? Thank you.

Code:

nHeight := REAL;
nTolerance := REAL
j := INT;
n := INT;
ArrLen := INT; 
arrReceive := : ARRAY [0..ArrLen] OF REAL;

Code:

FOR j:=0 TO ArrLen DO
IF (arrReceive[j] < nHeight*(1-nTolerance)) OR (arrReceive[j] > nHeight*(1+Tolerance)) THEN
	FOR n := j+1 TO ArrLen DO // find next value that meets the conditions
		IF (arrReceive[n] > nHeight*(1-nTolerance) AND arrReceive[n] < nHeight*(1+nTolerance)) THEN
	arrReceive[j] := arrReceive[n]; //value that meets the conditions is writen in the j value that didnt meet the conditions
		END_IF
	END_FOR
	ELSE arrReceive[j] := arrReceive[j];
	END_IF
END_FOR

kolyur · Jan 20, 2021

I suspect the problem is that you need to terminate the inner loop once the first acceptable value is found. Otherwise, the loop will continue running to the very end of the array as you have seen. So you could add an EXIT command to the innermost IF statement:

Code:

FOR j:=0 TO ArrLen DO
  IF (arrReceive[j] < nHeight*(1-nTolerance)) OR (arrReceive[j] > nHeight*(1+Tolerance)) THEN
    FOR n := j+1 TO ArrLen DO // find next value that meets the conditions
      IF (arrReceive[n] > nHeight*(1-nTolerance) AND arrReceive[n] < nHeight*(1+nTolerance)) THEN
        arrReceive[j] := arrReceive[n]; //value that meets the conditions is writen in the j value that didnt meet the conditions
        EXIT;    
      END_IF
    END_FOR
  ELSE arrReceive[j] := arrReceive[j];
  END_IF
END_FOR

drbitboy · Jan 20, 2021

Vijolica555 said:
...calculate mean value of 100 received arrays. In the moment we have some weird surface and received values in the array are sometimes very wrong.

Google "resistant average outlier"

Using a median is one way to deal with this, assuming the outliers are either few or occur on both sides of the target value.

drbitboy · Jan 20, 2021

1) @kolyur nailed it.

2) an alternative approach to solve the problem:

When an outlier is detectedi:
i) don't add anything to the sum
ii) decrement the divisor (which started at 100) for the average.

This has the advantage of not requiring a separate loop (although the current approach does not either).

E.g. take an average of 95 believable values i.e. effectively

SUM(valid_values[1:95]) / 95

instead of

(SUM(valid_values[1:95]) + SUM(replacement_values(1:5)) / 100

The answer will be different than the algorithm proposed in the OP, but not by much, assuming the number of outliers is low (<10 or so).

3) the statement

ELSE arrReceive[j] := arrReceive[j];

is effectively a no-op and can be removed.

Vijolica555 · Jan 21, 2021

Thank you! EXIT is the Answer, i tried with RETURN.

Vijolica555 · Jan 21, 2021

Hello!
Thank you for you proposal. We have tried that. But on the surface we are measuring in the moment the outliers came to often. Sometimes more of them in every Array so we couldn't take this approach.

drbitboy · Jan 21, 2021

Vijolica555 said:
Hello!
Thank you for you proposal. We have tried that. But on the surface we are measuring in the moment the outliers came to often. Sometimes more of them in every Array so we couldn't take this approach.

The approach will not matter: statistically (over time) the difference between the two methods* will have a mean of zero**. See the code below which suggests this is the case: the average difference between the two methods over 100k samples with numbers varying from 0 to ~1000 is of order 0.1, or around 0.01%.

Also note that the OP algorithm performance is O(N**2) worst case, while O(N) is possible by running the loop index backwards from Arrlen to 0.

Finally, the OP might be better served finding out where so many outliers are coming from: if existing sensors are returning values that are known to be invalid, then something is wrong with the process; perhaps something is loose.

* If implemented correctly. the current algorithm does not behave as intended: if the last several values in the array are outliers, then they will not be replaced and those outliers will contribute to the average; and since OP says there are many outliers, this issue will have a larger effect.

** assuming a reasonable (uniform or Gaussian) distribution.

update: fix bugs, give more info.

Code:

import os
from random import uniform as u, gauss as g
from math import sqrt

outer = 0
diffsum,diffcount = 0.0,0.0
midsmeansum,midsmeansquaredsum = 0.0,0.0
arrmeansum,arrmeansquaredsum = 0.0,0.0
r100 = range(100)
do_gauss = "GAUSS" in os.environ
maxdiff = 0.0
while outer < 100000:
  outer += 1
  if do_gauss:
    arr = [int(g(500.0,250.0)) for i in r100]
  else:
    arr = [int(u(0,1000)) for i in r100]
  mids = [arrval for arrval in arr if arrval>249 and arrval<750]
  if len(mids)<1: continue
  for i in r100:
    if arr[i]>249 and arr[i]<750: continue
    j = i + 1
    while j<100:
      if arr[j]>249 and arr[j]<750:
        arr[i] = arr[j]
        break
      j += 1
    if j>99:
      j = 0
      while j<i:
        if arr[j]>249 and arr[j]<750:
          arr[i] = arr[j]
          break
        j += 1
    assert j<100
  
  midsmean = sum(mids) / float(len(mids))
  arrmean = sum(arr) / 100.0
  diff = midsmean - arrmean
  diffsum += diff
  maxdiff = max([maxdiff,diff])

  midsmeansum += midsmean
  midsmeansquaredsum += midsmean*midsmean
  arrmeansum += arrmean
  arrmeansquaredsum += arrmean*arrmean

  diffcount += 1.0

midsmean,arrmean = midsmeansum / diffcount, arrmeansum/diffcount

midsstddev = sqrt(((midsmeansquaredsum/diffcount) - (midsmean**2)) * diffcount / (diffcount-1.0))
arrstddev = sqrt(((arrmeansquaredsum/diffcount) - (arrmean**2)) * diffcount / (diffcount-1.0))

print((diffsum/diffcount,diffcount,maxdiff,midsmean,arrmean,midsstddev,arrstddev,do_gauss))

Loops in twincat 3

Vijolica555

Member

kolyur

Lifetime Supporting Member + Moderator

drbitboy

Lifetime Supporting Member

drbitboy

Lifetime Supporting Member

Vijolica555

Member

Vijolica555

Member

drbitboy

Lifetime Supporting Member

Similar Topics