Floating Point Math Anomaly

[dmmons]

Processor: Micrologix 1400

When doing subtraction using floating points (SUB F34:0 F94:10 F94:15 ) I am subtracting two values which have a maximum of 3 decimal places and always end in either 0 or 5.

For instance 12.440 - 11.445

I would expect and answer of 0.995, instead I get an answer of 0.9949999.

I suspect it is something to do with floating point math, but can't quite figure it out. Rockwell tech support is also working on a solution.

Anyone here have any ideas?

Thanks

 

[dmmons]

AB Tech Note 59372 sums up the answer pretty clearly.

 

[lostcontrol]

You would think tech support would also be aware of the tech note..

 

[nwboson]

Try wikipedia of 'floating point'. Those are 32 bit single precision numbers on the 1400. Every floating point number has a limited amount of precision: 24 binary bits is around 7 decimal digits. If you need fixed precision you'll want to use integers: the 1400 supports 32 bit integers (L).

 

[daba]

Processor: Micrologix 1400

When doing subtraction using floating points (SUB F34:0 F94:10 F94:15 ) I am subtracting two values which have a maximum of 3 decimal places and always end in either 0 or 5.

For instance 12.440 - 11.445

I would expect and answer of 0.995, instead I get an answer of 0.9949999.

I suspect it is something to do with floating point math, but can't quite figure it out. Rockwell tech support is also working on a solution.

Anyone here have any ideas?

Thanks

So you don't consider 0.994999 to be close enough to 0.995 to be useable ? It is my understanding they landed men on the moon with far less precision than that in their maths!!

 

[rootboy]

So you don't consider 0.994999 to be close enough to 0.995 to be useable ? It is my understanding they landed men on the moon with far less precision than that in their maths!!

Yes, but we didn't land close enough to get "Two cheeseburgers and a Big Mac to go"

 

[kamenges]

That is a blast from the past, rootboy. I never could land that goofy thing without turning it into a bunch of little line segments!!

Keith

 

[rootboy]

That is a blast from the past, rootboy. I never could land that goofy thing without turning it into a bunch of little line segments!!

Keith

I wondered how long it would take for someone to get the reference. I never managed to land the darn thing either!

Remember the size of the paper tape program? It took 43 minutes to load it up (and lord help you if you were the guy to take McDonalds out with your lander).

 

[bernie_carlton]

If the imprecision bugs you do the same math with integers or longs (use the numbers * 1000).

Each computerized method of storing numbers has its own strengths and weaknesses. They are tools. Study them. Use the best one for the particular job you are doing.

 

[Peter Nachtwey]

If the imprecision bugs you do the same math with integers or longs (use the numbers * 1000).

Each computerized method of storing numbers has its own strengths and weaknesses. They are tools. Study them. Use the best one for the particular job you are doing.

Use fractions where it takes 3 integers to represent a number. There is the whole number, the numerator and the denominator. Fractions are precise. Where things go wrong is when you try to convert the fractions to floating point. Then 199/200 still gets converted to 0.9949999

 

[daba]

Use fractions where it takes 3 integers to represent a number. There is the whole number, the numerator and the denominator. Fractions are precise. Where things go wrong is when you try to convert the fractions to floating point. Then 199/200 still gets converted to 0.9949999

Logix 5562, f/w rev 20