Add REAL numbers in Logix (Compact) - odd precision

[phuz]

I can't say I recall ever encountering this, but when adding a decimal number to a REAL, I'm getting a strange precision.

For example, starting with a REAL value of 3.4, I add 0.4 and get 3.8000002. Next add I get 4.2000003. Why is Logix doing this? Is this a IEEE 754 format issue?

 

[dmroeder]

I can't say I recall ever encountering this, but when adding a decimal number to a REAL, I'm getting a strange precision.

For example, starting with a REAL value of 3.4, I add 0.4 and get 3.8000002. Next add I get 4.2000003. Why is Logix doing this? Is this a IEEE 754 format issue?

I wouldn't call it an "issue", just the way floating point numbers work.

https://www.youtube.com/watch?v=PZRI1IfStY0

 

[drbitboy]

https://www.youtube.com/watch?v=jg1mYsIrFPs&t=65s

Since the values 3.4, 3.8, and 4.2 cannot be exactly represented in IEEE-754, or any, floating-point binary (base 2) format, when converting to decimal symbols ("1.234567890..."), at some point the symbols must diverge from the rational number (34/10).

When adding two such numbers together (3.4 and 0.4), the combined "error" will show up in the least-significant digits.

• 3.4
  • may display as 3.4,
  • but it is internally 3.400000095367431640625000000
    • I don't know why it does not display as 3.4000001
• 0.4
  • may display as 0.4,
  • but it is internally 0.400000005960464477539062500
• The sum of those two internal values is 3.80000019073486328125000000000000000
• which will display as 3.8000002

Logix REAL format representation is 32-bit: 1 bit for sign; 8 bits for exponent; 24 bits for mantissa. The mantissa represents a number in the range [1:0b10) i.e. in the range [1:2) N.B. the right side is exclusive. In binary:

0b1*0000000000000000000000   <= equal to 1

0b1*1111111111111111111111   <= slightly less than 2

​

Where

• 0b => prefix indicating the following symbol is a number in binary, that is, the only available digits are 0 and 1
• * => a "binal" point i.e. a binary analog to a decimal point,
  • so,
    • in decimal i.e. base 10: (15 ÷ 10) = 1.5
    • in binary i.e. base 2: (0b11 ÷ 0b10) = 0b1*1
      • N.B.
        • 0b11 = 3 (decimal)
        • 0b10 = 2 (decimal)
        • ∴
          • (0b11 ÷ 0b10) = (3 ÷ 2) = 1.5 decimal = 0b1*1 binary

So, 0b1*11111111111111111111111 = (0b111111111111111111111111 ÷ 0b100000000000000000000000) = (16777215 ÷ 8388608) decimal = (2 - (1 ÷ 8388608)) decimal

The exponent part of the IEEE-754 moves that binal point to the left or the right, the same way the exponent in the scientific notation representation 1.50000e3 (1.5 x 10³) moves it to be the equivalent 1500.00 in non-exponential decimal symbol format.

And finally: the magnitude of the right-most bit is between one-8388608th and one-16777216th relative to the value itself. i.e. around one part in ten million i.e. 1 part in 10⁷. So any error between the rational number expressed in the IEEE-754 binary number (0b1... ÷ 2^exponent) and its decimal representation will show up around the seventh significant decimal digit to the right of the most significant decimal digit.

Caveats

• I chose * instead of . for the "binal" point so the two would not be confused here
• The term "binal" point is not a real word; I just use it because calling the delimiter between digit places representing values greater than or equal to 1 (1, 2, 4, 8, 16, ...) from those less than 1 (0.5, 0.25, 0.125, 0.0625, ...) a decimal point might be confusing

-

 

[drbitboy]

I expanded the previous post and fixed some typos.

 

[phuz]

Thanks, makes sense. I've seen similar behavior in javascript but I'm surprised I've never seen this in logix.