multiplication wrong answer

The precision of floating point numbers is finite.

The problem is with how many decimal digits of precision do you want to be represented, a 32bit REAL can only offer approximately 7 digits, if it's represented with more than 7 the rest is garbage.

Instead a 64bit REAL offers up to 14.
 
Isn't this what the Basic card was for?

The real number in a SLC was limited to 7 digits, if more were entered it rounds the 7th digit, adding an EE if needed.

In a ControlLogix it is limited to 9 digits (on my computer)
 
32 bit floats are so last millennium.

On the topic of 64 bit floats.. excel uses a modified ieee 64 bit float which has the same precision regardless of number, whereas 64 bit reals conforming to ieee standard have MORE precision than excel, albeit a variable precision. So if you do a 64 bit math in excel and a 64 bit math on a PLC, you may have slightly different answers.
 
The precision of floating point numbers is finite.

Exactly. You've got a limited number of bits available for the mantissa, so expect a limited answer. You don't get Pi to a quadrillion places on your '80's era solar-powered calculator either. Everything has its limits.

Although I would have expected it to get the whole number portion of the answer correct. Call me picky...

Searching for "Arbitrary Precision" will lead to any number of calculators that will pump out various lengths of precision. Some of them with *less* precision (go figure), and some even incorrect.

This is what I get over at: https://www.mathsisfun.com/calculator-precision.html

221383538.5323974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974358974359


As a side note, do any other old-timers here remember the Pi wars from the early days of hand-held calculators? Pi was calculated or stored differently depending on the TI and HP brands.

And does anyone know the fraction that was used to approximate Pi?
 
But does it do 64-bit floating-point math ?

On a Micro820, all are LREAL variables, just the same result than EXCEL

10mii2s.png
 
I disagree, micro820 uses ieee 64 bit floats, where excel uses a modified version. I would describe the results as similar but not the same.

Then the modified version of excel generates exactly the same 15 most significant decimal digits ...
 

Similar Topics

I've got a Q68DAIN (Mitsubishi Q series analogue output module). On the HMI, Basically on the screen, the user types in a value as a percentage...
Replies
11
Views
2,199
In my last year at Glasgow Uni in 1966 I indulged myself by taking the newly introduced course in "Computing". This allowed us a few seconds...
Replies
4
Views
2,894
We have a quad monitor setup with FT SE and we are utilizing a header screen at the top of every display. when we open a new page we abort the...
Replies
0
Views
77
Hi all, I am having an issue with FT View Studio ME when I try to open the propoerties of any object, the size of it is very big and doesn't let...
Replies
0
Views
96
ewrong aka the censored brand of remote access this thing is kicking my butt and im not sure if i am just irritated enough that i am missing...
Replies
5
Views
592
Back
Top Bottom