Having a Problem with some calculations (inaccuracies)

[nullschritt]

Binary FP is inherently prone to approximation and truncation to values that can actually be represented internally.

 

What you say about BigNum surprises me. For instance you can get pretty close to 15 when doing this:

#include "..\include\bignum.au3"

Local $sqrt3 = _BigNum_SQRT("3", 140)
Local $res = _BigNum_Mul($sqrt3, _BigNum_Mul($sqrt3, "5"))
ConsoleWrite($res & @LF)

You can still increase required accuracy if you need more than 140 decimal digits...

 

Can you give an example where BigNum fails to give a correct answer?

The example script I posted does exactly that. It uses the bignum UDF for ALL it's math operations. It takes an original set of numbers(a matrix) and multiplies it by several other sets of numbers(also matricies).

Then it does the exact reverse of it's original operation, using bignum it divides the sets of numbers by the inverses of the original sets of data, which SHOULD return the original set of numbers.

 

If you run the example script and change $keycount = 7 to $keycount = 6 you will see the data returned at the end is the same as the original data. (because it only multiplies the data 6 times over) But when changed to 7 or 8 the end result is way off from the original data we started out with. Because the numbers get bigger and bigger.

 

If you add _ArrayDisplay($datax) after the line $datax = $hologram you can see the actual numbers output at the end of the multiplication process from the bignum udf.

See an example of the result of the multiplication below you will see that they truancate after 17 digits.

Edited March 9, 2015 by nullschritt

[jchd]

Once again, can you post a short self-contained example of failure.

And again I don't experience the issue you say you have:

#include "..\include\bignum.au3"

Local $sqrt2 = _BigNum_SQRT("2", 140)
Local $sqrt3 = _BigNum_SQRT("3", 140)
Local $sqrt5 = _BigNum_SQRT("5", 140)
Local $res = _BigNum_Mul(_BigNum_Mul(_BigNum_Mul($sqrt2, "5"), $sqrt3), $sqrt5)
ConsoleWrite($res & @LF)
$res = _BigNum_Div(_BigNum_Div(_BigNum_Div($res, $sqrt3), $sqrt5), $sqrt2)
ConsoleWrite($res & @LF)

[nullschritt]

Truancated Data.

[Geir1983]

Not sure if its relevant for matrixes but maybe this can help?

[nullschritt]

Once again, can you post a short self-contained example of failure.

 

And again I don't experience the issue you say you have:

#include "..\include\bignum.au3"

Local $sqrt2 = _BigNum_SQRT("2", 140)
Local $sqrt3 = _BigNum_SQRT("3", 140)
Local $sqrt5 = _BigNum_SQRT("5", 140)
Local $res = _BigNum_Mul(_BigNum_Mul(_BigNum_Mul($sqrt2, "5"), $sqrt3), $sqrt5)
ConsoleWrite($res & @LF)
$res = _BigNum_Div(_BigNum_Div(_BigNum_Div($res, $sqrt3), $sqrt5), $sqrt2)
ConsoleWrite($res & @LF)

 

Here you go, a perfect example of the problem I'm encountering in an extremely simplified format.

#include "bignum.au3"
 
dim $numbers[51]
for $i=0 to 50
$numbers[$i] = Random(111111111, 999999999, 1)
Next
 
$output= 2 ;original number
 
for $i=0 to 50
$output = _BigNum_Parse($output*$numbers[$i])
Next
 
ConsoleWrite($output&@CRLF) ;this shouldn't be truancated
 
for $i=0 to 50
$output = _BigNum_Parse($output/$numbers[50-$i])
Next
 
ConsoleWrite($output&@CRLF) ;this should be 2

Edited March 9, 2015 by nullschritt

[nullschritt]

I've also updated the original post with this code instead of the overly-complicated one. As it produces the same error and seems to be easier for people to understand.

[jchd]

You're using BigNum wrong:

#include "bignum.au3"

Local $numbers[51]
for $i=0 to 50
$numbers[$i] = Random(111111111, 999999999, 1)
Next

$output= 2 ;original number

for $i=0 to 50
$output = _BigNum_Mul($output, $numbers[$i])
Next

ConsoleWrite($output&@CRLF) ;this shouldn't be truancated

for $i=0 to 50
$output = _BigNum_Div($output, $numbers[50-$i])
Next

ConsoleWrite($output&@CRLF) ;this should be 2   <-- it is!

Do you now see why?

[nullschritt]

Not sure if its relevant for matrixes but maybe this can help?

 Anything Math related is matrix related. matrix math is just doing math operations on several numbers in a certain order ( :

I took a quick look at it but I'll look more when I get home. If it can produce accurate representations of floats, and if that is indeed the problem, then it is a possible solution.

[nullschritt]

You're using BigNum wrong:

#include "bignum.au3"

Local $numbers[51]
for $i=0 to 50
$numbers[$i] = Random(111111111, 999999999, 1)
Next

$output= 2 ;original number

for $i=0 to 50
$output = _BigNum_Mul($output, $numbers[$i])
Next

ConsoleWrite($output&@CRLF) ;this shouldn't be truancated

for $i=0 to 50
$output = _BigNum_Div($output, $numbers[50-$i])
Next

ConsoleWrite($output&@CRLF) ;this should be 2   <-- it is!

Do you now see why?

Okay, yes, I've corrected my code, but sadly it's just too slow for practical use in matrix multiplication, it takes, several minutes for an operation that would otherwise take a fraction of a second, thanks for your pointer, but are there possibly any faster alternative?

It seems bignum is a little heavy for what I'm doing, I don't need complete accuracy, I just need accuracy to say about 35 digits, and while the bignum divide function has an option to limit the decimal places, the bignum multiply, subtract, and add functions do not.

Edited March 9, 2015 by nullschritt

[czardas]

I ran a comparison with Fraction vs Doubles:

Accuracy with the current implementation of the Fraction UDF can be improved after recent discoveries I have made, including the functions I posted here. It may at that point serve as a viable alternative to using division with extremely high accuracy and hopefully not be too slow. Numbers cannot become huge, but accuracy is guaranteed within the ranges specified. I thought of incorporating the bignum UDF, but I'm still not sure about doing that - maybe for extreme cases.

The results of the test in the link above are as follows:

0.333333333327651 <== Corruption after 29 iterations using floats
0.333333333333333 <== Using Fraction for the same process
0 Approximations

The value started out at 1/3.

Edited March 9, 2015 by czardas

[nullschritt]

I ran a comparison with Fraction vs Doubles:

Accuracy with the current implementation of the Fraction UDF can be improved after recent discoveries I have made, including the functions I posted here. It may at that point serve as a viable alternative to using division with extremely high accuracy and hopefully not be too slow. Numbers cannot become huge, but accuracy is guaranteed within the ranges specified. I thought of incorporating the bignum UDF, but I'm still not sure about doing that - maybe for extreme cases.

See my post on that thread, I had a question I posted there about possibly using the fractions udf.

[jchd]

Correct: BigNum is slow as snail because it can only use strings of digits and awfully pedestrian base-10 operations, all in slow scripting language.

But at least it gives you a correct result (lest undisclosed bugs).

For speed you'd have better time using one of the many high-performance arbitrary-precision libraries available. There Google is your friend.

[nullschritt]

Thanks for all the advice. I don't think I'll be using an external library because I have never even used dllcall() and wouldn't know where to begin writing a wrapper, I just thought maybe someone had written a wrapper for a math library, but from what I can tell the only UDF for this sort of thing in autoit is the bignum UDF.

However, it's possible that at least part of the problem lies in autoit not properly representing floats, because the algorithm I'm working on works 50% of the time, while the other 50% of the time the data is not accurate enough to retreive, so this is leading me to believe maybe the problem lies in the handling of float values, so I'm attempting to work with the fractions UDF to preform all my math in the form of fractions, but I'm having a problem with storing arrays in arrays, specifically when I try to access a 2 dimensional array containing another array.

[czardas]

I'm not sure if Fraction UDF is suitable or not: it depends on how large your numbers grow. I would not store the arrays inside another array. Instead you should probably use a delimiter and store both dividend and divisor in one element - something like this:

#include <Array.au3>
#include 'Fraction.au3'

; Partly populated main array
Local $aVariants[2][2] = [['41|152','7|3'],['1567|100','']]
_ArrayDisplay($aVariants, "Partly populated")

; Add a value
Local $aWrite = _Fraction(21, 12)
$aVariants[1][1] = $aWrite[0] & '|' & $aWrite[1]
_ArrayDisplay($aVariants, "New Fraction Added")

; Read a value
Local $aRead = StringSplit($aVariants[0][0], '|', 2) ; Read data from the first cell

; To use other functions in the Fraction UDF, fractions should be formatted as two doubles
$aRead[0] = Number($aRead[0], 3) ; Convert both strings to doubles - very important
$aRead[1] = Number($aRead[1], 3) ; This is done automatically if using _Fraction()
_ArrayDisplay($aRead, "Fraction read")

MsgBox(0, "Convert to float", $aRead[0] / $aRead[1])

;

Or even better, use a 3D array: in which case you don't need strings or StringSplit() and can let _Fraction() handle all conversions.

#include 'Fraction.au3'

Global $g_iDividend = 0, $g_iDivisor = 1

; Partly populated main array - Dividends table , Divisors table
Local $aVariants[2][2][2] = [[[41,7],[1567,'']] , [[152,3],[100,'']]]

; Add a value
Local $aWrite = _Fraction(21, 12) ; Values will be correctly formatted by _Fraction()
$aVariants[$g_iDividend][1][1] = $aWrite[$g_iDividend] ; Add to the 3D array
$aVariants [$g_iDivisor][1][1] = $aWrite[$g_iDivisor]  ; As above

Local $aRead ; Read all values
For $i = 0 To 1
    For $j = 0 To 1
        ; You don't necessarily need to call fraction here IF THE VALUES ARE ALREADY FORMATTED as integer doubles.
        ; You could just read the values directly to a 2D array, ready to pass to another function from the UDF.
        ; _Fraction() is called here for illustration purposes.
        $aRead = _Fraction($aVariants[$g_iDividend][$i][$j], $aVariants[$g_iDivisor][$i][$j]) ; Call _Fraction()
        ConsoleWrite($aRead[$g_iDividend] & "/" & $aRead[$g_iDivisor] & @CRLF)
    Next
Next

;

My suggested 3D array layout is $aName [dividend = 0, or divisor = 1]  [row] [column]

The 3D array consists of two tables, one above the other. One table contains the dividends and the other stores the divisors.

Edited March 9, 2015 by czardas

[nullschritt]

 

I'm not sure if Fraction UDF is suitable or not: it depends on how large your numbers grow. I would not store the arrays inside another array. Instead you should probably use a delimiter and store both dividend and divisor in one element - something like this:

#include <Array.au3>
#include 'Fraction.au3'

; Partly populated main array
Local $aVariants[2][2] = [['41|152','7|3'],['1567|100','']]
_ArrayDisplay($aVariants, "Partly populated")

; Add a value
Local $aWrite = _Fraction(21, 12)
$aVariants[1][1] = $aWrite[0] & '|' & $aWrite[1]
_ArrayDisplay($aVariants, "New Fraction Added")

; Read a value
Local $aRead = StringSplit($aVariants[0][0], '|', 2) ; Read data from the first cell

; To use other functions in the Fraction UDF, fractions should be formatted as two doubles
$aRead[0] = Number($aRead[0], 3) ; Convert both strings to doubles - very important
$aRead[1] = Number($aRead[1], 3) ; This is done automatically if using _Fraction()
_ArrayDisplay($aRead, "Fraction read")

MsgBox(0, "Convert to float", $aRead[0] / $aRead[1])

;

Or even better, use a 3D array: in which case you don't need strings or StringSplit() and can let _Fraction() handle all conversions.

#include 'Fraction.au3'

Global $g__iDividend = 0, $g__iDivisor = 1

; Partly populated main array
Local $aVariants[2][2][2] = [[[41,7],[1567,'']],[[152,3],[100,'']]]

; Add a value
Local $aWrite = _Fraction(21, 12)
$aVariants[$g__iDividend][1][1] = $aWrite[$g__iDividend]
$aVariants [$g__iDivisor][1][1] = $aWrite[$g__iDivisor]

Local $aRead ; Read all values
For $i = 0 To 1
    For $j = 0 To 1
        $aRead = _Fraction($aVariants[$g__iDividend][$i][$j], $aVariants[$g__iDivisor][$i][$j])
        ConsoleWrite($aRead[$g__iDividend] & "/" & $aRead[$g__iDivisor] & @CRLF)
    Next
Next

With the function I'm using this with, all the values HAVE to be stored in a 2D array, its for matrix multiplication, and a matrix requires a 2d array to hold the rows and columns. In my code each row and column of the 2D array should contain an array containing the dividend and divisor.

I'm thinking I may have to completley modify your UDF to work on strings instead of arrays. Because as I show in my other thread where I posted an example of the problem I am having, you will see that if I try to access the values with _arraydisplay() I can see your function indeed returned an array with 2 values, but then when I try to directly access the array returned from your function, I get a dimension range exceeded error, even though I can visually see that the values exist in _arraydisplay().

[jchd]

AiutoIt uses standard 64-bit IEEE doubles, such as engraved in stock hardware.

[czardas]

I get a dimension range exceeded error, even though I can visually see that the values exist in _arraydisplay().

 

Not with the examples that I posted above: you perhaps should use Ubound(). The Fraction UDF is designed to work with numbers, not with strings. I added some more comments to the examples. Read them again. Two arrays can also be used: a dividend array and a divisor array. That's 4 options mentioned so far. I'm still not sure if you will remain within range though. It depends on how many multiplications you do.

Edited March 9, 2015 by czardas