LarsJ Posted March 25, 2018 Share Posted March 25, 2018 (edited) FGAudf is a sub-UDF of FAMudf, Fast Array Management Functions UDF. FAMudf is needed to use FGAudf. FASudf, Fast Array Sorting and Management Functions UDF is needed to run examples. FGAudf that contains general array manipulation functions, is the first actual UDF that implements fast array management functions. This is the list of functions as shown in Includes\General.au3: expandcollapse popup; Functions\Initialization.au3 ; ---------------------------- ; FGA_GeneralInitDll Load compiled code from C++ DLL file. ; FGA_GeneralInitVbSrc Compile and load .NET code from source file. ; FGA_GeneralInitVbDll Load .NET code from .NET assembly DLL file. ; Functions\Conversions.au3 ; ---------------------------- ; FGA_Array1DTo2D Convert a 1D array to a 2D array with one column. ; FGA_Array2DTo1D Convert a 2D array with one column to a 1D array. ; FGA_IntStructToArray Convert a DllStruct of integers to a 1D array. ; Functions\ReadWrite.au3 ; ---------------------------- ; FGA_ArrayWrite Write a 1D/2D array to disk as binary data. Optimized C++ code. ; FGA_ArrayWriteAu3 Write a 1D/2D array to disk as binary data. Pure AutoIt code. ; FGA_ArrayRead Read a 1D/2D array from disk files. Optimized C++ code. ; FGA_ArrayReadAu3 Read a 1D/2D array from disk files. Pure AutoIt code. ; Functions\RowFuncs.au3 ; ---------------------------- ; FGA_RowsAdd Add one or more new rows to the end of an existing 1D or 2D array. ; FGA_RowsAllocate Allocate more space in a 1D or 2D array by inserting one or more empty rows ; at the specified (row) indices. ; FGA_RowsClear Clear (blank) one or more rows at the specified (row) indices in a 1D or 2D array. ; FGA_RowsDelete Delete one or more rows at the specified (row) indices in a 1D or 2D array. ; FGA_RowsExtract Extract one or more rows at the specified (row) indices in a 1D or 2D array, ; and return the extracted rows as a new 1D or 2D array. ; FGA_RowsInsert Insert one or more new rows at the specified (row) indices in a 1D or 2D array. ; FGA_RowsMove Move one or more rows from one to another position in a 1D or 2D array. ; Functions\ColumnFuncs.au3 ; ---------------------------- ; FGA_ColumnsAdd Add one or more new columns after last column in an existing 2D array. ; FGA_ColumnsAllocate Allocate more space in a 2D array by inserting one or more empty columns ; at the specified (column) indices. ; FGA_ColumnsClear Clear (blank) one or more columns at the specified (column) indices in a 2D array. ; FGA_ColumnsDelete Delete one or more columns at the specified (column) indices in a 2D array. ; FGA_ColumnsExtract Extract one or more columns at the specified (column) indices in a 2D array, ; and return the extracted columns as a new 2D array. ; FGA_ColumnsInsert Insert one or more new columns at the specified (column) indices in a 2D array. ; FGA_ColumnsMove Move one or more columns from one to another position in a 2D array. #include-once #include "Functions\Initialization.au3" #include "Functions\Conversions.au3" #include "Functions\ReadWrite.au3" #include "Functions\RowFuncs.au3" #include "Functions\ColumnFuncs.au3" The functions are optimized with small pieces of C++ code. The last group of column functions is optimized with VB code. The ReadWrite functions retains the data type of the array elements as shown by VarGetType. Examples Examples\3) ReadWrite\FGA_ArrayWrite.au3: #AutoIt3Wrapper_Au3Check_Parameters=-d -w- 1 -w 2 -w 3 -w 4 -w 5 -w 6 #include "..\..\..\FGAudf\Includes\General.au3" FGA_GeneralInitDll( "..\..\..\FGAudf\DLLFiles" ) #include "..\..\..\FASudf\Includes\SortingWr.au3" FAS_SortingInitVbSrc( "..\..\..\FASudf\Sources\Sorting.vb" ) #include "..\..\..\Display\Display.au3" Opt( "MustDeclareVars", 1 ) Example( 100000 ) Func Example( $iRows ) ; Create array Local $aCols = "sifdtr", $sSource = "TheQuickBrownFoxJumpsOverTheLazyDog" Local $aArray = FAS_Random2DArrayWr( $iRows, $aCols, $sSource ) Local $aDisplay = Display_GetRandom2DArrayFeatures( $aCols ) _ArrayDisplayEx( $aArray, "Test", $aDisplay[1], 0, $aDisplay[0] ) ; Write array FGA_ArrayWrite( "Test", $aArray, 10, 1 ) ; 10 = $iStrBufSize, 1 = $bOverWrite If @error Then Return ConsoleWrite( "FGA_ArrayWrite: @error = " & @error & @CRLF ) EndFunc Examples\3) ReadWrite\FGA_ArrayRead.au3: #AutoIt3Wrapper_Au3Check_Parameters=-d -w- 1 -w 2 -w 3 -w 4 -w 5 -w 6 #include "..\..\..\FGAudf\Includes\General.au3" FGA_GeneralInitDll( "..\..\..\FGAudf\DLLFiles" ) #include "..\..\..\Display\Display.au3" Opt( "MustDeclareVars", 1 ) Example() Func Example() ; Read array Local $aArray = FGA_ArrayRead( "Test" ) If @error Then Return ConsoleWrite( "FGA_ArrayRead: @error = " & @error & @CRLF ) Local $aDisplay = Display_GetRandom2DArrayFeatures( "sifdtr" ) _ArrayDisplayEx( $aArray, "Test", $aDisplay[1], 0, $aDisplay[0] ) EndFunc Runtimes FGA_IntStructToArray.au3: expandcollapse popupCode executed as 32 bit code Code executed as 64 bit code ============================ ============================ 100 integers 100 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 0.1341 AutoIt loop: 0.1089 Optimized code: 0.5447 Optimized code: 0.6026 500 integers 500 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 3.2448 AutoIt loop: 6.4405 Optimized code: 0.6029 Optimized code: 0.7472 1,000 integers 1,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 2.6441 AutoIt loop: 4.8112 Optimized code: 0.3219 Optimized code: 0.6430 2,000 integers 2,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 4.8996 AutoIt loop: 4.1164 Optimized code: 0.3784 Optimized code: 0.3294 5,000 integers 5,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 11.2276 AutoIt loop: 10.3660 Optimized code: 0.3848 Optimized code: 0.5430 10,000 integers 10,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 15.7094 AutoIt loop: 20.5262 Optimized code: 0.7261 Optimized code: 1.2802 20,000 integers 20,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 28.0012 AutoIt loop: 24.1433 Optimized code: 1.2046 Optimized code: 1.2193 50,000 integers 50,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 73.8528 AutoIt loop: 56.2135 Optimized code: 3.3196 Optimized code: 3.5119 100,000 integers 100,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 138.5275 AutoIt loop: 116.7239 Optimized code: 4.9702 Optimized code: 5.3537 250,000 integers 250,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 336.8909 AutoIt loop: 287.8890 Optimized code: 12.0601 Optimized code: 11.4880 500,000 integers 500,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 670.0658 AutoIt loop: 565.7541 Optimized code: 23.7252 Optimized code: 23.4895 750,000 integers 750,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 1008.0166 AutoIt loop: 861.2616 Optimized code: 42.6570 Optimized code: 36.7331 1,000,000 integers 1,000,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 1365.3218 AutoIt loop: 1132.6611 Optimized code: 62.7739 Optimized code: 49.5879 2,000,000 integers 2,000,000 integers Copy $tInts to $aArray Copy $tInts to $aArray -------------------------- -------------------------- AutoIt loop: 2692.6966 AutoIt loop: 2327.1560 Optimized code: 103.7193 Optimized code: 103.0408 FGA_ArrayReadWrite.au3: expandcollapse popupCode executed as 32 bit code Code executed as 64 bit code ============================ ============================ 100 rows, 6 columns 100 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 4.7610 Create array: 6.5227 Write array Write array _FileWriteFromArray: 6.4917 _FileWriteFromArray: 5.6351 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 29.8564 Pure AutoIt code: 28.5150 Optimized C++ code: 6.8341 Optimized C++ code: 5.6176 Read array Read array _FileReadToArray: 3.2264 _FileReadToArray: 3.2267 Compare arrays: 2.7499 Compare arrays: 2.3853 Compare errors: 400 Compare errors: 400 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 17.5141 Pure AutoIt code: 12.7714 Optimized C++ code: 0.8331 Optimized C++ code: 1.0852 Compare arrays: 2.2986 Compare arrays: 2.4859 Compare errors: 0 Compare errors: 0 500 rows, 6 columns 500 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 11.7793 Create array: 15.4549 Write array Write array _FileWriteFromArray: 27.1932 _FileWriteFromArray: 12.9956 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 39.4211 Pure AutoIt code: 39.7807 Optimized C++ code: 3.9448 Optimized C++ code: 4.1642 Read array Read array _FileReadToArray: 6.7294 _FileReadToArray: 7.5012 Compare arrays: 7.3084 Compare arrays: 6.0434 Compare errors: 2,000 Compare errors: 2,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 35.5247 Pure AutoIt code: 31.2209 Optimized C++ code: 1.0486 Optimized C++ code: 0.8660 Compare arrays: 7.6694 Compare arrays: 6.6172 Compare errors: 0 Compare errors: 0 1,000 rows, 6 columns 1,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 4.0845 Create array: 3.8559 Write array Write array _FileWriteFromArray: 10.9981 _FileWriteFromArray: 8.3587 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 55.5071 Pure AutoIt code: 52.7710 Optimized C++ code: 8.8200 Optimized C++ code: 5.6248 Read array Read array _FileReadToArray: 15.0617 _FileReadToArray: 10.9449 Compare arrays: 14.1940 Compare arrays: 29.8340 Compare errors: 4,000 Compare errors: 4,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 71.5238 Pure AutoIt code: 56.8183 Optimized C++ code: 2.0787 Optimized C++ code: 1.5273 Compare arrays: 16.5320 Compare arrays: 13.2324 Compare errors: 0 Compare errors: 0 2,000 rows, 6 columns 2,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 11.0394 Create array: 7.7617 Write array Write array _FileWriteFromArray: 30.5510 _FileWriteFromArray: 16.4486 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 82.6690 Pure AutoIt code: 113.8084 Optimized C++ code: 16.6955 Optimized C++ code: 14.0420 Read array Read array _FileReadToArray: 29.0516 _FileReadToArray: 25.0453 Compare arrays: 27.5484 Compare arrays: 30.7150 Compare errors: 8,000 Compare errors: 8,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 137.6881 Pure AutoIt code: 115.6213 Optimized C++ code: 3.4159 Optimized C++ code: 2.6823 Compare arrays: 29.7154 Compare arrays: 26.6976 Compare errors: 0 Compare errors: 0 5,000 rows, 6 columns 5,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 24.0729 Create array: 19.4468 Write array Write array _FileWriteFromArray: 50.8904 _FileWriteFromArray: 42.6960 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 212.7077 Pure AutoIt code: 266.9652 Optimized C++ code: 22.9378 Optimized C++ code: 20.4070 Read array Read array _FileReadToArray: 64.6467 _FileReadToArray: 54.5238 Compare arrays: 72.3796 Compare arrays: 62.9066 Compare errors: 20,000 Compare errors: 20,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 946.7204 Pure AutoIt code: 266.5835 Optimized C++ code: 8.5645 Optimized C++ code: 6.8610 Compare arrays: 76.3950 Compare arrays: 78.4471 Compare errors: 0 Compare errors: 0 10,000 rows, 6 columns 10,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 40.6110 Create array: 38.1977 Write array Write array _FileWriteFromArray: 99.6068 _FileWriteFromArray: 84.5119 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 406.3738 Pure AutoIt code: 525.4707 Optimized C++ code: 50.0717 Optimized C++ code: 42.6650 Read array Read array _FileReadToArray: 134.7227 _FileReadToArray: 114.4924 Compare arrays: 145.7609 Compare arrays: 122.7954 Compare errors: 40,000 Compare errors: 40,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 670.0151 Pure AutoIt code: 539.2475 Optimized C++ code: 17.6031 Optimized C++ code: 14.5589 Compare arrays: 153.2677 Compare arrays: 137.4507 Compare errors: 0 Compare errors: 0 20,000 rows, 6 columns 20,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 94.7963 Create array: 80.8854 Write array Write array _FileWriteFromArray: 202.6457 _FileWriteFromArray: 166.4799 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 794.6454 Pure AutoIt code: 1031.5769 Optimized C++ code: 96.9212 Optimized C++ code: 85.9356 Read array Read array _FileReadToArray: 265.0509 _FileReadToArray: 231.8126 Compare arrays: 291.5814 Compare arrays: 249.0380 Compare errors: 80,000 Compare errors: 80,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 1330.1447 Pure AutoIt code: 1073.0838 Optimized C++ code: 35.7195 Optimized C++ code: 30.1892 Compare arrays: 308.6617 Compare arrays: 264.7126 Compare errors: 0 Compare errors: 0 50,000 rows, 6 columns 50,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 251.3502 Create array: 241.0808 Write array Write array _FileWriteFromArray: 491.0023 _FileWriteFromArray: 397.4234 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 1991.3867 Pure AutoIt code: 2578.5686 Optimized C++ code: 232.9565 Optimized C++ code: 222.1867 Read array Read array _FileReadToArray: 648.6312 _FileReadToArray: 559.4462 Compare arrays: 699.0293 Compare arrays: 600.1184 Compare errors: 200,000 Compare errors: 200,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 3310.3602 Pure AutoIt code: 2679.1066 Optimized C++ code: 96.9672 Optimized C++ code: 80.6161 Compare arrays: 749.4446 Compare arrays: 668.2451 Compare errors: 0 Compare errors: 0 100,000 rows, 6 columns 100,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 483.9308 Create array: 396.8676 Write array Write array _FileWriteFromArray: 972.0078 _FileWriteFromArray: 794.8432 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 3965.4751 Pure AutoIt code: 5115.0041 Optimized C++ code: 462.2521 Optimized C++ code: 458.2148 Read array Read array _FileReadToArray: 1312.4095 _FileReadToArray: 1143.8532 Compare arrays: 1385.8987 Compare arrays: 1195.7637 Compare errors: 400,000 Compare errors: 400,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 6702.6805 Pure AutoIt code: 5420.8355 Optimized C++ code: 188.6855 Optimized C++ code: 163.0105 Compare arrays: 1492.8053 Compare arrays: 1328.1761 Compare errors: 0 Compare errors: 0 250,000 rows, 6 columns 250,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 992.7113 Create array: 1055.9462 Write array Write array _FileWriteFromArray: 2413.1376 _FileWriteFromArray: 1980.3471 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 10042.4486 Pure AutoIt code: 12951.3586 Optimized C++ code: 1256.0259 Optimized C++ code: 1210.3004 Read array Read array _FileReadToArray: 3296.9876 _FileReadToArray: 2837.6217 Compare arrays: 3459.7479 Compare arrays: 2990.7930 Compare errors: 1,000,000 Compare errors: 1,000,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 17687.6209 Pure AutoIt code: 14008.8673 Optimized C++ code: 469.3641 Optimized C++ code: 411.3772 Compare arrays: 3743.6195 Compare arrays: 3397.4697 Compare errors: 0 Compare errors: 0 500,000 rows, 6 columns 500,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 2277.7159 Create array: 2212.4201 Write array Write array _FileWriteFromArray: 5055.9309 _FileWriteFromArray: 4108.9125 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 20951.2827 Pure AutoIt code: 26377.1770 Optimized C++ code: 3186.9102 Optimized C++ code: 2735.0599 Read array Read array _FileReadToArray: 6610.8270 _FileReadToArray: 5615.6630 Compare arrays: 6887.5677 Compare arrays: 6045.2805 Compare errors: 2,000,000 Compare errors: 2,000,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 37089.5668 Pure AutoIt code: 30486.3904 Optimized C++ code: 931.2514 Optimized C++ code: 800.6115 Compare arrays: 7483.9338 Compare arrays: 6656.9123 Compare errors: 0 Compare errors: 0 750,000 rows, 6 columns 750,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 3254.4706 Create array: 2896.7223 Write array Write array _FileWriteFromArray: 7504.3306 _FileWriteFromArray: 6183.5898 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 31334.5265 Pure AutoIt code: 40094.0106 Optimized C++ code: 5033.8636 Optimized C++ code: 4593.2607 Read array Read array _FileReadToArray: 9898.2904 _FileReadToArray: 8410.9800 Compare arrays: 10398.6393 Compare arrays: 8950.6256 Compare errors: 3,000,000 Compare errors: 3,000,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 62823.5177 Pure AutoIt code: 51804.0693 Optimized C++ code: 1407.3050 Optimized C++ code: 1211.1620 Compare arrays: 11273.4910 Compare arrays: 10455.2244 Compare errors: 0 Compare errors: 0 1,000,000 rows, 6 columns 1,000,000 rows, 6 columns Write/read array to/from disk Write/read array to/from disk ----------------------------------------- ----------------------------------------- Create array: 4565.6595 Create array: 4257.2067 Write array Write array _FileWriteFromArray: 10393.2012 _FileWriteFromArray: 10472.7202 FGA_ArrayWrite FGA_ArrayWrite Pure AutoIt code: 42031.4300 Pure AutoIt code: 60211.2321 Optimized C++ code: 6624.7612 Optimized C++ code: 6436.1374 Read array Read array _FileReadToArray: 13164.0322 _FileReadToArray: 11280.1248 Compare arrays: 13851.5845 Compare arrays: 11944.7559 Compare errors: 4,000,000 Compare errors: 4,000,000 FGA_ArrayRead FGA_ArrayRead Pure AutoIt code: 87125.7872 Pure AutoIt code: 74393.8941 Optimized C++ code: 1884.6618 Optimized C++ code: 1610.3300 Compare arrays: 15078.5623 Compare arrays: 13405.1787 Compare errors: 0 Compare errors: 0 FGA_RowsInsert.au3: expandcollapse popupCode executed as 32 bit code Code executed as 64 bit code ============================ ============================ 100 rows, 6 columns 100 rows, 6 columns Insert one row at index 25 Insert one row at index 25 ------------------------------- ------------------------------- _ArrayInsert: 0.7159 _ArrayInsert: 0.6087 FGA_RowsInsert: 2.6422 FGA_RowsInsert: 3.0201 500 rows, 6 columns 500 rows, 6 columns Insert one row at index 125 Insert one row at index 125 ------------------------------- ------------------------------- _ArrayInsert: 6.7059 _ArrayInsert: 13.4817 FGA_RowsInsert: 3.5246 FGA_RowsInsert: 6.3012 1,000 rows, 6 columns 1,000 rows, 6 columns Insert one row at index 250 Insert one row at index 250 ------------------------------- ------------------------------- _ArrayInsert: 11.9255 _ArrayInsert: 10.2926 FGA_RowsInsert: 3.6268 FGA_RowsInsert: 5.0115 2,000 rows, 6 columns 2,000 rows, 6 columns Insert one row at index 500 Insert one row at index 500 ------------------------------- ------------------------------- _ArrayInsert: 14.6414 _ArrayInsert: 10.9816 FGA_RowsInsert: 8.5630 FGA_RowsInsert: 5.3611 5,000 rows, 6 columns 5,000 rows, 6 columns Insert one row at index 1,250 Insert one row at index 1,250 ------------------------------- ------------------------------- _ArrayInsert: 32.8972 _ArrayInsert: 28.7703 FGA_RowsInsert: 18.5431 FGA_RowsInsert: 14.3076 10,000 rows, 6 columns 10,000 rows, 6 columns Insert one row at index 2,500 Insert one row at index 2,500 ------------------------------- ------------------------------- _ArrayInsert: 65.9608 _ArrayInsert: 56.4863 FGA_RowsInsert: 37.4105 FGA_RowsInsert: 29.5826 20,000 rows, 6 columns 20,000 rows, 6 columns Insert one row at index 5,000 Insert one row at index 5,000 ------------------------------- ------------------------------- _ArrayInsert: 134.4746 _ArrayInsert: 115.3622 FGA_RowsInsert: 74.8673 FGA_RowsInsert: 55.7314 50,000 rows, 6 columns 50,000 rows, 6 columns Insert one row at index 12,500 Insert one row at index 12,500 ------------------------------- ------------------------------- _ArrayInsert: 328.4011 _ArrayInsert: 283.3676 FGA_RowsInsert: 187.4815 FGA_RowsInsert: 138.7198 100,000 rows, 6 columns 100,000 rows, 6 columns Insert one row at index 25,000 Insert one row at index 25,000 ------------------------------- ------------------------------- _ArrayInsert: 636.1870 _ArrayInsert: 560.0946 FGA_RowsInsert: 374.8816 FGA_RowsInsert: 278.7284 250,000 rows, 6 columns 250,000 rows, 6 columns Insert one row at index 62,500 Insert one row at index 62,500 ------------------------------- ------------------------------- _ArrayInsert: 1619.4215 _ArrayInsert: 1389.8712 FGA_RowsInsert: 926.4933 FGA_RowsInsert: 693.0172 500,000 rows, 6 columns 500,000 rows, 6 columns Insert one row at index 125,000 Insert one row at index 125,000 ------------------------------- ------------------------------- _ArrayInsert: 3218.2437 _ArrayInsert: 2823.3835 FGA_RowsInsert: 1857.2046 FGA_RowsInsert: 1411.2169 750,000 rows, 6 columns 750,000 rows, 6 columns Insert one row at index 187,500 Insert one row at index 187,500 ------------------------------- ------------------------------- _ArrayInsert: 4816.4340 _ArrayInsert: 4111.5426 FGA_RowsInsert: 2803.2133 FGA_RowsInsert: 2070.2246 1,000,000 rows, 6 columns 1,000,000 rows, 6 columns Insert one row at index 250,000 Insert one row at index 250,000 ------------------------------- ------------------------------- _ArrayInsert: 6483.6767 _ArrayInsert: 5592.3917 FGA_RowsInsert: 3754.6153 FGA_RowsInsert: 2764.8844 2,000,000 rows, 6 columns 2,000,000 rows, 6 columns Insert one row at index 500,000 Insert one row at index 500,000 ------------------------------- ------------------------------- _ArrayInsert: 12841.9769 _ArrayInsert: 11450.4454 FGA_RowsInsert: 7481.8667 FGA_RowsInsert: 5691.6660 FGA_ColumnsDelete.au3: expandcollapse popupCode executed as 32 bit code Code executed as 64 bit code ============================ ============================ 100 rows, 12 columns 100 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 6.4743 _ArrayColDelete: 8.3762 FGA_ColumnsDelete: 12.5685 FGA_ColumnsDelete: 18.2236 500 rows, 12 columns 500 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 12.6899 _ArrayColDelete: 10.8788 FGA_ColumnsDelete: 7.8767 FGA_ColumnsDelete: 6.9910 1,000 rows, 12 columns 1,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 15.0193 _ArrayColDelete: 11.2731 FGA_ColumnsDelete: 9.4110 FGA_ColumnsDelete: 6.1250 2,000 rows, 12 columns 2,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 27.5130 _ArrayColDelete: 23.8507 FGA_ColumnsDelete: 17.1692 FGA_ColumnsDelete: 14.2918 5,000 rows, 12 columns 5,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 67.8251 _ArrayColDelete: 56.3844 FGA_ColumnsDelete: 48.3484 FGA_ColumnsDelete: 35.0711 10,000 rows, 12 columns 10,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 135.4069 _ArrayColDelete: 112.9494 FGA_ColumnsDelete: 90.1592 FGA_ColumnsDelete: 83.2508 20,000 rows, 12 columns 20,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 263.8233 _ArrayColDelete: 221.9908 FGA_ColumnsDelete: 174.9637 FGA_ColumnsDelete: 163.3729 50,000 rows, 12 columns 50,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 678.3949 _ArrayColDelete: 577.2319 FGA_ColumnsDelete: 435.9140 FGA_ColumnsDelete: 341.0131 100,000 rows, 12 columns 100,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 1322.1306 _ArrayColDelete: 1105.7526 FGA_ColumnsDelete: 933.8638 FGA_ColumnsDelete: 715.9550 250,000 rows, 12 columns 250,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 3341.2247 _ArrayColDelete: 2839.3375 FGA_ColumnsDelete: 2389.5969 FGA_ColumnsDelete: 1830.7820 500,000 rows, 12 columns 500,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 6801.1776 _ArrayColDelete: 5568.0622 FGA_ColumnsDelete: 4960.1362 FGA_ColumnsDelete: 3837.6910 750,000 rows, 12 columns 750,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 9854.7078 _ArrayColDelete: 8530.9082 FGA_ColumnsDelete: 7366.5377 FGA_ColumnsDelete: 5761.3387 1,000,000 rows, 12 columns 1,000,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 13090.2626 _ArrayColDelete: 11169.6369 FGA_ColumnsDelete: 10153.7175 FGA_ColumnsDelete: 7796.6218 A look at the runtime measurements shows that FGA_IntStructToArray is 20 to 30 times faster than the corresponding conventional AutoIt code. That's great. Conversely, at best, FGA_RowsInsert and FGA_ColumnsDelete are only twice as fast as the corresponding _ArrayInsert and _ArrayColDelete. These results are probably representative for both groups of row and column functions. That's pretty bad results. It's compiled code. It should be faster. What's the reason for these results and what can be done about it? Runtime measurements are performed with arrays with 6/12 columns and a varying number of rows. The 6 columns are "sifdtr" columns. Ie. strings, integers, floats, dates, times and row/col columns. Dates and times are integers. The row/col column is strings. The 12 columns are "sifdtr" & "sifdtr" columns. If the string columns are replaced by floats columns so that the arrays becomes pure arrays of numbers, the results looks better. Now the compiled code is at best four times faster: FGA_RowsInsert-Numbers.au3: expandcollapse popupCode executed as 32 bit code Code executed as 64 bit code ============================ ============================ 100 rows, 6 columns 100 rows, 6 columns Insert one row at index 25 Insert one row at index 25 ------------------------------- ------------------------------- _ArrayInsert: 1.2827 _ArrayInsert: 2.7468 FGA_RowsInsert: 1.5758 FGA_RowsInsert: 2.3249 500 rows, 6 columns 500 rows, 6 columns Insert one row at index 125 Insert one row at index 125 ------------------------------- ------------------------------- _ArrayInsert: 15.0842 _ArrayInsert: 14.2354 FGA_RowsInsert: 3.9896 FGA_RowsInsert: 3.2785 1,000 rows, 6 columns 1,000 rows, 6 columns Insert one row at index 250 Insert one row at index 250 ------------------------------- ------------------------------- _ArrayInsert: 14.3390 _ArrayInsert: 13.2319 FGA_RowsInsert: 2.7953 FGA_RowsInsert: 2.1401 2,000 rows, 6 columns 2,000 rows, 6 columns Insert one row at index 500 Insert one row at index 500 ------------------------------- ------------------------------- _ArrayInsert: 13.5841 _ArrayInsert: 14.7590 FGA_RowsInsert: 3.1203 FGA_RowsInsert: 2.2363 5,000 rows, 6 columns 5,000 rows, 6 columns Insert one row at index 1,250 Insert one row at index 1,250 ------------------------------- ------------------------------- _ArrayInsert: 30.6353 _ArrayInsert: 26.0359 FGA_RowsInsert: 7.4227 FGA_RowsInsert: 5.7619 10,000 rows, 6 columns 10,000 rows, 6 columns Insert one row at index 2,500 Insert one row at index 2,500 ------------------------------- ------------------------------- _ArrayInsert: 62.1590 _ArrayInsert: 56.1475 FGA_RowsInsert: 15.6602 FGA_RowsInsert: 13.0668 20,000 rows, 6 columns 20,000 rows, 6 columns Insert one row at index 5,000 Insert one row at index 5,000 ------------------------------- ------------------------------- _ArrayInsert: 123.7628 _ArrayInsert: 110.9456 FGA_RowsInsert: 31.2173 FGA_RowsInsert: 23.0951 50,000 rows, 6 columns 50,000 rows, 6 columns Insert one row at index 12,500 Insert one row at index 12,500 ------------------------------- ------------------------------- _ArrayInsert: 311.6687 _ArrayInsert: 268.6157 FGA_RowsInsert: 74.9177 FGA_RowsInsert: 67.6460 100,000 rows, 6 columns 100,000 rows, 6 columns Insert one row at index 25,000 Insert one row at index 25,000 ------------------------------- ------------------------------- _ArrayInsert: 608.1551 _ArrayInsert: 517.9057 FGA_RowsInsert: 150.9420 FGA_RowsInsert: 120.8896 250,000 rows, 6 columns 250,000 rows, 6 columns Insert one row at index 62,500 Insert one row at index 62,500 ------------------------------- ------------------------------- _ArrayInsert: 1515.0866 _ArrayInsert: 1318.7368 FGA_RowsInsert: 380.4741 FGA_RowsInsert: 306.2980 500,000 rows, 6 columns 500,000 rows, 6 columns Insert one row at index 125,000 Insert one row at index 125,000 ------------------------------- ------------------------------- _ArrayInsert: 3055.9588 _ArrayInsert: 2592.6736 FGA_RowsInsert: 739.0425 FGA_RowsInsert: 603.7840 750,000 rows, 6 columns 750,000 rows, 6 columns Insert one row at index 187,500 Insert one row at index 187,500 ------------------------------- ------------------------------- _ArrayInsert: 4579.8788 _ArrayInsert: 3906.9467 FGA_RowsInsert: 1138.3392 FGA_RowsInsert: 912.8767 1,000,000 rows, 6 columns 1,000,000 rows, 6 columns Insert one row at index 250,000 Insert one row at index 250,000 ------------------------------- ------------------------------- _ArrayInsert: 6158.5664 _ArrayInsert: 5282.8649 FGA_RowsInsert: 1550.3611 FGA_RowsInsert: 1206.8620 2,000,000 rows, 6 columns 2,000,000 rows, 6 columns Insert one row at index 500,000 Insert one row at index 500,000 ------------------------------- ------------------------------- _ArrayInsert: 12131.6853 _ArrayInsert: 10471.6424 FGA_RowsInsert: 2976.3306 FGA_RowsInsert: 2425.4074 FGA_ColumnsDelete-Numbers.au3: expandcollapse popupCode executed as 32 bit code Code executed as 64 bit code ============================ ============================ 100 rows, 12 columns 100 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 6.5523 _ArrayColDelete: 7.3856 FGA_ColumnsDelete: 11.4315 FGA_ColumnsDelete: 18.3203 500 rows, 12 columns 500 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 25.8863 _ArrayColDelete: 14.6102 FGA_ColumnsDelete: 4.2210 FGA_ColumnsDelete: 4.1808 1,000 rows, 12 columns 1,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 18.8860 _ArrayColDelete: 14.3506 FGA_ColumnsDelete: 4.0459 FGA_ColumnsDelete: 3.8863 2,000 rows, 12 columns 2,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 26.4980 _ArrayColDelete: 21.2871 FGA_ColumnsDelete: 8.1876 FGA_ColumnsDelete: 6.5035 5,000 rows, 12 columns 5,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 62.5781 _ArrayColDelete: 52.2130 FGA_ColumnsDelete: 23.2558 FGA_ColumnsDelete: 20.7574 10,000 rows, 12 columns 10,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 123.1954 _ArrayColDelete: 111.9750 FGA_ColumnsDelete: 41.5448 FGA_ColumnsDelete: 38.3079 20,000 rows, 12 columns 20,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 248.6938 _ArrayColDelete: 210.4331 FGA_ColumnsDelete: 83.8376 FGA_ColumnsDelete: 70.0086 50,000 rows, 12 columns 50,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 616.1587 _ArrayColDelete: 541.6447 FGA_ColumnsDelete: 221.0789 FGA_ColumnsDelete: 198.3267 100,000 rows, 12 columns 100,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 1235.3286 _ArrayColDelete: 1048.3622 FGA_ColumnsDelete: 536.6809 FGA_ColumnsDelete: 386.0548 250,000 rows, 12 columns 250,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 3031.3522 _ArrayColDelete: 2622.3827 FGA_ColumnsDelete: 1346.9687 FGA_ColumnsDelete: 896.6716 500,000 rows, 12 columns 500,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 6039.6947 _ArrayColDelete: 5288.0697 FGA_ColumnsDelete: 2659.5023 FGA_ColumnsDelete: 1928.5887 750,000 rows, 12 columns 750,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 9096.9044 _ArrayColDelete: 7954.2082 FGA_ColumnsDelete: 4000.9829 FGA_ColumnsDelete: 2900.8610 1,000,000 rows, 12 columns 1,000,000 rows, 12 columns Delete column at index 1 Delete column at index 1 ------------------------------- ------------------------------- _ArrayColDelete: 12468.9790 _ArrayColDelete: 10382.1064 FGA_ColumnsDelete: 5308.2076 FGA_ColumnsDelete: 4039.3171 Maybe two sets of functions should have been used for arrays with and without strings? The column functions, which are the slowest, are VB code. Maybe it should have been C++ code? What's nextWhy are the row and column functions not faster? What can be done about it? FGAudf.7zFAMudf.7z must be downloaded and unzipped to use FGAudf. FGAudf.7z must be unzipped in the same folder structure as FAMudf.7z. See Fast Array Management Functions UDF for details. FASudf.7z is needed to run examples. Start running some of the examples in FAMproj\FGAudf\Examples\ (with F5 in SciTE) and look at the code in the examples. You need AutoIt 3.3.10 or later. Tested on Windows 10 and Windows 7. Comments are welcome. Let me know if there are any issues. FGAudf.7z Edited April 27, 2018 by LarsJ Minor updates Trong 1 Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
mLipok Posted March 25, 2018 Share Posted March 25, 2018 (edited) It looks promising. I will look closer when I get some spare time. Thanks Edited March 25, 2018 by mLipok LarsJ 1 Signature beginning:* Please remember: "AutoIt"..... * Wondering who uses AutoIt and what it can be used for ? * Forum Rules ** ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Code * for other useful stuff click the following button: Spoiler Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API * ErrorLog.au3 UDF - A logging Library * Include Dependency Tree (Tool for analyzing script relations) * Show_Macro_Values.au3 * My contribution to others projects or UDF based on others projects: * _sql.au3 UDF * POP3.au3 UDF * RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF * SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane * Useful links: * Forum Rules * Forum etiquette * Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * Wiki: * Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX IE Related: * How to use IE.au3 UDF with AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskScheduler * IE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) * PDF Related: * How to get reference to PDF object embeded in IE * IE on Windows 11 * I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions * EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *I also encourage you to check awesome @trancexx code: * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuff * OnHungApp handler * Avoid "AutoIt Error" message box in unknown errors * HTML editor * winhttp.au3 related : * https://www.autoitscript.com/forum/topic/206771-winhttpau3-download-problem-youre-speaking-plain-http-to-an-ssl-enabled-server-port/ "Homo sum; humani nil a me alienum puto" - Publius Terentius Afer"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming" , be and \\//_. Anticipating Errors : "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty." Signature last update: 2023-04-24 Link to comment Share on other sites More sharing options...
LarsJ Posted March 26, 2018 Author Share Posted March 26, 2018 (edited) mLipok, Thank you. Let me know if you run into any issues. In another thread there is a list of optimization options: Use pure AutoIt code for small arrays Use compiled code for medium arrays Use multi-threaded code for large arrays Consider data (strings take longer) Limit the number of array transfers In this thread, only the bold-typed optimization is implemented. So do not be too disappointed with the poor performance. It'll be improved in new versions. To use the functions download the 3 7z-files (FAMudf.7z, FASudf.7z, FGAudf.7z (above)), move the files eg. to C:\Windows\Temp and unzip the files. This will create a FAMproj folder and 4 subfolders: FAMproj\ Display\ FAMudf\ FASudf\ FGAudf\ Now you can test the examples in FASudf\Examples and FGAudf\Examples. Edited March 26, 2018 by LarsJ Controls, File Explorer, ROT objects, UI Automation, Windows Message MonitorCompiled code: Accessing AutoIt variables, DotNet.au3 UDF, Using C# and VB codeShell menus: The Context menu, The Favorites menu. Shell related: Control Panel, System Image ListsGraphics related: Rubik's Cube, OpenGL without external libraries, Navigating in an image, Non-rectangular selectionsListView controls: Colors and fonts, Multi-line header, Multi-line items, Checkboxes and icons, Incremental searchListView controls: Virtual ListViews, Editing cells, Data display functions Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now