Jump to content

Recommended Posts

Posted (edited)

Certainly better performance for complex JSON documents and therefore great to have another tool in the toolbox of JSON problem solvers. 👍

I took the case of the individual array extraction from the meta.json to carry out the performance comparison mentioned above (test script in the appendix). 

This was my result:

 image.png.719f72e71567e20b707f460aa63a46b1.png

 

 

 

 

speed comparison.zipFetching info...

Edited by AspirinJunkie
improved RegEx-Pattern and force 64 bit mode
Posted (edited)

Here are the results with jq added to the mix.  This was run on a 10 year old Dell XPS 8700 running Windows 7 64bit.  That explains the slower speeds overall.

image.png.c3520aafda3702542fd08c8e1540f8a3.png

speed comparison.zipFetching info...

Edited by TheXman
Updated the results using Aprinjunkie's updated script
Posted

I am very excited about the jq UDF now.  It looks so fast.  Thanks @TheXman.  I'll definitely check it out.

Cheers, Sean.

See my other UDFs:

  Reveal hidden contents
Chrome UDF - Automate Chrome | SAP UDF - Automate SAP | Java UDF - Automate Java Applications & Applets | Tesseract (OCR) UDF - Capture text from applications, controls and the desktop | Textract (OCR) UDF - Capture text from applications and controls | FileSystemMonitor UDF - File, Folder, Drive and Shell Monitoring | VLC (Media Player) UDF - Creating and controlling a VLC control in AutoIT | Google Maps UDF - Creating and controlling Google Maps (inc. GE) in AutoIT | SAPIListBox (Speech Recognition) UDF - Speech Recognition via the Microsoft Speech (SAPI) ListBox | eBay UDF - Automate eBay using the eBay API | ChildProc (Parallel Processing) UDF - Parallel processing functions for AutoIT | HyperCam (Screen Recording) UDF - Automate the HyperCam screen recorder | Twitter UDF - Automate Twitter using OAuth and the Twitter API | cURL UDF - a UDF for transferring data with URL syntax

See my other Tools:

  Reveal hidden contents
Rapid Menu Writer - Add menus to DVDs in seconds | TV Player - Automates the process of playing videos on an external TV / Monitor | Rapid Video Converter - A tool for resizing and reformatting videos | [topic130531]Rapid DVD Creator - Convert videos to DVD fast and for free | ZapPF - A tool for killing processes and recycling files | Sean's eBay Bargain Hunter - Find last minute bargains in eBay using AutoIT | Sean's GUI Inspector - A scripting tool for querying GUIs | TransLink Journey Planner with maps - Incorporating Google Maps into an Australian Journey Planner | Automate Qt and QWidgets | Brisbane City Council Event Viewer - See what's going on in Brisbane, Australia
Posted

jq is very fast and it is very powerful too.  It can do much more than parse JSON.  jq is a JSON processor.  Meaning, a lot of the post-processing of JSON data that you would normally do using AutoIt functions can be accomplished using jq, in a single jq filter, and be done much quicker too.

The jq UDF has a few examples included in the zip file and jqPlayground has many more (20+) example jq filters to help you get started learning what jq can do.  :)  jqPlayground can be very helpful as a learning tool because it is interactive and allows you to save and restore sessions.

Posted (edited)

Have you changed anything in the script?
Maybe so that $sScript is suddenly empty or something?
Have you checked the result to see if this is really what the code produces? (an _ArrayDisplay() on $aActions for example)

Or may I guess?: You are running your script as 32-bit instead of 64-bit?
Then the results would be plausible, as nothing is being processed (you should be able to tell from the result).

The results don't look very plausible (and I can't reproduce them either).
The For-Loop alone to get the AutoIt array from the JsonC results is just 40% faster than the StringRegExp solution in my tests - there is not even a single Json-C code involved. This clashes with your results that the whole call should be ~230x faster than the StringRegExp solution.

There was a small indexing error in the script, which I fixed. At the same time, I also added jq to my results above.

 

Edited by AspirinJunkie
Posted (edited)
  On 4/16/2025 at 4:41 AM, AspirinJunkie said:

Or may I guess?: You are running your script as 32-bit instead of 64-bit?

Expand  

image.png.49a6ad7cf1cb70dc380baf4aa7c551aa.png

...had no clue that had to be x64, or rather that I'd have to run the script x64 instead of click-click :( 


do add something along the lines of:

#AutoIt3Wrapper_UseX64=y
If Not @AutoItX64 Then Exit MsgBox(0, @ScriptName, "Please run as x64", 10)

for those "click-click-ers" like myself :)

Edited by argumentum

Follow the link to my code contribution ( and other things too ).
FAQ - Please Read Before Posting.
autoit_scripter_blue_userbar.png

Posted (edited)
  On 4/16/2025 at 4:41 AM, AspirinJunkie said:

I also added jq to my results above.

Expand  

Your updated zip file includes jq code in the SpeedComparison.au3 script, but it does not include the required jq.udf and jq-win64.exe files.

Edited by TheXman
Posted

I added your UDF to the wiki :) 

My UDFs and Tutorials:

  Reveal hidden contents

 

Posted (edited)
;~ _ArrayDisplay($a_Results, "Results over " & $i_Runs & " loops", "", 16 + 64, Default, "name|time[ms]|factor|Std. Dev|Std. Err.|min|max|range")
;~ #include <SQLite.au3>
ConsoleWriteResult()
Func ConsoleWriteResult()
    ConsoleWrite(@CRLF & RegRead("HKLM\HARDWARE\DESCRIPTION\System\CentralProcessor\0", "ProcessorNameString") & _
            " | Runs: " & $i_Runs & " | WinVer: " & FileGetVersion("WinVer.exe") & @CRLF & @CRLF)
    Local $aHead = StringSplit("name|time[ms]|factor|Std. Dev|Std. Err.|min|max|range", "|", 2) ; header from _ArrayDisplay()
    ReDim $a_Results[UBound($a_Results) + 2][UBound($a_Results, 2)]
    For $n = UBound($a_Results) - 3 To 0 Step -1
        For $m = 0 To UBound($a_Results, 2) - 1
            $a_Results[$n + 2][$m] = $a_Results[$n][$m]
        Next
    Next
    For $m = 0 To UBound($a_Results, 2) - 1
        $a_Results[0][$m] = $aHead[$m]
        $a_Results[1][$m] = "------"
    Next
    _SQLite_Display2DResult($a_Results, 0, False, "|", "|" & @CRLF)
EndFunc   ;==>ConsoleWriteResult
Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz | Runs: 100 | WinVer: 10.0.22621.1

 name                 | time[ms] | factor | Std. Dev | Std. Err. | min     | max     | range  |
 ------               | ------   | ------ | ------   | ------    | ------  | ------  | ------ |
 StringRegExp only    | 2.807    | 1      | 0.225    | 0.022     | 2.638   | 3.65    | 1.012  |
 JsonC-UDF            | 90.724   | 32.32  | 2.622    | 0.262     | 87.264  | 104.752 | 17.487 |
 jq UDF               | 163.349  | 58.19  | 8.394    | 0.839     | 135.862 | 182.644 | 46.783 |
 pure AutoIt JSON-UDF | 171.625  | 61.14  | 3.261    | 0.326     | 164.824 | 184.02  | 19.196 |
 JSMN-based JSON-UDF  | 190.401  | 67.83  | 3.561    | 0.356     | 184.436 | 203.804 | 19.369 |

Since sharing pictures ( due to forum user max file(s) space ) gets expensive, sharing the data as text is better for us.
This is running @TheXman's test file.

Am still wondering on why Win7 is so different from Win11/10 results. Or is it HDD vs NVMe. Or ... 🤔

Edit: another PC:

12th Gen Intel(R) Core(TM) i9-12900H | Runs: 100 | WinVer: 24H2 (26100)

 name                 | time[ms] | factor | Std. Dev | Std. Err. | min    | max     | range  |
 ------               | ------   | ------ | ------   | ------    | ------ | ------  | ------ |
 StringRegExp only    | 1.691    | 1      | 0.351    | 0.035     | 1.304  | 3.167   | 1.863  |
 jq UDF               | 32.933   | 19.48  | 2.929    | 0.293     | 29.308 | 43.169  | 13.861 |
 JsonC-UDF            | 51.086   | 30.21  | 3.205    | 0.321     | 45.625 | 63.46   | 17.835 |
 pure AutoIt JSON-UDF | 97.916   | 57.9   | 5.685    | 0.569     | 86.362 | 113.467 | 27.105 |
 JSMN-based JSON-UDF  | 108.248  | 64.01  | 5.512    | 0.551     | 99.029 | 130.864 | 31.835 |
Intel(R) Pentium(R) Silver N6000 @ 1.10GHz | Runs: 100 | WinVer: 10.0.14393.0

 name                 | time[ms] | factor | Std. Dev | Std. Err. | min     | max     | range  |
 ------               | ------   | ------ | ------   | ------    | ------  | ------  | ------ |
 StringRegExp only    | 3.255    | 1      | 0.198    | 0.02      | 3.076   | 4.043   | 0.967  |
 jq UDF               | 50.396   | 15.48  | 1.756    | 0.176     | 47.438  | 59.49   | 12.052 |
 JsonC-UDF            | 112.045  | 34.42  | 8.776    | 0.878     | 106.967 | 181.859 | 74.892 |
 pure AutoIt JSON-UDF | 199.033  | 61.15  | 3.439    | 0.344     | 194.587 | 211.019 | 16.432 |
 JSMN-based JSON-UDF  | 222.099  | 68.23  | 3.823    | 0.382     | 216.491 | 239.937 | 23.446 |
Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz | Runs: 100 | WinVer: 10.0.26100.1

 name                 | time[ms] | factor | Std. Dev | Std. Err. | min     | max     | range  |
 ------               | ------   | ------ | ------   | ------    | ------  | ------  | ------ |
 StringRegExp only    | 3.15     | 1      | 0.15     | 0.015     | 3.002   | 3.628   | 0.626  |
 jq UDF               | 65.22    | 20.7   | 1.928    | 0.193     | 63.274  | 74.064  | 10.791 |
 JsonC-UDF            | 110.34   | 35.03  | 3.961    | 0.396     | 106.711 | 137.913 | 31.202 |
 pure AutoIt JSON-UDF | 209.692  | 66.57  | 4.29     | 0.429     | 203.294 | 225.663 | 22.368 |
 JSMN-based JSON-UDF  | 226.071  | 71.77  | 4.477    | 0.448     | 218.189 | 247.016 | 28.827 |

 

Edited by argumentum
more

Follow the link to my code contribution ( and other things too ).
FAQ - Please Read Before Posting.
autoit_scripter_blue_userbar.png

Posted

Are your tests being run on virtual machines or physical machines?

Posted (edited)

In my "coding life", I rarely use JSON and when I do is infrequent and a small string, so speed/efficiency is not a big deal for me in my cases.
Having a prettified output is quite important to me because I use it a lot to discern how to go about a JSON when I have to deal with one.
.,.back to topic.

The "N6000" is a Server 2016 VM but the others are hardware.
In my trend of thought, build in ( all AutoIt ) is preferable than having an external file but if is an external file a DLL would be a faster data exchange, but that EXE is kicking other methods ass "from here to China town" as they say. I just don't understand why/how.
Also don't get the inconsistencies between one PC and another. And that is my "hmm" moment 🤔
 

Edited by argumentum
English

Follow the link to my code contribution ( and other things too ).
FAQ - Please Read Before Posting.
autoit_scripter_blue_userbar.png

Posted (edited)

CPU architecture changes and that I think is the difference.
 

  Reveal hidden contents

So I guess that is the answer to my question. Nope, not it.

Edit2:
playing around with the script:

  Reveal hidden contents

I got to see what is what ( to a degree )
 

Intel(R) Pentium(R) Silver N6000 @ 1.10GHz | Runs: 100 | Entry: "copilot" | Count: 15 | WinVer: 10.0.14393.0

 name                 | time[ms] | factor | Std. Dev | Std. Err. | min     | max     | range  |
 ------               | ------   | ------ | ------   | ------    | ------  | ------  | ------ |
 StringRegExp only    | 1.151    | 1      | 0.039    | 0.004     | 1.102   | 1.281   | 0.179  |
 JsonC-UDF            | 2.664    | 2.31   | 0.919    | 0.092     | 2.157   | 8.052   | 5.894  |
 jq UDF               | 46.975   | 40.81  | 2.475    | 0.248     | 44.542  | 55.569  | 11.027 |
 pure AutoIt JSON-UDF | 217.143  | 188.66 | 4.757    | 0.476     | 206.448 | 236.374 | 29.926 |
 JSMN-based JSON-UDF  | 237.231  | 206.11 | 7.779    | 0.778     | 228.528 | 279.891 | 51.364 |

Intel(R) Pentium(R) Silver N6000 @ 1.10GHz | Runs: 100 | Entry: "actions" | Count: 5061 | WinVer: 10.0.14393.0

 name                 | time[ms] | factor | Std. Dev | Std. Err. | min     | max     | range  |
 ------               | ------   | ------ | ------   | ------    | ------  | ------  | ------ |
 StringRegExp only    | 3.585    | 1      | 0.205    | 0.021     | 3.261   | 4.168   | 0.907  |
 jq UDF               | 52.809   | 14.73  | 1.572    | 0.157     | 50.744  | 58.847  | 8.104  |
 JsonC-UDF            | 116.794  | 32.58  | 5.827    | 0.583     | 112.52  | 158.619 | 46.099 |
 pure AutoIt JSON-UDF | 221.849  | 61.88  | 3.665    | 0.367     | 216.139 | 239.11  | 22.971 |
 JSMN-based JSON-UDF  | 235.16   | 65.6   | 4.233    | 0.423     | 227.894 | 251.581 | 23.687 |


12th Gen Intel(R) Core(TM) i9-12900H | Runs: 100 | Entry: "copilot" | Count: 15 | WinVer: 11 pro

 name                 | time[ms] | factor | Std. Dev | Std. Err. | min    | max     | range  |
 ------               | ------   | ------ | ------   | ------    | ------ | ------  | ------ |
 StringRegExp only    | 0.692    | 1      | 0.17     | 0.017     | 0.547  | 1.215   | 0.668  |
 JsonC-UDF            | 1.098    | 1.59   | 0.203    | 0.02      | 0.882  | 1.88    | 0.998  |
 jq UDF               | 30.073   | 43.46  | 2.542    | 0.254     | 26.073 | 36.233  | 10.161 |
 pure AutoIt JSON-UDF | 98.456   | 142.28 | 5.62     | 0.562     | 87.521 | 116.823 | 29.301 |
 JSMN-based JSON-UDF  | 107.644  | 155.55 | 6.317    | 0.632     | 94.134 | 125.391 | 31.257 |

12th Gen Intel(R) Core(TM) i9-12900H | Runs: 100 | Entry: "actions" | Count: 5061 | WinVer: 11 pro

 name                 | time[ms] | factor | Std. Dev | Std. Err. | min     | max     | range  |
 ------               | ------   | ------ | ------   | ------    | ------  | ------  | ------ |
 StringRegExp only    | 1.852    | 1      | 0.403    | 0.04      | 1.364   | 3.101   | 1.738  |
 jq UDF               | 34.722   | 18.75  | 2.798    | 0.28      | 29.726  | 42.913  | 13.187 |
 JsonC-UDF            | 52.277   | 28.23  | 3.896    | 0.39      | 45.481  | 64.209  | 18.728 |
 pure AutoIt JSON-UDF | 101.908  | 55.03  | 5.282    | 0.528     | 92.217  | 121.343 | 29.127 |
 JSMN-based JSON-UDF  | 111.208  | 60.05  | 6.03     | 0.603     | 101.079 | 133.279 | 32.201 |

and ... I should learn RegEx ( nah, too hard. I'll keep shamelessly copy'n'pasteing ).
But am satisfied with the results, and it now makes sense to me how a 61 kb DLL vs 962 kb EXE fare against each other given the conditions.
( understood that one is a general purpose library/DLL while the other a highly optimized program for it )

Edited by argumentum
more

Follow the link to my code contribution ( and other things too ).
FAQ - Please Read Before Posting.
autoit_scripter_blue_userbar.png

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...