Search the Community

This UDF brings the power and flexibility of jq to AutoIt scripts. jq is an open-source, powerful, and flexible command-line based JSON processor. As it says on their website, jq is like 'sed' for JSON. jq can be used for the simplest of tasks like retrieving JSON objects and values (parsing), to very advanced JSON processing using its numerous built-in functions and conditional processing. Its built-in functions can handle math, selection, conditional processing, mapping, object and array manipulation, flattening, reduction, grouping, and much more. You can even create your own jq functions. You can learn more about jq and even play with it in real-time, using jq's online jq playground, all on their website. Here and some helpful links to get you more familiar with jq, what can be done with it, its built-in functions, and its syntax. jq Home Page: https://jqlang.github.io/jq/ jq Manual: https://jqlang.github.io/jq/manual/ jqWiki (FAQ, Cookbook, Advanced Topics) https://github.com/jqlang/jq/wiki jq is a single 32 or 64 bit executable that has no other dependencies. Just like using the SQLite UDF, the only requirement to use this UDF is that the jq executable reside in a location in which the UDF can execute it. The latest win32 & win64 versions have been included in the UDF download. You can always get newer versions from the jq website. jq at a high level Like 'sed', jq reads JSON in, either through STDIN or one or more files, processes it thru one or more "filters", and outputs the results. You can, optionally, supply "options" that affect how it reads the input, where it gets its "filters", and how it writes its output. It looks a little like this: JSON ---> jq processor (using supplied filters and options) ---> Output So in jq lingo, you basically use "Filters" to tell jq what you want it to do. So in the UDF file, that is why the main functions ( _jqExec() and _jqExecFile() ) refer to filters and options. Please make note that jq works with relatively strict JSON. This means that all JSON read must be conform to the standard. Luckily, jq is pretty good at identifying where a format error exists in non standard JSON. The jq UDF There are 2 main funtions in the UDF file, _jqExec and jqExecFile. With these 2 functions, you can pretty much do anything that jq can do. The only difference between to two functions is whether the JSON is supplied by a string or a file. The 2 primary functions simply call the jq executable with the supplied information, after properly formatting the parameters. There are additional functions in the UDF to easily pretty-print your json, compact-print your json, dump the json data with its associated paths, and see if specific JSON keys exist, but they all just execute the _jqExec or _jqExecFile function with the proper filter. There are also a couple of extra functions to display what version of the UDF and jq executable you are currently using. There are also a couple of functions to enable and disable logging of jq information for debugging purposes. Most of the jq UDF file functions return an @error if unsuccessful. Some also include @extended info. Please see the actual function headers for more information on their usage and return values. The 2 primary functions below just format your jq request and pass it on the jq executable. The functions will also properly escape double quotes (") that are used in the filter. For most simple tasks, you just need to supply the JSON source and a filter. _jqExec($sJson, $sFilter, $sOptions = Default, $sWorkingDir = Default) Or _jqExecFile($sJsonFile, $sFilter, $sOptions = Default, $sWorkingDir = Default) Using jq in your script As stated earlier, the jq executable must reside somewhere where the script can locate and execute it. The _jqInit() function always has to be executed before any jq processing occurs. _jqInit() merely locates the executable or uses the supplied path. It also clears any previous debug log. The jq UDF folder contains a jq example script that has several examples to how to do some of the most common JSON processing tasks. Here are a few examples to get you started: How to pretty-print some JSON #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"fruits":[{"Apple":{"color":"Red","season":"Fall"}}, {"Banana":{"color":"Yellow","season":"Summer"}}]}' $sCmdOutput = _jqPrettyPrintJson($sJson) ConsoleWrite(@CRLF & "Pretty-Print JSON" & @CRLF & $sCmdOutput & @CRLF) How to compact-print some JSON #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{ "fruits" : [{"Apple" : {"color":"Red","season":"Fall"}}, {"Banana":{"color":"Yellow","season":"Summer"}}]}' $sCmdOutput = _jqCompactPrintJson($sJson) ConsoleWrite(@CRLF & "Compact-Print JSON" & @CRLF & $sCmdOutput & @CRLF) Dump JSON data (paths and values) #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{ "fruits" : [{"Apple" : {"color":"Red","season":"Fall"}}, {"Banana":{"color":"Yellow","season":"Summer"}}]}' $sCmdOutput = _jqDump($sJson) ConsoleWrite(@CRLF & "Dump JSON paths and values" & @CRLF & $sCmdOutput & @CRLF) How to GET JSON values #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple" : {"color":"Red","season":"Fall"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = '.Banana.color' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("Get color of banana" & @CRLF) ConsoleWrite("Input: : " & _jqCompactPrintJson($sJson) & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF) or #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple" : {"color":"Red","season":"Fall"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = 'getpath(["Banana", "color"])' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("Get color of banana" & @CRLF) ConsoleWrite("Input: : " & _jqCompactPrintJson($sJson) & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF) Check for the existence of a key #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple":{"color":"Red","season":"Fall"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = '.Banana | has("color")' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("Check for existence of color key within Banana object" & @CRLF) ConsoleWrite("Input: : " & _jqCompactPrintJson($sJson) & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF) Count of how many Items in an object #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple":{"color":"Red"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = '.Banana | length' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("How many items in the Banana object" & @CRLF) ConsoleWrite("Input: : " & _jqCompactPrintJson($sJson) & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF) How to PUT/Create/Modify JSON #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sInput = "" $sFilter = 'setpath(["Apple","color"];"Red") | setpath(["Banana","color"];"Yellow") | setpath(["Banana","season"];"Summer")' $sOptions = '-n' ;required if no input supplied $sCmdOutput = _jqExec($sInput, $sFilter, $sOptions) ConsoleWrite("Update/Create JSON" & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & @CRLF & $sCmdOutput & @CRLF) List all of the fruits (top-level keys) #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '{"Apple":{"color":"Red"}, "Banana":{"color":"Yellow","season":"Summer"}}' $sFilter = 'keys | .[]' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("List all top-level keys (fruits)" & @CRLF) ConsoleWrite("Input : " & $sJson & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & @CRLF & $sCmdOutput & @CRLF) Calculate the sum of all of the objects' price * qty #include "jq.au3" ;Initialize jq environment _jqInit() If @error Then Exit ConsoleWrite("ERROR: Unable to initialize jq - @error = " & @error & @CRLF) $sJson = '[{"id":1,"price":20.00,"qty":10},{"id":2,"price":15.00,"qty":20.25},{"id":3,"price":10.50,"qty":30}]' $sFilter = 'map(.price * .qty) | add' $sCmdOutput = _jqExec($sJson, $sFilter) ConsoleWrite("Calculate the sum of all of the objects' price * qty" & @CRLF) ConsoleWrite("Input : " & $sJson & @CRLF) ConsoleWrite("Filter : " & $sFilter & @CRLF) ConsoleWrite("Output : " & $sCmdOutput & @CRLF) The examples above, and the ones in the example files, merely scratch the surface of what jq can do. It may look intimidating at first but it really isn't that bad once you start playing with it. You can find several more examples and play around with them or create your own using jqPlayground. If you have any questions regarding the UDF, or how to perform a certain task using jq, I'll try my best to answer them. Since jq has been around for a while now, there's also several jq-related questions and answers on StackOverflow.

Create / Test / Learn JSON Processing jqPlayground is an interactive, jq-based, tool created using AutoIt. The purpose of the tool is to help in the creation, testing, and understanding of JSON filters/queries used in the processing of JSON datasets. Internally, it uses the jq UDF for all JSON processing. The dot and bracket notation access in jq are similar to other existing AutoIt JSON parsing UDFs and tools. Therefore, jqPlayground can be used as a general purpose testing & learning tool, regardless of the ultimate UDF or utility you choose to use. jqPlayground comes with numerous examples to help you see and understand how it can be used for simple parsing and much more advanced JSON processing. You can modify and play with the example filters or you can create and test your own. CONFIGURATION The only requirement needed to run jqPlayground is that it can find a jq executable. JQ executables can be found on the JQ website on the home page, its download section, or as a part of my JQ UDF in the AutoIt Downloads section. The latest jq executables have been included in the zip file. jqPlayground will look for the jq executable in the following order of precedence: 1. If a jqPlayground.ini file exists in the script directory, it will get the path to the jq executable under to following section and key: [CONFIG] JqExePath=<full path to jq exe> 2. A jq-win32.exe or jq-win64.exe, depending on the OS, in the script directory. 3. A jq.exe in the script directory. USAGE The interface is pretty simple and straight forward. Paste, load or write whatever JSON you want to play with in the INPUT section. Paste or write whatever parsing or processing filter(s) you want to test in the FILTER section. If necessary, you can select or enter any jq-specific flags you want. Then, either press the RUN button, F5 or CTRL+ENTER to execute your filter. You will see the output in the OUTPUT section. If your command was not successful, you will see the error message generated by JQ in the output section. There are also numerous examples that can be selected from EXAMPLES dropdown list. Upon selecting an example, it populates the filter, flags, and input as necessary. It then executes the example, showing the output in the OUTPUT section. The examples use one of 2 JSON datasets. BOOKS is a small, simple, JSON dataset that contains a catalog of book information. The NFL JSON dataset is much larger and complex. It is a snapshot of NFL game information during week 1 of the 2018 regular season. Some of the NFL JSON dataset examples really show off the speed and processing capabilities of JQ. If you want to dump the full path of every scalar JSON value in your JSON dataset, you can type "dump" in the FILTER section and press F5, CTRL+ENTER or the RUN button. The output is the same as the jqDump() function in the jq UDF. "Dump" is not a jq filter command. It is a special command that I added to help those new to JSON understand how to access a given JSON value using dot-notation. Lastly, right below the output section, you will see the time it took to execute your filter. This can be useful to those in which timimg is a major concern. jqPlayground HotKeys -------------------- Open the online jq manual - F1 Run the jq filter - F5 or Ctrl+Enter Clear/reset all fields - Alt+C Load a JSON from a file - Alt+L Save your current session - Ctrl+S Load a saved session - Ctrl+L Special Commands (Enter and run in filter) ------------------------------------------ dump - List full path of every scalar JSON value clear - Clear/reset all fields (same as Alt+C) USEFUL LINKS jq Home Page: https://jqlang.github.io/jq/ jq Manual: https://jqlang.github.io/jq/manual/ jq Downloads: https://jqlang.github.io/jq/download/ jq Tutorial: https://jqlang.github.io/jq/tutorial/ jq Wiki: https://github.com/jqlang/jq/wiki