JSON UDF in pure AutoIt

[littlebigman]

2 hours ago, mccree said:

is that what you mean?

Exactly. Thanks!

[0Ethan0]

Hello,

Thank you for this useful UDF!

I did notice an issue/bug - I am unsure about the right terminology so I will give an example:

#include <JSON.au3>

Global $mMap
_JSON_addChangeDelete($mMap, "Test.A[0]", "123") ; Adding
_JSON_addChangeDelete($mMap, "Test.A[1]", "456") ; Adding

_JSON_addChangeDelete($mMap, "Test.A[0]")       ; Deleting

ConsoleWrite(_JSON_Generate($mMap) & @CRLF & @CRLF)

Hypothetically we should be getting:

{
    "Test": {
        "A": [
            "456"
        ]
    }
}

Instead we are getting - notice the "" in the 4th line:

{
    "Test": {
        "A": [
            "",
            "456"
        ]
    }
}

There is also a small typo in the latest JSON.au3, line #309, the description of _JSON_addChangeDelete shows _JSON_addChange.

Thanks again for this UDF!

 

 

[0Ethan0]

I came up with a small "fix" - untested fully and it involves adding the Array.au3 library (though not a must if one wishes to replace the array related functions):

In line #34, below the:

#include <String.au3>

add:
 

#include <Array.au3>

Around line #382 where you have:

If $oNext = Default Then ; delete the current level
    If $sCurrenttype = "Map" Then
        MapRemove($oObject, $aLevels[$iRecLevel][1])
Else

add the following code just before the 'Else':

ElseIf $sCurrenttype = "Array" Then
    Local $sPatternTemp = _ArrayToString($aLevels, ".", -1, UBound($aLevels)-3, ".", 1, 1)
    Local $aTemp = _JSON_Get($oObject, $sPatternTemp)
    _ArrayDelete($aTemp, Number($aLevels[UBound($aLevels)-2][1]))
    $oObject = $aTemp

Not the most elegant but worked for a few small tests that I ran.

I hope this won't break anything else 😅

Edited May 10, 2023 by 0Ethan0

[AspirinJunkie]

Thank you for the report.
The case that an array element is to be deleted using this was simply not yet taken into account.

I have uploaded the new version.

[Rurorita]

Just gonna leave this here in case you actually didnt see the pr https://github.com/Sylvan86/autoit-json-udf/pull/1

[AspirinJunkie]

Oh thank you. I actually didn't see that. I started the github story to learn how to use github.
So far I didn't have a pull request and thought I would get some notification in that case.
But I have received nothing.
Let's see if I can set something.

Now to your changes:
Basically you can drive several approaches.
Either you design your code so that it is usable with as many use cases as possible or you optimize it for certain special cases.
With the latter you buy speed compared to more code and possibly also clarity.

So far I have used exactly this general approach with _JSON_Generate(). If you take a closer look, you could consider two more common special cases when using it here: On the one hand the case of the most compact JSON code possible (which funnily enough I simplified just yesterday with the _JSON_GenerateCompact()) and on the other hand a more common parameter choice for good human readability. The latter I have covered so far via the default parameters of _JSON_Generate().

Now, of course, you can write special functions for both and, unsurprisingly, achieve a performance boost for these cases. 
For one special case - the compact JSON code - you have now done this and in my measurements I get a performance increase of 20-25% compared to the general functions.

That's not little and now you can already ask if the tradeoff of performance improvement vs. code complexity makes sense or not. I myself needed the performance so far with _JSON_Parse(). With _JSON_Generate() I never reached a limit where I needed more. But that can be different for other users.

Therefore you can consider to add special functions for this, if there is a demand for it.
However, I would not like to take over the adjustments from you directly, for the following reasons:

_JSON_Generate() defaults to the nicely formatted _JSON code. With youre changes it becomes script-breaking because you now suddenly define the compact JSON code as the default case. Why not leave _JSON_Generate as the universal form and cast your case to a _JSON_GenerateCompact() instead? That way the backward compatibility would still exist.
I don't really like the $_JSON_TYPES approach considering the result achieved with this. You need another function which is executed once (and which itself could be ~80% faster), another global constant and the readability decreases (slightly). Result with my measurements: 1-2% faster.
That is not worth it, at least for me.

So in summary: Yes I could already imagine special _JSON_GenerateX functions for the two cases. But if then without the $_JSON_TYPES approach and without changing the behavior of _JSON_Generate().

But this concerns only my repository.
The nice thing about open source is that everyone can fork it and rewrite it according to their own ideas.

Otherwise, big thanks again for letting me continue to learn collaborative work via github through you!

Edited May 24, 2023 by AspirinJunkie

[HurleyShanabarger]

If I read a json file and save it again, string characters are not beeing escaped.

Is it possible to adjust this?

[AspirinJunkie]

The _JSON_Generate() function escapes the characters that must be replaced in valid JSON.
This concerns back space, form feed, new line, carriage return, tab, double quote and backslash.
Since JSON requires UTF-8 (see RFC 8259) there is no need to escape additional characters to create valid JSON.
If you still want to replace single characters beyond the necessary ones, you can simply do that via StringReplace (or StringRegExp so you don't accidentally replace key names) after _JSON_Generate().

A parameter to switch to a mode to replace more characters is not useful, because nobody knows which characters you want to replace beyond what is necessary.
This is just your personal taste.
_JSON_Generate() has the task to output valid JSON - that's exactly what it does.

[TheDcoder]

59 minutes ago, AspirinJunkie said:

because nobody knows which characters you want to replace beyond what is necessary.

Let's escape all characters which are not ASCII

[AspirinJunkie]

Why only non-ASCII? If then already completely and also all ASCII characters escaped.
So then you are really sure that the JSON produced in this way is no longer human-readable and also becomes significantly larger.

[TheDcoder]

1 hour ago, AspirinJunkie said:

Why only non-ASCII? If then already completely and also all ASCII characters escaped.

Because everyone understands ASCII, duh 🤷‍♂️

Also I'm joking in case that isn't obvious. But perhaps maybe an ASCII compliant JSON generator might be of use in certain situations.

[HurleyShanabarger]

Thank you for the feedback!

To be honest, I did not think about the validity of json I am processing in the first place. I just starting using your UDF by reading a json and just saving it. The difference I saw was only  that "/" in a string is beeing escaped to "\/". As this seems to only happen in json that I receive from a particular program, I will adapte my script to use StringReplace/StringRegExpReplace as you suggested.

[AspirinJunkie]

6 minutes ago, HurleyShanabarger said:

The difference I saw was only  that "/" in a string is beeing escaped to "\/"

Yes, the slash may be escaped in JSON. Unlike the backslash, however, this is not mandatory. That means "/" would be as valid in JSON as "\/" and would have the same result. I decided not to escape the slash, because I save one character and the readability is better.
 

6 hours ago, TheDcoder said:

Also I'm joking in case that isn't obvious.

That was obvious, of course.
My answer was also not really meant seriously.
If it didn't come across that way, it's probably because of my English deficits or because of my nationality, which has a reputation - well, as they say here: having a stick up your ass. 🙃

[Stormgrade]

Hello,

I'd like to update the firefox handlers.json with this UDF but I get nothing.

I'd tried it in different way without success.

#include <JSON.au3>
#include <Array.au3>

$s_String  = '{"defaultHandlersVersion":{"fr":3},"mimeTypes":{"application/pdf":{"action":2,"extensions":["pdf"],"handlers":[{"name":"AdobeAcrobat","path":"C:\\ProgramFiles\\Adobe\\AcrobatDC\\Acrobat\\Acrobat.exe"}]},"text/xml":{"action":0,"extensions":["xml"]},"image/svg+xml":{"action":0,"extensions":["svg"]},"image/webp":{"action":3,"extensions":["webp"]},"application/zip":{"action":0,"ask":false,"handlers":[{"name":"Explorer.exe","path":"C:\\WINDOWS\\Explorer.exe"}],"extensions":["zip"]},"application/x-7z-compressed":{"action":0,"ask":false,"extensions":["7z"]},"application/vnd.openxmlformats-officedocument.presentationml.presentation":{"action":4,"extensions":["pptx"]},"application/x-zip-compressed":{"action":0,"ask":false,"extensions":["7z","zip"]},"image/avif":{"action":3,"extensions":["avif"]},"application/vnd.openxmlformats-officedocument.wordprocessingml.document":{"action":4,"extensions":["docx"]},"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet":{"action":4,"extensions":["xlsx"]},"application/vnd.ms-excel.sheet.macroenabled.12":{"action":4,"extensions":["xlsm"]},"application/msword":{"action":4,"extensions":["doc"]},"text/x-csv":{"action":4,"extensions":["csv"]},"application/vnd.ms-powerpoint":{"action":4,"extensions":["ppt"]}},"schemes":{"irc":{"stubEntry":true,"handlers":[null,{"name":"Mibbit","uriTemplate":"https://www.mibbit.com/?url=%s"}]},"ircs":{"stubEntry":true,"handlers":[null,{"name":"Mibbit","uriTemplate":"https://www.mibbit.com/?url=%s"}]},"mailto":{"stubEntry":true,"handlers":[null,{"name":"Yahoo!Mail","uriTemplate":"https://compose.mail.yahoo.com/?To=%s"},{"name":"Gmail","uriTemplate":"https://mail.google.com/mail/?extsrc=mailto&url=%s"}]},"zoommtg":{"action":4},"hidglobal-approve":{"action":4}},"isDownloadsImprovementsAlreadyMigrated":true,"isSVGXMLAlreadyMigrated":true}'
$o_Object = _JSON_Parse($s_String)
If @error Then ConsoleWrite($o_Object&" "&@error&@CRLF)
$s_Type = _JSON_Get($o_Object, "[0].mimeTypes[0].application/pdf[0].extensions[0].handlers[0].name")
ConsoleWrite("name: " & $s_Type & @CRLF)

I want to update PDF path ,what did I do wrong ?

 

[AspirinJunkie]

You should also query the @error value of _JSON_Get() to get hints where it hangs.
In your case @error=4 is set, which means (see function header of _JSON_Get()): "index query on none array object".

Then it should become clear where the problem is.
You keep referring arrays where there are none - namely with your [0]-constructs.
The correct query for your example would look like this:

$s_Type = _JSON_Get($o_Object, "mimeTypes.application/pdf.handlers[0].name")

[Stormgrade]

Thank you for your help.

Quote

You keep referring arrays where there are none - namely with your [0]-constructs.

Because I don't really understand how JSON_Parse work. Can I display a table/string from it to see which one is an array ?

 

[AspirinJunkie]

 

23 minutes ago, Stormgrade said:

Can I display a table/string from it to see which one is an array ?

Arrays in JSON are the constructs which are surrounded by "[" and "]". Objects/Maps are the constructs which are surrounded by "{" and "}".
 

23 minutes ago, Stormgrade said:

Because I don't really understand how JSON_Parse work.

_JSON_Parse does nothing more than rebuild the structure declared as JSON string with AutoIt data structures.

So, for example, if you have the following JSON string:
{ "test": [1,2,3], "example": 3.14}

Then _JSON_Parse() makes it an AutoIt map with 2 elements: Under the key "test" there is an array with 3 elements: the numbers 1, 2 and 3. And under the key "example" there is a number with the value 3.14.
In order to access elements on these lower levels, you have to navigate through the structure. For example, if we want to have the last value of the array, we start at the outside: This is a map/object and we want the value for the key "test". So the query is just "test". So we would get back an array with the 3 elements with _JSON_Get(). But if we want to have directly only one specific one we can also expand our query: "test[2]".

So it is very important that the hierarchical structure of the JSON string is clear. In the form you have it there - without any indentations, line breaks etc. - it is very difficult to see the structure. It is therefore a good idea to format the JSON string accordingly. You can use the _JSON_Generate() function to do this - it will turn your $o_Object into an appropriately formatted string. Or you can use various online viewers that format JSON strings accordingly. With >>this one<< (only one example of thousands) you would even get the corresponding query path when you click on an element.

[Stormgrade]

Thank you for your explanation I get it

 

[Yaerox]

Running the JSON Test File provided I always get the same error. Am I missing something?

[Musashi]

As far as I know, this UDF works with datatype Maps and therefore requires AutoIt version 3.3.16.1 or higher.