A Non-Strict JSON UDF (JSMN)

[jugador]

@Danp2 don't know what error you are talking
just run @mLipok code with the modified Json_ObjExists & here's the output

+>Setting Hotkeys...--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
Test #5 Empty: {"capabilities":"{}"}
- 158 $v_Test = True
- 161 $v_Test = False
- 164 $v_Test = False
Test #5 Result: {"capabilities":{"alwaysMatch":"{}","firstMatch":[{"proxy":{"noProxy":["www.google.com","www.yahoo.com"]}}]}}
- 173 $v_Test = 2
- 176 $v_Test = 1
- 179 $v_Test = False
- 182 $v_Test = True
- 185 $v_Test = False
- 188 $v_Test = True
+>18:05:10 AutoIt3.exe ended.rc:0

 

+>Setting Hotkeys...--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
Test #5 Empty: {"capabilities":"{}"}
- 158 $v_Test = True
- 161 $v_Test = False
+>18:13:52 AutoIt3.exe ended.rc:0

 

[Danp2]

@jugadorMaybe I misunderstood, but see my comment added below --

Func Json_ObjExists(ByRef $Object, $Key)
    Local $DynObject = $Object
    Local $Keys = StringSplit($Key, ".")
    For $x = 1 To $Keys[0]
        If $DynObject.Exists($Keys[$x]) Then                    ;~~~~ The error was occurring here
            If $x = $Keys[0] Then
                Return True
            Else
                $DynObject = Json_ObjGet($DynObject, $Keys[$x])
                If Not IsObj($DynObject) Then Return False      ;~~~~ add this line to Check if $DynObject is object
            EndIf
        Else
            Return False
        EndIf
    Next
    Return False
EndFunc   ;==>Json_ObjExists

 

[jugador]

@Danp2 

for 2nd example 
when Json_ObjGet try to find 'alwaysMatch' it failed as 'alwaysMatch' not exist
so it return string

$DynObject = Json_ObjGet($DynObject, $Keys[$x])
ConsoleWrite(VarGetType($DynObject)  & @CRLF)

that's why it failed on this line during 2nd loop

If $DynObject.Exists($Keys[$x]) Then

 

Edited April 10, 2022 by jugador

[mLipok]

9 hours ago, jugador said:

@mLipok try with this Json_ObjExists modification

Func Json_ObjExists(ByRef $Object, $Key)
    Local $DynObject = $Object
    Local $Keys = StringSplit($Key, ".")
    For $x = 1 To $Keys[0]
        If $DynObject.Exists($Keys[$x]) Then
            If $x = $Keys[0] Then
                Return True
            Else
                $DynObject = Json_ObjGet($DynObject, $Keys[$x])
                If Not IsObj($DynObject) Then Return False      ;~~~~ add this line to Check if $DynObject is object
            EndIf
        Else
            Return False
        EndIf
    Next
    Return False
EndFunc   ;==>Json_ObjExists

 

It fixes this issue.

[mLipok]

The same problem occurs in:

#include "Json.au3"

test6()

Func test6()
    Local $_WD_CAPS__OBJECT

    Json_Put($_WD_CAPS__OBJECT, '[capabilities]', '{}')
    ConsoleWrite("Test #5 Empty: " & Json_Encode($_WD_CAPS__OBJECT) & @LF)

    Local $DynObject = Json_ObjGet($_WD_CAPS__OBJECT, 'capabilities.alwaysMatch')
    #forceref $DynObject
EndFunc   ;==>test6

 

Edited April 10, 2022 by mLipok

[mLipok]

@Danp2 take a look on this testing script:

expandcollapse popup

#include "Json.au3"

test5()

Func test5()
    Local $_WD_CAPS__OBJECT
    Local $v_Test

    Json_Put($_WD_CAPS__OBJECT, '[capabilities]', '{}')
    ConsoleWrite("Test #5 Empty: " & Json_Encode($_WD_CAPS__OBJECT) & @LF)

    Local $DynObject = Json_ObjGet($_WD_CAPS__OBJECT, 'capabilities')
    $DynObject = Json_ObjGet($DynObject, 'alwaysMatch')

    $v_Test = Json_ObjExists2($_WD_CAPS__OBJECT, 'capabilities.alwaysMatch')
    ConsoleWrite("- " & @ScriptLineNumber &  ' $v_Test = ' & $v_Test  & @CRLF)

EndFunc   ;==>test5

Func Json_ObjExists2(ByRef $Object, $Key)
;~  https://www.autoitscript.com/forum/topic/148114-a-non-strict-json-udf-jsmn/?do=findComment&comment=1499979
    Local $DynObject = $Object
    Local $Keys = StringSplit($Key, ".")
    For $x = 1 To $Keys[0]
ConsoleWrite("> " & $x & ' / ' & $Keys[0] & ' $Keys[$x] = ' & $Keys[$x] & @CRLF)

        If $DynObject.Exists($Keys[$x]) Then
ConsoleWrite(@ScriptLineNumber & @CRLF)

            If $x = $Keys[0] Then
ConsoleWrite(@ScriptLineNumber & @CRLF)
                Return True
            Else
ConsoleWrite(@ScriptLineNumber & ' ' & VarGetType($DynObject) & @CRLF)
                $DynObject = Json_ObjGet($DynObject, $Keys[$x])
ConsoleWrite(@ScriptLineNumber & ' ' & VarGetType($DynObject) & ' >> ' & $DynObject & @CRLF)
;~                 If Not IsObj($DynObject) Then Return False      ;~~~~ add this line to Check if $DynObject is object
            EndIf
        Else
            Return False
        EndIf
    Next
    Return False
EndFunc   ;==>Json_ObjExists

 

[mLipok]

I would like to propose to change Json_ObjExists() function to this form:

; #FUNCTION# ====================================================================================================================
; Name ..........: Json_ObjExists
; Description ...: Check if desired key in JSON structure exist or not
; Syntax ........: Json_ObjExists(Byref $Object, $Key)
; Parameters ....: $Object              - reference to object containing JSON structure
;                  $Key                 - location for desired key in JSON structure
; Return values .: True/False
; Author ........: Ward
; Modified ......: Jos, jugador, mLipok
; Remarks .......: 
; Related .......: 
; Link ..........: 
; Example .......: No
; ===============================================================================================================================
Func Json_ObjExists(ByRef $Object, $Key)
    Local $DynObject = $Object
    Local $Keys = StringSplit($Key, ".")
    For $x = 1 To $Keys[0] ; go through the keys
        If Not $DynObject.Exists($Keys[$x]) Then Return False ; if current key not exist
        If $x = $Keys[0] Then Return True ; if exist and it is the last key

        $DynObject = Json_ObjGet($DynObject, $Keys[$x]) ; move to next key
        If Not IsObj($DynObject) Then Return False ; if next key not exist
    Next
    Return False ; if not found
EndFunc   ;==>Json_ObjExists

instead current version:

Func Json_ObjExists2(ByRef $Object, $Key)
    Local $DynObject = $Object
    Local $Keys = StringSplit($Key, ".")
    For $x = 1 To $Keys[0]
        If $DynObject.Exists($Keys[$x]) Then
            If $x = $Keys[0] Then
                Return True
            Else
                $DynObject = Json_ObjGet($DynObject, $Keys[$x])
            EndIf
        Else
            Return False
        EndIf
    Next
    Return False
EndFunc   ;==>Json_ObjExists

 

Edited April 10, 2022 by mLipok

[mLipok]

I would like to propose to change Json_ObjExists() function to this form:

; #FUNCTION# ====================================================================================================================
; Name ..........: Json_ObjGet
; Description ...: Get object refrence to any descendants key in JSON structure
; Syntax ........: Json_ObjGet(Byref $Object, $Key)
; Parameters ....: $Object              - reference to object containing JSON structure
;                  $Key                 - location for desired key in JSON structure
; Return values .: #TODO
; Author ........: Ward
; Modified ......: Jos, mLipok
; Remarks .......: if last key is returning in empty object which contain "{}" it will return string not object
; Related .......:
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func Json_ObjGet(ByRef $Object, $Key)
    Local $DynObject = $Object
    Local $Keys = StringSplit($Key, ".")
    For $x = 1 To $Keys[0] ; go through the keys
        If Not $DynObject.Exists($Keys[$x]) Then Return SetError(1, 0, '') ; if current key not exist
        If $x = $Keys[0] Then Return $DynObject.Item($Keys[$x]) ; if exist and it is the last key

        $DynObject = Json_ObjGet($DynObject, $Keys[$x]) ; move to next key
        If Not IsObj($DynObject) Then Return SetError(2, 0, '')  ; if next key not exist
    Next
    Return SetError(3, 0, '') ; if not found
EndFunc   ;==>Json_ObjGet

instead:

Func Json_ObjGet(ByRef $Object, $Key)
    Local $DynObject = $Object
    Local $Keys = StringSplit($Key, ".")
    For $x = 1 To $Keys[0]
        If $DynObject.Exists($Keys[$x]) Then
            If $x = $Keys[0] Then
                Return $DynObject.Item($Keys[$x])
            Else
                $DynObject = Json_ObjGet($DynObject, $Keys[$x])
            EndIf
        EndIf
    Next
    Return SetError(1, 0, '')
EndFunc   ;==>Json_ObjGet

 

[mLipok]

here is some new testing script for Json_ObjGet() function:

expandcollapse popup

#include "Json.au3"

test6()

Func test6()
    Local $_WD_CAPS__OBJECT, $DynObject

    Json_Put($_WD_CAPS__OBJECT, '[capabilities]', '{}')
    ConsoleWrite("Test #6 Empty: " & Json_Encode($_WD_CAPS__OBJECT) & @LF)

    $DynObject = Json_ObjGet($_WD_CAPS__OBJECT, 'capabilities')
    ConsoleWrite("- " & @ScriptLineNumber & ' @error = ' & @error & ' IsObj = ' & IsObj($DynObject) & ' $DynObject = ' & Json_Encode($DynObject) & @LF)

    $DynObject = Json_ObjGet($_WD_CAPS__OBJECT, 'capabilities.alwaysMatch')
    ConsoleWrite("- " & @ScriptLineNumber & ' @error = ' & @error & ' IsObj = ' & IsObj($DynObject) & ' $DynObject = ' & Json_Encode($DynObject) & @LF)

    Json_Put($_WD_CAPS__OBJECT, '[capabilities][alwaysMatch]', '{}')
    ConsoleWrite("Test #6 Empty: " & Json_Encode($_WD_CAPS__OBJECT) & @LF)

    $DynObject = Json_ObjGet($_WD_CAPS__OBJECT, 'capabilities')
    ConsoleWrite("- " & @ScriptLineNumber & ' @error = ' & @error & ' IsObj = ' & IsObj($DynObject) & ' $DynObject = ' & Json_Encode($DynObject) & @LF)

    $DynObject = Json_ObjGet($_WD_CAPS__OBJECT, 'capabilities.alwaysMatch')
    ConsoleWrite("- " & @ScriptLineNumber & ' @error = ' & @error & ' IsObj = ' & IsObj($DynObject) & ' $DynObject = ' & Json_Encode($DynObject) & @LF)

EndFunc   ;==>test6

; #FUNCTION# ====================================================================================================================
; Name ..........: Json_ObjGet
; Description ...: Get object refrence to any descendants key in JSON structure
; Syntax ........: Json_ObjGet(Byref $Object, $Key)
; Parameters ....: $Object              - reference to object containing JSON structure
;                  $Key                 - location for desired key in JSON structure
; Return values .: #TODO
; Author ........: Ward
; Modified ......: Jos, mLipok
; Remarks .......: if last key is returning in empty object which contain "{}" it will return string not object
; Related .......:
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func Json_ObjGet2(ByRef $Object, $Key)
    Local $DynObject = $Object
    Local $Keys = StringSplit($Key, ".")
    For $x = 1 To $Keys[0] ; go through the keys
        If Not $DynObject.Exists($Keys[$x]) Then Return SetError(1, 0, '') ; if current key not exist
        If $x = $Keys[0] Then Return $DynObject.Item($Keys[$x]) ; if exist and it is the last key

        $DynObject = Json_ObjGet($DynObject, $Keys[$x]) ; move to next key
        If Not IsObj($DynObject) Then Return SetError(2, 0, '')  ; if next key not exist
    Next
    Return SetError(3, 0, '') ; if not found
EndFunc   ;==>Json_ObjGet2

 

Edited April 10, 2022 by mLipok

[mLipok]

I've always had the feeling that I don't have an advanced example here that clearly shows how to formulate KEYS to achieve the desired JSON structure.
Hopefully this example below will make things clearer for other users of this UDF, I mean those who haven't fully understood it yet.

expandcollapse popup

#include "Json.au3"

test7()

Func test7()
    Local $Obj
    Json_Put($Obj, ".Primitive_1", "Test") ; JSON_PRIMITE "Primitive_1"
    Json_Put($Obj, ".Primitive_2", True) ; JSON_PRIMITE "Primitive_2"
    Json_Put($Obj, ".Primitive_3", False) ; JSON_PRIMITE "Primitive_3"
    Json_Put($Obj, ".Primitive_4", Null) ; JSON_PRIMITE "Primitive_4"

    Json_Put($Obj, ".List_A[0]", "Test") ; JSON_ARRAY "List_A"
    Json_Put($Obj, ".List_A[1]", True) ; JSON_ARRAY "List_A"
    Json_Put($Obj, ".List_A[2]", False) ; JSON_ARRAY "List_A"
    Json_Put($Obj, ".List_A[3]", Null) ; JSON_ARRAY "List_A"
    Json_Put($Obj, ".List_A[4]", "value_A=100") ; JSON_ARRAY "List_A"

    Json_Put($Obj, ".List_A[4].Object_1", "{}") ; JSON_OBJECT "Object_1" as subelement of JSON_ARRAY "List_A"

    Json_Put($Obj, ".List_A[4].Object_1.List_A_B[0]", "Test") ; JSON_ARRAY "List_A_B" as subelement of JSON_OBJECT "Object_1" which is subelement of JSON_ARRAY "List_A"
    Json_Put($Obj, ".List_A[4].Object_1.List_A_B[1]", True) ; JSON_ARRAY "List_A_B" as subelement of JSON_OBJECT "Object_1" which is subelement of JSON_ARRAY "List_A"
    Json_Put($Obj, ".List_A[4].Object_1.List_A_B[2]", False) ; JSON_ARRAY "List_A_B" as subelement of JSON_OBJECT "Object_1" which is subelement of JSON_ARRAY "List_A"
    Json_Put($Obj, ".List_A[4].Object_1.List_A_B[3]", Null) ; JSON_ARRAY "List_A_B" as subelement of JSON_OBJECT "Object_1" which is subelement of JSON_ARRAY "List_A"
    Json_Put($Obj, ".List_A[4].Object_1.List_A_B[4]", "value_A_B=100") ; JSON_OBJECT

    Json_Put($Obj, ".List_C[0]", "Test") ; JSON_CRRAY "List_C"
    Json_Put($Obj, ".List_C[1]", True) ; JSON_CRRAY "List_C"
    Json_Put($Obj, ".List_C[2]", False) ; JSON_CRRAY "List_C"
    Json_Put($Obj, ".List_C[3]", Null) ; JSON_CRRAY "List_C"
    Json_Put($Obj, ".List_C[4]", "value_C=100") ; JSON_CRRAY "List_C"


    Json_Put($Obj, ".List_D[0].Object_2.List_D_E[0]", "Test") ; JSON_DRRAY "List_D_E" as subelement of JSON_OBJECT "Object_2" which is subelement of JSON_DRRAY "List_D"
    Json_Put($Obj, ".List_D[0].Object_2.List_D_E[1]", True) ; JSON_DRRAY "List_D_E" as subelement of JSON_OBJECT "Object_2" which is subelement of JSON_DRRAY "List_D"
    Json_Put($Obj, ".List_D[0].Object_2.List_D_E[2]", False) ; JSON_DRRAY "List_D_E" as subelement of JSON_OBJECT "Object_2" which is subelement of JSON_DRRAY "List_D"
    Json_Put($Obj, ".List_D[0].Object_2.List_D_E[3]", Null) ; JSON_DRRAY "List_D_E" as subelement of JSON_OBJECT "Object_2" which is subelement of JSON_DRRAY "List_D"
    Json_Put($Obj, ".List_D[0].Object_2.List_D_E[4]", "value_D_E=100") ; JSON_OBJECT

    Json_Put($Obj, ".List_D[0].Object_3.List_D_F[0]", "Test") ; JSON_DRRAY "List_D_F" as subelement of JSON_OBJECT "Object_3" which is subelement of JSON_DRRAY "List_D"
    Json_Put($Obj, ".List_D[0].Object_3.List_D_F[1]", True) ; JSON_DRRAY "List_D_F" as subelement of JSON_OBJECT "Object_3" which is subelement of JSON_DRRAY "List_D"
    Json_Put($Obj, ".List_D[0].Object_3.List_D_F[2]", False) ; JSON_DRRAY "List_D_F" as subelement of JSON_OBJECT "Object_3" which is subelement of JSON_DRRAY "List_D"
    Json_Put($Obj, ".List_D[0].Object_3.List_D_F[3]", Null) ; JSON_DRRAY "List_D_F" as subelement of JSON_OBJECT "Object_3" which is subelement of JSON_DRRAY "List_D"
    Json_Put($Obj, ".List_D[0].Object_3.List_D_F[4]", "value_D_F=100") ; JSON_OBJECT

    Json_Put($Obj, ".Empty_object", "{}") ; JSON_OBJECT "Empty_object" with out any elements

    ConsoleWrite("JSON structure: " & Json_Encode($Obj) & @LF)
    ConsoleWrite("JSON structure PRETTY PRINT: " & @LF & Json_Encode($Obj, $JSON_PRETTY_PRINT) & @LF)

EndFunc   ;==>test7

 

it produce:

expandcollapse popup

{
    "Primitive_1": "Test",
    "Primitive_2": true,
    "Primitive_3": false,
    "Primitive_4": null,
    "List_A": [
        "Test",
        true,
        false,
        null,
        {
            "Object_1": {
                "List_A_B": [
                    "Test",
                    true,
                    false,
                    null,
                    "value_A_B=100"
                ]
            }
        }
    ],
    "List_C": [
        "Test",
        true,
        false,
        null,
        "value_C=100"
    ],
    "List_D": [
        {
            "Object_2": {
                "List_D_E": [
                    "Test",
                    true,
                    false,
                    null,
                    "value_D_E=100"
                ]
            },
            "Object_3": {
                "List_D_F": [
                    "Test",
                    true,
                    false,
                    null,
                    "value_D_F=100"
                ]
            }
        }
    ],
    "Empty_object": "{}"
}

 

[Mateocedillo]

Hello,
Can you speed up the processing speed on long JSONs that contains much items? That is, I have a JSON that has an array of more than a thousand elements and I am getting them all in an Array but it takes a long time, depending on the number of elements. In other cases like browsers or Python it gets them all almost instantly. I have the latest dependency (2021.11.06).

[Jos]

Let me know when you are done and we will implement your version.

Edited July 2, 2022 by Jos

[Danyfirex]

@Jos don't be rude.  I'm sure He was not requesting/asking in a imposed way. I think He is asking if is possible to speed it up. You woke up on the wrong foot today. 😢

 

@Mateocedillo try this UDF.

 

 

PD: Mateocedillo is a blind person.

Saludos

 

 

 

 

Edited July 2, 2022 by Danyfirex

[TheDcoder]

1 hour ago, Danyfirex said:

@Mateocedillo try this UDF.

This UDF looks dangerous, lots of opportunities for code injection exploits! I wouldn't recommend anyone to use this.

@Mateocedillo I'd suggest you to use an external tool like jq to parse your large JSON data. I think someone may have even made an AutoIt UDF which uses jq

[mLipok]

 

 

[jugador]

If @TheDcoder said it "This UDF looks dangerous" then it has to be dangerous 
Mr @TheDcoder as you have decoded the malicious intent then why not do a favour to Autoit community and report the Mod to delete Json2 thread.

To those who still want to use it even after @TheDcoder warning
1) json2.js => github link given
2) link to Danyfirex thread given from where Base64 String taken
3) or use Base64 decoder to check the Base64 String

[TheDcoder]

@jugador You seem offended, sorry. But I wasn't suggesting that your UDF is malicious, I was simply pointing out the fact that it is not secure.

As per my understanding, you use Microsoft's JScript engine to parse JSON by passing it as input to your own function, but the input is not sanitized to protect against arbitrary code, am I right?

A malicious JSON payload might contain code within the JSON or even outside it which is executed without the user's or script author's knowledge. It is important that the JSON input is sanitized because it is directly being injected as code which is run.

[Jos]

@TheDcoder,

Isn't the same true for Compiled script by definition with the encoded script being in the PE header

[TheDcoder]

@Jos Are we talking about compiled AutoIt scripts? If so I don't think I understand, please explain

Edited July 3, 2022 by TheDcoder

[Jos]

4 hours ago, TheDcoder said:

Are we talking about compiled AutoIt scripts? If so I don't think I understand, please explain

Yes ....  and no, not up for further discussion in the open forum. My point is that this talk about code injection is somewhat far-fetched.