Map data from/to file

[Sascha]

Hello together,

I would like to know what may be the best way to get data with the follwing structure from/to a file:

Local $mMap[]
Local $mPerson[]

$mPerson["no"] = 42;
$mPerson["first"] = "Jerry"
$mPerson["name"] = "Mouse"

$mMap[42] =  $mPerson

$mPerson["no"] = 43;
$mPerson["first"] = "Tom"
$mPerson["name"]  = "Cat"

$mMap[43] = $mPerson

ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : 43 = ' & $mMap[43].first & @crlf & '>Error code: ' & @error & @crlf) ;### Debug Console
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : 42  = ' & $mMap[42].first  & @crlf & '>Error code: ' & @error & @crlf) ;### Debug Console

 

After loading i would like to do some changes or additions and then save the changes.

Should i use json? and if so how would the json file look like to access the data by key?

Regards

Sascha

 

Edited November 20, 2023 by Sascha

[Nine]

I am tempted to suggest XML format (although json would be also a good choice).  I like XML because the COM object is well documented and straightforward.

expandcollapse popup

#include <Array.au3>

; Read all list

Local $oXML = ObjCreate("Microsoft.XMLDOM")
$oXML.load("PersonList.xml")
Local $oList = $oXML.getElementsByTagName("Person")
ConsoleWrite("The file contains " & $oList.length & " persons" & @CRLF)
Local $mPerson[], $mMap[]
For $i = 0 To $oList.length - 1
  $mPerson.no = int($oList.item($i).getAttribute("No"))
  $mPerson.first = $oList.item($i).getAttribute("First")
  $mPerson.name = $oList.item($i).getAttribute("Name")
  $mMap[int($mPerson.no)] = $mPerson
Next

ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : 43 = ' & $mMap[43].first & @crlf & '>Error code: ' & @error & @crlf) ;### Debug Console
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : 42  = ' & $mMap[42].name  & @crlf & '>Error code: ' & @error & @crlf) ;### Debug Console

; Make modification here

$mPerson.no = 44
$mPerson.first = "Lassy"
$mPerson.name = "Dog"
$mMap[44] = $mPerson

; Save all list

Local $aList = MapKeys($mMap)
Local $oPerson = $oList.item(0)
$oList.removeAll()
$oRoot = $oXML.documentElement

For $i = 0 To UBound($aList) - 1
  $oPerson.setAttribute("No", $aList[$i])
  $oPerson.setAttribute("First", $mMap[$aList[$i]].first)
  $oPerson.setAttribute("Name", $mMap[$aList[$i]].name)
  $oClone = $oPerson.cloneNode(True)
  $oRoot.appendChild($oClone)
Next

$oXML.save("New PersonList.xml")

Xml file format :

Spoiler

<?xml version="1.0" encoding="utf-8"?>
<Persons>
  <Person No="42" First="Jerry" Name="Mouse" />
  <Person No="43" First="Tom" Name="Cat" />
</Persons>

 

[AspirinJunkie]

Briefly to understand: The map $mMap is to be read from a file in its nested structure and then written again with the corresponding changes?

If yes: The term for this is serialize/deserialize.

Basically, you are completely free to do this. You can write something of your own for the special structure. However, it is easier to use the corresponding ones.
You have already mentioned one of these: >>JSON<<
That would be very suitable for this.
Basically, you only need one call to fill $mMap from a file and only one call to write the file again.
That should also answer your question:

2 hours ago, Sascha said:

and if so how would the json file look like to access the data by key?

You don't need to worry about the structure of the JSON file. It is automatically structured according to the structure of your nested AutoIt map.
JSON is a text format. If you prefer a binary format that produces smaller files, you can also take a look at >>CBOR<<.
The operation is basically exactly the same as in the case of JSON: One call to serialize the map into a file and one to read it back from the file.

Let me give you a basic example of how this would work in your case with CBOR:

expandcollapse popup

#include "CBOR.au3"

; initial data if not exists already
If not FileExists("myData.bin") Then
    Local $mMap[]
    Local $mPerson[]

    $mPerson["no"] = 42;
    $mPerson["first"] = "Jerry"
    $mPerson["name"] = "Mouse"

    $mMap[42] =  $mPerson

    $mPerson["no"] = 43;
    $mPerson["first"] = "Tom"
    $mPerson["name"]  = "Cat"

    $mMap[43] = $mPerson

    ; save $mMap in file
    FileDelete("myData.bin")
    FileWrite("myData.bin", _cbor_encode($mMap))
EndIf

; --------- read data from file ---------------
Local $mMap = _cbor_decode(FileRead("myData.bin"))

; -------- add data (example for changing data) -------------------
Local $mPerson[]
$mPerson["no"] = 44;
$mPerson["first"] = "Bucks"
$mPerson["name"] = "Bunny"
$mMap[44] = $mPerson

; ------- save the new $mMap into file ----------
FileDelete("myData.bin")
FileWrite("myData.bin", _cbor_encode($mMap))


; ------- The next time the program is called, $mMap contains the updated data ----------


; print the data to see if ok
For $no In MapKeys($mMap)
    ConsoleWrite($no & @CRlF)

    $mP = $mMap[$no]
    For $key in MapKeys($mP)
        ConsoleWrite(@TAB & $key & ": " & $mP[$key] & @CRLF)
    Next
Next

And for JSON, the process is basically almost the same:

expandcollapse popup

#include "JSON.au3"

; initial data if not exists already
If not FileExists("myData.json") Then
    Local $mMap[]
    Local $mPerson[]

    $mPerson["no"] = "42";
    $mPerson["first"] = "Jerry"
    $mPerson["name"] = "Mouse"

    $mMap["42"] =  $mPerson

    $mPerson["no"] = 43;
    $mPerson["first"] = "Tom"
    $mPerson["name"]  = "Cat"

    $mMap["43"] = $mPerson

    ; save $mMap in file
    FileWrite("myData.json", _JSON_GenerateCompact($mMap))
EndIf

; --------- read data from file ---------------
Local $sFile = FileRead("myData.json")
Local $mMap = _JSON_Parse($sFile)

; -------- add data (example for changing data) -------------------
Local $mPerson[]
$mPerson["no"] = 44;
$mPerson["first"] = "Bucks"
$mPerson["name"] = "Bunny"
$mMap["44"] = $mPerson

; ------- save the new $mMap into file ----------
FileDelete("myData.json")
FileWrite("myData.json", _JSON_GenerateCompact($mMap))


; ------- The next time the program is called, $mMap contains the updated data ----------


; print the data to see if ok
For $no In MapKeys($mMap)
    ConsoleWrite($no & @CRlF)

    $mP = $mMap[$no]
    For $key in MapKeys($mP)
        ConsoleWrite(@TAB & $key & ": " & $mP[$key] & @CRLF)
    Next
Next

 

Edited November 20, 2023 by AspirinJunkie

[Sascha]

Since i had to much bad experience with C++/MFC and serialisation i choose the JSON example.

Thank you for your help.

 

Edited November 21, 2023 by Sascha

[ioa747]

one more approach

expandcollapse popup

; https://www.autoitscript.com/forum/topic/211135-map-data-fromto-file/?do=findComment&comment=1527073
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

#include <File.au3>

Global $mMap[], $mPerson[]

Global $sFilePath = StringTrimRight(@ScriptFullPath, 4) & ".map.csv"

If Not FileExists($sFilePath) Then ; if not exist .map.csv Make one
    $mPerson.id = 42 ;
    $mPerson.first = "Jerry"
    $mPerson.name = "Mouse"
    $mMap[$mPerson.id] = $mPerson

    $mPerson.id = 43 ;
    $mPerson.first = "Tom"
    $mPerson.name = "Cat"
    $mMap[$mPerson.id] = $mPerson

    _MapSave($mMap, $sFilePath) ; Save map to file
EndIf

$mMap = _MapLoad($sFilePath) ; load map from file

ConsoleWrite("$mMap[43].first:" & $mMap[43].first & @CRLF)

$mPerson = $mMap[43]
ConsoleWrite('$mPerson.first:' & $mPerson.first & @CRLF)
ConsoleWrite("" & @CRLF)

$mPerson.id = 44
$mPerson.first = "Daffy"
$mPerson.name = "Duck"
$mMap[$mPerson.id] = $mPerson ; add data to map


_MapSave($mMap, $sFilePath) ; Save map to file

_MapCW($mMap)

;----------------------------------------------------------------------------------------
Func _MapSave(ByRef $mMap, $sFilePath)
    Local $aMap, $mKey, $aKey, $aArray, $Data
    $aMap = MapKeys($mMap)
    If IsArray($aMap) Then
        $mKey = $mMap[$aMap[0]]
        $aKey = MapKeys($mKey)
        If IsArray($aKey) Then
            _ArrayTranspose($aKey)
            $aArray = $aKey
            Local $iRowsCnt = UBound($aMap, $UBOUND_ROWS)
            Local $iColsCnt = UBound($aArray, $UBOUND_COLUMNS)
            ReDim $aArray[$iRowsCnt + 1][$iColsCnt]
        EndIf

        For $R = 0 To $iRowsCnt - 1
            $mKey = $mMap[$aMap[$R]]
            $aKey = MapKeys($mKey)
            If IsArray($aKey) Then
                For $i = 0 To $iColsCnt - 1
                    $Data = $mKey[$aKey[$i]]
                    ;ConsoleWrite($i & ") " & $aKey[$i] & ":" & $Data & @CRLF)
                    $aArray[$R + 1][$i] = $Data
                Next
            EndIf
        Next
    EndIf
    ;_ArrayDisplay($aArray, "$aArray")
    _FileWriteFromArray($sFilePath, $aArray, Default, Default, ";")
EndFunc   ;==>_MapSave
;----------------------------------------------------------------------------------------
Func _MapLoad($sFilePath)
    Local $aArray, $Data, $mNewMap[], $mKey[]
    _FileReadToArray($sFilePath, $aArray, $FRTA_NOCOUNT, ";")
    Local $iRowsCnt = UBound($aArray, $UBOUND_ROWS)
    Local $iColsCnt = UBound($aArray, $UBOUND_COLUMNS)
    ;_ArrayDisplay($aArray, "$aArray")

    For $R = 1 To $iRowsCnt - 1
        For $C = 0 To $iColsCnt - 1
            $Data = $aArray[0][$C]
            If $C = 0 Then
                Local $iID = Int($aArray[$R][$C])
                $mKey[$Data] = $iID
            Else
                $mKey[$Data] = $aArray[$R][$C]
            EndIf
            ;ConsoleWrite("." & $Data & "=" & $mKey[$Data] & " Type: " & VarGetType($mKey[$Data]) & @CRLF)
        Next
        $mNewMap[$iID] = $mKey
    Next

    Return $mNewMap
EndFunc   ;==>_MapLoad
;----------------------------------------------------------------------------------------
Func _MapCW(ByRef $mMap)
    For $ID In MapKeys($mMap)
        Local $mKey = $mMap[$ID]
        ConsoleWrite("[" & $ID & "] ")
        For $key In MapKeys($mKey)
            ConsoleWrite($key & ":" & $mKey[$key] & @TAB)
        Next
        ConsoleWrite("" & @CRLF)
    Next
EndFunc   ;==>_MapCW
;----------------------------------------------------------------------------------------

 

Edited November 24, 2023 by ioa747
UpDate

[Sascha]

@ioa747

Thank you but i think the json version is the best for me.