Search the Community

Dear community, I would like to ask what is the fast possible way to get or set specific data file. file type doesn't matter. file should contains around 100,000 multiple row tables. time to get or set info should be close to 0ms to avoid lagging main script. Example how should works such as script. Received task to find table with user name: "Tom" Read file for user table and return it. Manage user table: add, remove, replace values. Save edited table to file. I already tested and this can't be used. FileRead FileWrite FileReadToArray ==> 10000ms Microsoft.XMLDOM ==> 100ms Msxml2.DOMDocument.6.0 ==> 25ms If you have an idea what could help please post simple example. Solution: Use XML file to store users data Transfer it from XML to dictionary Manage add/edit/remove inside dictionary Save results to file XML every some time. Warring! This is not real-time read/write this is memory reading if you need other solution read whole thread ppls post there other solutions. ;---------------------------------------------------------- ; Config for script. ;---------------------------------------------------------- Global $TIME = 0 Global $FILE = "users.xml" ; Name of file with users. Global $PATH = @ScriptDir & "\" & $FILE ; Path where we create users file xml Global $TOTAL = 100000 ; Amount of users to create. ObjEvent("AutoIt.Error", "error") ; Register COM event handler. ;---------------------------------------------------------- ; Create satabase file. ;---------------------------------------------------------- setTime("Creating database xml with " & $TOTAL & " users..") createXMLDatabase($PATH, $TOTAL) print("Database has been created.") getTimeDiff() ;---------------------------------------------------------- ; Transfer satabase from xml file to dictionary. ;---------------------------------------------------------- setTime("Transfering Database from xml file to dictionary it take some time don't close script..") Dim $base = transferXMLToDictionary($FILE) Dim $xmlbase = $base[0] Dim $database = $base[1] print("Transfer has end.") getTimeDiff() ;---------------------------------------------------------- ; Search database for user. ;---------------------------------------------------------- setTime("Serching dictionary for user nr " & $TOTAL & "..") Dim $user = getDatabaseUser("User" & $TOTAL, $database) getTimeDiff() If $user <> -1 Then print("User has been found.") For $i = 0 To UBound($user) - 1 print($user[$i]) Next Else print("User not found.") EndIf ;---------------------------------------------------------- ; Change user name for Ascer. ;---------------------------------------------------------- setTime("Change user name for Ascer..") $user[0] = "Ascer" $database.item("User" & $TOTAL) = $user getTimeDiff() print("Changed user name for Ascer") ;---------------------------------------------------------- ; Save database to file. ;---------------------------------------------------------- setTime("Save database to file xml..") saveDatabaseToXML($path, $xmlbase, $database) print("Saved database do file.") getTimeDiff() #Region 1.1, Functions. Func saveDatabaseToXML($path, ByRef $xmlbase, ByRef $database) Local $xml, $row $xml &= '<?xml version="1.0" encoding="UTF-8" standalone="no"?>' & @CRLF $xml &= '<users>' & @CRLF For $user in $database.keys $row = $database.item($user) $xml &= @TAB & '<user name="' & $row[0] & '" passwd="' & $row[1] & '" range="' & $row[2] & '" group="' & $row[3] & '"/>' & @CRLF Next $xml &= "</users>" FileDelete($path) FileWrite($path, $xml) EndFunc ;==> Save database do file xml. Func getDatabaseUser($name, ByRef $handle) Local $user = $handle.item($name) If IsArray($user) Then Return $user Return -1 EndFunc ;==> Return info about user in database or -1 if not found. Func transferXMLToDictionary($file) Local $msxml, $users, $database, $name $msxml = ObjCreate('MSXML2.DOMDocument.6.0') $msxml.load($file) $users = $msxml.SelectNodes("//users/user") $database = ObjCreate("Scripting.Dictionary") For $user In $users $name = $user.getAttribute("name") Local $table = [$name, $user.getAttribute("passwd"), $user.getAttribute("range"), $user.getAttribute("group")] $database.add($name, $table) Next Local $retn = [$msxml, $database] Return $retn EndFunc ;==> Read path for file and export it to dictionary. Return array with [0] - xml handle, [1] - dictionary handle. Func createXMLDatabase($path, $amount) Local $xml $xml &= '<?xml version="1.0" encoding="UTF-8" standalone="no"?>' & @CRLF $xml &= '<users>' & @CRLF For $i = 1 To $amount $xml &= @TAB & '<user name="User' & $i & '" passwd="123456" range="6" group="none"/>' & @CRLF Next $xml &= "</users>" FileDelete($path) FileWrite($path, $xml) EndFunc ;==> Create database xml with users Func error() Return -1 EndFunc Func setTime($msg="") print("setTime: " & $msg) $TIME = TimerInit() EndFunc Func getTimeDiff() print("getTimeDiff: " & Int(TimerDiff($TIME) * 100) / 100 & "ms") EndFunc Func print($data) Return ConsoleWrite($data & @CRLF) EndFunc ;==> Wpisuje wartość + znak nowej linji do SCITE.print() #EndRegion Sped results setTime: Creating database xml with 100000 users.. Database has been created. getTimeDiff: 386.17ms setTime: Transfering Database from xml file to dictionary it take some time don't close script.. Transfer has end. getTimeDiff: 11302.85ms setTime: Serching dictionary for user nr 100000.. getTimeDiff: 0.06ms User has been found. User100000 123456 6 none setTime: Change user name for Ascer.. getTimeDiff: 0.03ms Changed user name for Ascer setTime: Save database to file xml.. Saved database do file. getTimeDiff: 2063.39ms