dirty Posted November 27, 2011 Share Posted November 27, 2011 (edited) i have a file with 36105 lines to search for a string. enough said. it takes too long to find a string given, so expandcollapse popup#include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #Include <File.au3> #Include <GuiListView.au3> FileInstall ("Bibles\Russian.txt","Russian.txt",1) $MainGUI = GUICreate ("Bible Search",400,435) GUICtrlCreateGroup ("Search ?",5,5,390,45) $Input = GUICtrlCreateInput ("",10,20,300,20) $SearchButton = GUICtrlCreateButton ("Go !",315,20,70,20) GUICtrlCreateGroup ("Results.",5,50,390,325) $ListView = GUICtrlCreateListView ("Results of your search displayed here.|",10,65,380,300) _GUICtrlListView_SetColumnWidth ($ListView,0,375) $ShowButton = GUICtrlCreateButton ("Show Selected.",5,380,395,30) $Status = GUICtrlCreateLabel ("Whats going on ?",5,420,390,20) GUISetState (@SW_SHOW,$MainGUI) While 1 $msg = GUIGetMsg() If $msg = $GUI_EVENT_CLOSE Then FileDelete ("Russian.txt") Exit EndIf If $msg = $SearchButton Then $SearchFor = GUICtrlRead ($Input) _GUICtrlListView_DeleteAllItems(GUICtrlGetHandle($ListView)) For $i = 1 to 36105 GUICtrlSetData ($Status,"Whats going on ? Working") $SearchIn = FileReadLine ("Russian.txt",$i) If StringInStr ($SearchIn,$SearchFor) <> @error Then GUICtrlCreateListViewItem ($SearchIn,$ListView) EndIf $Loopmsg = GUIGetMsg() If $Loopmsg = $SearchButton Then MsgBox(0,'','search stoped') ExitLoop EndIf Next GUICtrlSetData ($Status,"Whats going on ? Nothing") EndIf If $msg = $ShowButton Then MsgBox(0,'',GUICtrlRead (GUICtrlRead ($ListView))) EndIf WEnd as simple as i could yet its too long. Is there a way to search faster ? I noticed searching for a whole phrase takes just as long as it takes to find a word in this phrase Edited November 30, 2011 by dirty Link to comment Share on other sites More sharing options...
Realm Posted November 27, 2011 Share Posted November 27, 2011 Hello dirty, I'm not sure if there is a limitation of how many lines can be read into an array with this function, however I know this function is faster for reading lines from a text file _FileReadToArray() this will read your text file into a 1 dimensional array with each element containing a line from your text file...then loop as you did in your example, only this time checking on your array. Largest file I read into an array was 18k lines, and looping for a check too all took only a few seconds. Depending on the severity of your check this could change the timing, however I still say that this way is faster than reading each line from the file individually My Contributions: Unix Timestamp: Calculate Unix time, or seconds since Epoch, accounting for your local timezone and daylight savings time. RegEdit Jumper: A Small & Simple interface based on Yashied's Reg Jumper Function, for searching Hives in your registry. Link to comment Share on other sites More sharing options...
guinness Posted November 27, 2011 Share Posted November 27, 2011 Why not use _FileReadToArray, because currently you're opening the file ... reading the line ... closing the file (repeat 36105 times) also the Array returned has the number of lines in the index 0 item, so you don't have to hard code the number of lines. UDF List: _AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples... Updated: 22/04/2018 Link to comment Share on other sites More sharing options...
dirty Posted November 27, 2011 Author Share Posted November 27, 2011 not friendly with arrays. can you give me example that will work with my script ? Link to comment Share on other sites More sharing options...
Realm Posted November 27, 2011 Share Posted November 27, 2011 There is an example in your help file. #include <file.au3> Dim $aRecords If Not _FileReadToArray("error.log",$aRecords) Then MsgBox(4096,"Error", " Error reading log to Array error:" & @error) Exit EndIf For $x = 1 to $aRecords[0] Msgbox(0,'Record:' & $x, $aRecords[$x]) Next My Contributions: Unix Timestamp: Calculate Unix time, or seconds since Epoch, accounting for your local timezone and daylight savings time. RegEdit Jumper: A Small & Simple interface based on Yashied's Reg Jumper Function, for searching Hives in your registry. Link to comment Share on other sites More sharing options...
guinness Posted November 27, 2011 Share Posted November 27, 2011 Look at the Example in the Help file for _FileReadToArray. UDF List: _AdapterConnections() • _AlwaysRun() • _AppMon() • _AppMonEx() • _ArrayFilter/_ArrayReduce • _BinaryBin() • _CheckMsgBox() • _CmdLineRaw() • _ContextMenu() • _ConvertLHWebColor()/_ConvertSHWebColor() • _DesktopDimensions() • _DisplayPassword() • _DotNet_Load()/_DotNet_Unload() • _Fibonacci() • _FileCompare() • _FileCompareContents() • _FileNameByHandle() • _FilePrefix/SRE() • _FindInFile() • _GetBackgroundColor()/_SetBackgroundColor() • _GetConrolID() • _GetCtrlClass() • _GetDirectoryFormat() • _GetDriveMediaType() • _GetFilename()/_GetFilenameExt() • _GetHardwareID() • _GetIP() • _GetIP_Country() • _GetOSLanguage() • _GetSavedSource() • _GetStringSize() • _GetSystemPaths() • _GetURLImage() • _GIFImage() • _GoogleWeather() • _GUICtrlCreateGroup() • _GUICtrlListBox_CreateArray() • _GUICtrlListView_CreateArray() • _GUICtrlListView_SaveCSV() • _GUICtrlListView_SaveHTML() • _GUICtrlListView_SaveTxt() • _GUICtrlListView_SaveXML() • _GUICtrlMenu_Recent() • _GUICtrlMenu_SetItemImage() • _GUICtrlTreeView_CreateArray() • _GUIDisable() • _GUIImageList_SetIconFromHandle() • _GUIRegisterMsg() • _GUISetIcon() • _Icon_Clear()/_Icon_Set() • _IdleTime() • _InetGet() • _InetGetGUI() • _InetGetProgress() • _IPDetails() • _IsFileOlder() • _IsGUID() • _IsHex() • _IsPalindrome() • _IsRegKey() • _IsStringRegExp() • _IsSystemDrive() • _IsUPX() • _IsValidType() • _IsWebColor() • _Language() • _Log() • _MicrosoftInternetConnectivity() • _MSDNDataType() • _PathFull/GetRelative/Split() • _PathSplitEx() • _PrintFromArray() • _ProgressSetMarquee() • _ReDim() • _RockPaperScissors()/_RockPaperScissorsLizardSpock() • _ScrollingCredits • _SelfDelete() • _SelfRename() • _SelfUpdate() • _SendTo() • _ShellAll() • _ShellFile() • _ShellFolder() • _SingletonHWID() • _SingletonPID() • _Startup() • _StringCompact() • _StringIsValid() • _StringRegExpMetaCharacters() • _StringReplaceWholeWord() • _StringStripChars() • _Temperature() • _TrialPeriod() • _UKToUSDate()/_USToUKDate() • _WinAPI_Create_CTL_CODE() • _WinAPI_CreateGUID() • _WMIDateStringToDate()/_DateToWMIDateString() • Au3 script parsing • AutoIt Search • AutoIt3 Portable • AutoIt3WrapperToPragma • AutoItWinGetTitle()/AutoItWinSetTitle() • Coding • DirToHTML5 • FileInstallr • FileReadLastChars() • GeoIP database • GUI - Only Close Button • GUI Examples • GUICtrlDeleteImage() • GUICtrlGetBkColor() • GUICtrlGetStyle() • GUIEvents • GUIGetBkColor() • Int_Parse() & Int_TryParse() • IsISBN() • LockFile() • Mapping CtrlIDs • OOP in AutoIt • ParseHeadersToSciTE() • PasswordValid • PasteBin • Posts Per Day • PreExpand • Protect Globals • Queue() • Resource Update • ResourcesEx • SciTE Jump • Settings INI • SHELLHOOK • Shunting-Yard • Signature Creator • Stack() • Stopwatch() • StringAddLF()/StringStripLF() • StringEOLToCRLF() • VSCROLL • WM_COPYDATA • More Examples... Updated: 22/04/2018 Link to comment Share on other sites More sharing options...
jchd Posted November 27, 2011 Share Posted November 27, 2011 Try this faster version. You can certainly go without arrays. expandcollapse popup#include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <File.au3> #include <GuiListView.au3> FileInstall("BiblesRussian.txt", "Russian.txt", 1) $MainGUI = GUICreate("Bible Search", 400, 435) GUICtrlCreateGroup("Search ?", 5, 5, 390, 45) $Input = GUICtrlCreateInput("", 10, 20, 300, 20) $SearchButton = GUICtrlCreateButton("Go !", 315, 20, 70, 20) GUICtrlCreateGroup("Results.", 5, 50, 390, 325) $ListView = GUICtrlCreateListView("Results of your search displayed here.|", 10, 65, 380, 300) _GUICtrlListView_SetColumnWidth($ListView, 0, 375) $ShowButton = GUICtrlCreateButton("Show Selected.", 5, 380, 395, 30) $Status = GUICtrlCreateLabel("Whats going on ?", 5, 420, 390, 20) Local $source = FileRead("Russian.txt") ; read the whole text at once Local $res GUISetState(@SW_SHOW, $MainGUI) While 1 $msg = GUIGetMsg() If $msg = $GUI_EVENT_CLOSE Then FileDelete("Russian.txt") Exit EndIf If $msg = $SearchButton Then $SearchFor = GUICtrlRead($Input) _GUICtrlListView_DeleteAllItems(GUICtrlGetHandle($ListView)) GUICtrlSetData($Status, "Whats going on ? Working") $SearchFor = StringRegExpReplace($SearchFor, "([.*+()?^$|{}[]])", "$1") $res = StringRegExp($source, "(?im)^.*" & $SearchFor & ".*$", 3) If Not @error Then For $line In $res GUICtrlCreateListViewItem($line, $ListView) Next EndIf GUICtrlSetData($Status, "Whats going on ? Nothing") EndIf If $msg = $ShowButton Then MsgBox(0, '', GUICtrlRead(GUICtrlRead($ListView))) EndIf WEnd This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
JohnOne Posted November 27, 2011 Share Posted November 27, 2011 (edited) I imagine even just keeping the file open until the search is complete would speed it up. This part. _GUICtrlListView_DeleteAllItems(GUICtrlGetHandle($ListView)) For $i = 1 to 36105 GUICtrlSetData ($Status,"Whats going on ? Working") $SearchIn = FileReadLine ("Russian.txt",$i) If StringInStr ($SearchIn,$SearchFor) <> @error Then GUICtrlCreateListViewItem ($SearchIn,$ListView) EndIf $Loopmsg = GUIGetMsg() If $Loopmsg = $SearchButton Then MsgBox(0,'','search stoped') ExitLoop EndIf Next To _GUICtrlListView_DeleteAllItems(GUICtrlGetHandle($ListView)) $hfile = FileOpen("Russian.txt") For $i = 1 to 36105 GUICtrlSetData ($Status,"Whats going on ? Working") $SearchIn = FileReadLine ($hfile,$i) If StringInStr ($SearchIn,$SearchFor) <> @error Then GUICtrlCreateListViewItem ($SearchIn,$ListView) EndIf $Loopmsg = GUIGetMsg() If $Loopmsg = $SearchButton Then MsgBox(0,'','search stoped') ExitLoop EndIf Next FileClose("Russian.txt") Edited November 27, 2011 by JohnOne AutoIt Absolute Beginners Require a serial Pause Script Video Tutorials by Morthawt ipify Monkey's are, like, natures humans. Link to comment Share on other sites More sharing options...
dirty Posted November 27, 2011 Author Share Posted November 27, 2011 (edited) Try this faster version. You can certainly go without arrays. Man this was instant result. But these two lines $SearchFor = [url="../../../autoit3/docs/functions/StringRegExpReplace.htm"]StringRegExpReplace[/url]($SearchFor, "([.*+()?^$|{}[]])", "$1") $res = [url="../../../autoit3/docs/functions/StringRegExp.htm"]StringRegExp[/url]($source, "(?im)^.*" & $SearchFor & ".*$", 3) driving me nuts !. They are as complex as your avatar. Speaking of me being stupid, how come there is no smiley face for being stupid ? It would be very appropriate in my case. Edited November 27, 2011 by dirty Link to comment Share on other sites More sharing options...
jchd Posted November 27, 2011 Share Posted November 27, 2011 (edited) They are not as complex as they seem.The speed is gained by loading the whole file in memory once for all. Then using a Regular Expression (RE) to extract all lines which contain what the user looks for.You were talking about 36K lines, so with 100 characters/line it's just a 3.6Mb block. I tested it with 50Mb files on a slow laptop and that still gave decent results.The first line you quote is needed due to the fact that the search string may contain special characters which the RE engine would normally interpret as "commands". This is achived by "escaping" them all (prefixing them with a backslash).The second statement performs the actual extraction of every line which contain, from the start of line followed by the search string followed by followed by an end of line or end of text.BTW if you better wish to ignore punctuation in the search string, it's possible to change that very easily. Just say so. Edited November 27, 2011 by jchd This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
dirty Posted November 28, 2011 Author Share Posted November 28, 2011 (edited) Using your example: How do i step back each time a * is found to previously defined chapter=#? For example my lisview has 4 columns. $SearchFor = StringRegExpReplace($SearchFor, "([.*+()?^$|{}[]])", "$1") $res = StringRegExp($source, "(?im)^.*" & $SearchFor & ".*$", 3) If Not @error Then For $line In $res $NewLine = StringTrimLeft ($line,StringInStr($line,"*")) ;becuase each line starts with Versus=1* or Versus=2* etc so i remove that to display in first column. GUICtrlCreateListViewItem($NewLine & "|Chapter|Book|Testament|", $ListView) Next EndIf Here with line generated by (your doing) i can trim it by finding special character i placed in each line * Because this is a bible, lines look like this Versus=30* а всем зверям земным, и всем птицам небесным, и всякому [гаду,] пресмыкающемуся по земле, в котором душа живая, дал Я всю зелень травную в пищу. И стало так. Versus=31* И увидел Бог все, что Он создал, и вот, хорошо весьма. И был вечер, и было утро: день шестой. @Chapter=2` Versus=1* Так совершены небо и земля и все воинство их. Versus=2* И совершил Бог к седьмому дню дела Свои, которые I somehow need to assign Chapter # into the second column of listview Note that each @Chapter=# and has a special character at the end ` and @ at the beginning to help me to find it, but with your code i cant really figure out how to go back to this symbol and read whats behind it (kinda go backwards i guess) I also dont understand why did u use $SearchFor twice ? Maybe thats how it works, i dont know Edited November 28, 2011 by dirty Link to comment Share on other sites More sharing options...
jchd Posted November 28, 2011 Share Posted November 28, 2011 Here's a version showing found text, verse, chapter, book & testament.As explained in my previous post, I use StringRegExpReplace to replace several special characters in what the user searches for to keep them from being intrepreted wrongly by the actual search. This replacement is very fast as it only deals with the text entered by the user, not on the whole Bible text.I need to make you aware of something that may be important to you: the search expression is performing caseless (that's the i in (?ims) options). BUT the search engine AutoIt currently uses only cares of lower ASCII (the 127 first characters in the ANSI table). There is a possibility to search for Unicode text while taking care of non-english letters (over the most used part of Unicode) but this compile option is not currently usable in AutoIt.Thus I'm unsure how caseless search will work for you. It will depend of your file being ANSI or UTF-8 (with BOM), which codepage you use if using ANSI. If that causes havoc, say so and I'll do something to keep it efficient and simple, but your users will have to use a russian codepage. This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
dirty Posted November 28, 2011 Author Share Posted November 28, 2011 (edited) Search works fine both for english and russian. by looking at $SearchFor = StringRegExpReplace($SearchFor, "([.*+()?^$|{}[]])", "$1") $res = StringRegExp($source, "(?im)^.*" & $SearchFor & ".*$", 3) i cant figure out how to make it find "@" that represents the beginning of the line with Chapter#` $line returns matching line of text u searched for and $res returns nothing at all. So i was thinking about doing it this way. $LastFoundChapter = "" ;declare previously so it gets changed when found $SearchInput = GUICtrlRead($Input) $SearchFor = StringRegExpReplace($SearchInput, "([.*+()?^$|{}[]])", "$1") $res = StringRegExp($source, "(?im)^.*" & $SearchFor & ".*$", 3) If Not @error Then For $line In $res If $line = "@" Then Assign ("LastFoundChapter",$line) ;when chapter line is found, assign it to the variable $NewLine = StringTrimLeft ($line,StringInStr($line,"*")) ;remove Versus=#* from the line to make it clean GUICtrlCreateListViewItem($NewLine & "|" & $LastFoundChapter & "|Book|Testament|", $ListView) ;write to listview but i cant figure out how to find the @ while looking for what is in the $SearchInput Edited November 28, 2011 by dirty Link to comment Share on other sites More sharing options...
jchd Posted November 28, 2011 Share Posted November 28, 2011 Ooops, I seem to have left the example code away from the post... Apologies for that. expandcollapse popup#include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <File.au3> #include <GuiListView.au3> FileInstall("BiblesRussian.txt", "Russian.txt", 1) $MainGUI = GUICreate("Bible Search", 400, 435) GUICtrlCreateGroup("Search ?", 5, 5, 390, 45) $Input = GUICtrlCreateInput("", 10, 20, 300, 20) $SearchButton = GUICtrlCreateButton("Go !", 315, 20, 70, 20) GUICtrlCreateGroup("Results.", 5, 50, 390, 325) $ListView = GUICtrlCreateListView("Results of your search displayed here.|Verse|Chapter|Book|Testament", 10, 65, 380, 300) _GUICtrlListView_SetColumnWidth($ListView, 0, 375) $ShowButton = GUICtrlCreateButton("Show Selected.", 5, 380, 395, 30) $Status = GUICtrlCreateLabel("Whats going on ?", 5, 420, 390, 20) Local $source = FileRead("Russian.txt") ; read the whole text at once Local $res Local $book = "Paul", $testament = "New" ; I don't know where you get this from GUISetState(@SW_SHOW, $MainGUI) While 1 $msg = GUIGetMsg() If $msg = $GUI_EVENT_CLOSE Then ;~ FileDelete("Russian.txt") Exit EndIf If $msg = $SearchButton Then $SearchFor = GUICtrlRead($Input) _GUICtrlListView_DeleteAllItems(GUICtrlGetHandle($ListView)) GUICtrlSetData($Status, "Whats going on ? Working") $SearchFor = StringRegExpReplace($SearchFor, "([.*+()?^$|{}[]])", "$1") $res = StringRegExp($source, "(?ims)@chapter=(d+)`.*?(?-s)^versus=(d+)*s(.*?" & $SearchFor & ".*?)$", 3) If Not @error Then For $i = 0 To UBound($res) - 1 Step 3 GUICtrlCreateListViewItem($res[$i + 2] & '|' & $res[$i + 1] & '|' & $res[$i] & '|' & $book & '|' & $testament, $ListView) Next EndIf GUICtrlSetData($Status, "Whats going on ? Nothing") EndIf If $msg = $ShowButton Then MsgBox(0, '', GUICtrlRead(GUICtrlRead($ListView))) EndIf WEnd This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
dirty Posted November 29, 2011 Author Share Posted November 29, 2011 (edited) Man i really appreciate you helping me out I attached English version if the file for you to look at. Maybe this will help you to help me how to get this right. English.rar Right now i have no clue why you predefine local $book, $testament = "New" and how would they work. Those are being located in the process of a search, kinda like searching for 4 things at same time. For example: 1st you search for what ever text you need. Once found, look back and see what chapter it belongs to, then what book and then the testament. I guess i could change each line in the text file from this: Versus=1* In the beginning God created the heaven and the earth. into: Book=Genesis Chapter=1 Versus=1* In the beginning God created the heaven and the earth. but it will take me forever. With help of autoit i can probably make that a matter of a day of coding and few hours of text processing (with my level of experience working with strings) This is how i replaced 1 In the beginning God created the heaven and the earth. with Versus=1* In the beginning God created the heaven and the earth. #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <File.au3> #Include <GuiStatusBar.au3> $MainGUI = GUICreate("Bible Search", 400, 435) $SearchButton = GUICtrlCreateButton("Go !", 315, 20, 70, 20) GUICtrlCreateGroup("Results.", 5, 50, 390, 325) $ListView = GUICtrlCreateEdit("", 10, 65, 380, 300) GUISetState(@SW_SHOW, $MainGUI) While 1 $msg = GUIGetMsg() If $msg = $GUI_EVENT_CLOSE Then Exit EndIf If $msg = $SearchButton Then For $i = 1 to 150 ;150 because psalms have 150 versus $source = FileRead("English.txt") $Replaced = StringReplace ($source, @CRLF & "CHAPTER " & $i & @CRLF,@CRLF & "@Chapter=" & $i & "`" & @CRLF) GUICtrlSetData($ListView,$Replaced) FileClose ($source) _GUICtrlStatusBar_SetText ($StatusBar,"Going "& $i) $OpenToWrite = FileOpen ("English.txt",2+128) FileWrite ($OpenToWrite,$Replaced) FileClose ($OpenToWrite) Next FileClose ("English.txt") EndIf WEnd This loops 150 times just to change 1 to Versus=1* so imagine how long its gonna take me to change it to: Book=Genesis Chapter=1 Versus=1* In the beginning God created the heaven and the earth. The good thing is, by the time its all over i will probably play guitar like a pro Unless you have a better plan ? Because i still have no clue (sorry for being dumb) how your script works and u are my only hope Edited November 29, 2011 by dirty Link to comment Share on other sites More sharing options...
jchd Posted November 29, 2011 Share Posted November 29, 2011 Your biggest problem is in the file format you use. You definitely need a better one, as you correctly identified. First, split things in two: Bible and NT to ease processing. Then we'll index each versus with a short complete reference as you devised, but there is no need to be unduly verbose here. Imagine you abbreviate things this way: BI = Bible NT = New Testament GEN = Genesis ... MAT = Mathiew PAU = Paul ... Affect every book a unique three-letter acronym (preferably using uppercase english letters). This will be used for any translation. Then we'll write a one-time utility (once for every language file) to produce two files having the required format, something like: BI,GEN,1,1§In the beginning God created the heaven and the earth. NT,MAT,3,14§But John forbad him, saying, I have need to be baptized of thee, and comest thou to me? Once done (it'll be very fast) we merge both files into one. Then I'll change the search script to find text and fill in references in a much more simple and efficient way. The idea is that extracting matching complete lines with this reference convention is easy, but "going backward" (there is no going backward with REs) to find chapter and book and testament is utterly slow, up to being worst than every other approach when searching very common substrings like 'he', due to RE backtraking intensely. Does that sound reasonable to you? This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
dirty Posted November 29, 2011 Author Share Posted November 29, 2011 (there is no going backward with REs)No wonder reverse engineering is a science I will start formatting text files, but wouldnt it be easier to have whole name of a testament and book instead of abbreviated one ?This way we wont have to rename NT,MAT,3,14§ into NEW TESTAMENT,MATHEW,3,14§ or how were you gonna go about setting whole name into listview column i dont know.Let me know before i start Link to comment Share on other sites More sharing options...
dirty Posted November 29, 2011 Author Share Posted November 29, 2011 Ok i got it Here is how i did it so i could understand. $SearchFor = StringRegExpReplace($SearchInput, "([.*+()?^$|{}[]])", "$1") $res = StringRegExp($source, "(?im)^.*" & $SearchFor & ".*$", 3) If Not @error Then For $line In $res $Loopmsg = GUIGetMsg() If $Loopmsg = $SearchButton Then GUICtrlSetData ($SearchButton,"Go !") ExitLoop EndIf $NewLine = StringTrimLeft ($line,StringInStr($line,"*")) $Chapter = StringLeft ($line,StringInStr($line,",","",3)) $TrimmedChapter = StringTrimLeft ($Chapter,StringInStr ($Chapter,"=",'',3)) $FixedChapter = StringTrimRight ($TrimmedChapter,1) $Book = StringLeft ($line,StringInStr($line,",","",2)) $TrimmedBook = StringTrimLeft ($Book,StringInStr ($Book,"=",'',2)) $FixedBook = StringTrimRight ($TrimmedBook,1) $Testament = StringLeft ($line,StringInStr($line,",","",1)) $TrimmedTestament = StringTrimLeft ($Testament,StringInStr ($Testament,"=")) $FixedTestament = StringTrimRight ($TrimmedTestament,1) GUICtrlCreateListViewItem($NewLine & "|" & $FixedChapter & "|" & $FixedBook & "|" & $FixedTestament & "|", $ListView) Assign ("Found",$Found+1) Next EndIf This is how each line of text looks like T=ВЕТХИЙ ЗАВЕТ,B=БЫТИЕ,C=1,V=1* В начале сотворил Бог небо и землю. So from here i simply count the instance of an = and trim it twise. I know trimming twice could be done once somehow, but hey it works like a champ ! all thanks to you Link to comment Share on other sites More sharing options...
dirty Posted November 30, 2011 Author Share Posted November 30, 2011 (edited) One perhaps last question:How would you make the advanced search to work with your example ? I mean for instance searching for:"hell" will also return hellicopter as well as hellicoptering and hellicapture and last but not least helliscloriousis.Is it possible to change your part of the code to search differently if * is provided ?For example *hell returns anystringbeforhell or hell* returns hellanystringafterhell.Right now it works like both, *hell* and returns anystringbeforhellanystringafterhell.So i was thinking to narrow the search result down by exact match like google does, for example"hell" will return only those lines that will have the word hell in them and ignore those with hellium and hellerious by wrapping a word into quotes.I had that feature in my other project managment app and it works like this:$GetNameFromGUI = GUICtrlRead($SearchByName) ;Name. Can be "Name" $Part1 = StringTrimLeft($GetNameFromGUI, 1) ;removes 1 quote and returns Name" $Part2 = StringTrimRight($Part1, 1) ;removes another quote from Name" and returns Name If $GetNameFromGUI = ('"' & $Part2 & '"') Then ;If $GetNameFromGUI had quotes surrounding it then ;whatever code to display it on gui or elsewhere using $GetNameFromResult Else ;whatever code to display it on gui or elsewhere using $GetNameFromResult EndIfIts funny how now that i am looking at it, i dont understand it but am sure you get the idea.In your case i dont know how to have that thing work the same way. Edited November 30, 2011 by dirty Link to comment Share on other sites More sharing options...
jchd Posted November 30, 2011 Share Posted November 30, 2011 (edited) Try this improved version which uses any * character in the searched string just like a classical DOS/Windows joker. Whith it, you can search *e*h*ee* and have all occurences of Bethleem appear enclosed in double quotes (and possibly other word matches). expandcollapse popup#include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <File.au3> #include <GuiListView.au3> FileInstall("BiblesRussian.txt", "Russian.txt", 1) $MainGUI = GUICreate("Bible Search", 400, 435) GUICtrlCreateGroup("Search ?", 5, 5, 390, 45) $Input = GUICtrlCreateInput("", 10, 20, 300, 20) $SearchButton = GUICtrlCreateButton("Go !", 315, 20, 70, 20) GUICtrlCreateGroup("Results.", 5, 50, 390, 325) $ListView = GUICtrlCreateListView("Results of your search displayed here.|Versus|Chapter|Book|Testament", 10, 65, 380, 300) _GUICtrlListView_SetColumnWidth($ListView, 0, 375) $ShowButton = GUICtrlCreateButton("Show Selected.", 5, 380, 395, 30) $Status = GUICtrlCreateLabel("Whats going on ?", 5, 420, 390, 20) Local $source = FileRead("Russian.txt") ; read the whole text at once Local $res, $refs, $match, $Found, $Versustext, $Versus, $Chapter, $book, $testament GUISetState(@SW_SHOW, $MainGUI) While 1 $msg = GUIGetMsg() If $msg = $GUI_EVENT_CLOSE Then ;~ FileDelete("Russian.txt") Exit EndIf If $msg = $SearchButton Then $SearchFor = GUICtrlRead($Input) _GUICtrlListView_DeleteAllItems(GUICtrlGetHandle($ListView)) GUICtrlSetData($Status, "Whats going on ? Working") $SearchFor = StringReplace($SearchFor, "*", "§") $SearchFor = StringRegExpReplace($SearchFor, "([.*+()?^$|{}[]])", "$1") $SearchFor = "b" & StringReplace($SearchFor, "§", ".*?") & "b" $res = StringRegExp($source, "(?im)^.*" & $SearchFor & ".*$", 3) If Not @error Then $Found = UBound($res) For $line In $res $Loopmsg = GUIGetMsg() ; If $Loopmsg = $SearchButton Then ; GUICtrlSetData($SearchButton, "Go !") ; what purpose does that serve ? ExitLoop ; EndIf ; $refs = StringRegExp($line, "(?i)^s*ts*=s*([^,]*)s*,s*bs*=s*([^,]*)s*,s*cs*=s*(d*)s*,s*vs*=s*(d*)*s*(.*)", 1) $testament = $refs[0] $book = $refs[1] $Chapter = $refs[2] $Versus = $refs[3] $Versustext = StringRegExpReplace($refs[4], "(?i)(" & $SearchFor & ")", '"$1"') GUICtrlCreateListViewItem($Versustext & "|" & $Versus & "|" & $Chapter & "|" & $book & "|" & $testament & "|", $ListView) Next EndIf GUICtrlSetData($Status, "Whats going on ? Nothing") EndIf If $msg = $ShowButton Then MsgBox(0, '', GUICtrlRead(GUICtrlRead($ListView))) EndIf WEnd Edited November 30, 2011 by jchd This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now