FileReadLine() - Start Line Offset

[Xandy]

 

Hi, I'd like to select a random line from file, then read up to 5 lines from that line point.

I tried FileReadLine($file, $line), to seek the first line.

Then after, FileReadLine($file) but I'm pretty sure the second FileReadLine($file) started from the beginning of file.

This code works, but could the seek line point be written better?

$file = FileOpen($filename)

If $file = -1 Then ConsoleWrite(@CRLF & "! _get_file_line: File not found: " & @CRLF & $filename)

; Seek line in file
For $i = 1 To $line - 1
    ; Dump lines (not sure better method yet)
    FileReadLine($file)
    ;$dump = FileReadLine($file)
    ;ConsoleWrite(@CRLF & "! Dump: " & $dump)
Next

; Hopefully read from offset just assigned above
For $i = 0 To 4; last one + 0..4 is 5
    $udv_val_return = FileReadLine($file)
    ;udv_value_set($command_list_id, $aCommand_list[$command_list_id][$list_cursor][$eCommand_list_udv_assign_file_get_line_udv_return_1 + $i], _
                ;$udv_val_return, $aUdv, $aUdv_local)
Next

FileClose($file)

 

[JLogan3o13]

Could you not read the file to an array, and then start at your random position? Something like this?

#include <File.au3>

Local $sFile = @DesktopDir & "\Test.txt"
Local $aArray = FileReadToArray($sFile)
Local $x = Random(0, UBound($aArray), 1)

    For $i = 0 To 4
        ConsoleWrite("Line " & $x & ": " & $aArray[$x] & @CRLF)
        $x += 1
    Next

 

[Xandy]

I should have thought of that.  I will compare the speed of our codes.  Thank you.

Edited December 5, 2016 by Xandy

[Xandy]

The higher the line number being seeked, the longer my method should take.

I wanted to compare dumping the file data line by line to the FileReadToArray()

From the following data it appears that dumping the data in my example is faster for large files if the line to be retrieved is significantly less than the total lines in file.

When the line(s) retrieved are about the total lines in file then the timing is about the same for both.  FileReadToArray might have been slightly faster at 500k lines it's so close it's hard to be sure.

FileReadToArray() seems to be worst case every time b/c it seeks the LF or CRLF for all lines every time.

These timings start at, line and read 15 x and y points from file.

These are the times each of the 15 coords took to retrieve from file:

Dump lines to seek method [ My example at first post ]
line: 250000 / 500000
---------------------- Times
1459.10332513993
1499.48895421612
1576.23103861955
1587.53221782644
1492.61088519865
1473.81215661321
1454.04440561778
1495.82207952515
1487.9765189388
1471.21202302181
1440.58576103469
1429.12665005534
1463.78106523067
1447.2362450101
1450.14127111859

line: 499970 / 500000
----------------------
3168.49993391869
3145.74117457355
3090.34360748826
3084.06231135573
2983.58581155244 
3053.37915310809
3018.82781456616
2964.68171007196
2952.84754302556
2947.06989361165
3069.15191913893
2925.46156134565
2961.19853631181
2911.42936163416
2914.15298307349

--xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Times - FileToArray method.txt [ JLogan's Example at second post ]
line: 250000 / 500000
----------------------
2837.21290988908
2807.74932170983
2835.92241085019
2801.38229847625
2810.5558637082
2795.08212197027
2755.67163866843
2648.07647265637
2673.03173382552
2628.9537160402
2675.16087864155
2615.96606023247
2613.23376402697
2624.87351425494
2590.68217850934

line: 499970 / 500000
----------------------
2790.41662860826
2815.86073836548
2917.43153440795
2762.32799086802
2861.17057294789
2815.50634865314
2663.96022466624
2713.36454888409
2697.13788847006
2694.68930814657
2648.51327264739
2651.66195762938
2630.77567207751
Subscript outta range but you get the idea

 

 

Example Line Data File:

#include <File.au3>
$filename = @ScriptDir & "\Big File.txt"
$file = FileOpen($filename, $FO_OVERWRITE)
For $i = 1 To 500000
    FileWriteLine($file, "Line " & $i & " abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
Next
FileClose($file)

I pasted my x, y data to the line I started reading from.

Line 499969 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
1
1
725
1
1075
1
1425
1
725
352
1075
352
1425
352
1
703
351
703
703
703
1053
703
1403
703
2300
703
2651
703
3001
703

 

Edited December 5, 2016 by Xandy

[Xandy]

Both methods have their uses.  I suppose if I want better speeds I should provide a open, close file option.  Or ability to store and index arrays in a User configurable way.

[JLogan3o13]

First to agree that FileOpen is going to be faster on the larger files. I would say had you pointed out you were going for speed in large files in your initial post I would have stayed away from suggesting filereadtoarray.

[Xandy]

My sincerest apologies.  It's a mess up here in my head.  I am thankful for your suggestion, I wanted to start somewhere.