_FileCountLines error?

[PoorLuzer]

Guys

_FileCountLines does not seem to count lines correctly.

For example, it's usually off by 1 line, as in the attached example.

I have come up with some code to report line numbers correctly - and it seems to work.

Have I found a bug in _FileCountLines?

If so, is my solution correct for all cases?

I would be very happy if the gurus can optimize the code.

LineCountTest.zip

_LineCount.au3

test.txt

[Andreik]

Guys

_FileCountLines does not seem to count lines correctly.

For example, it's usually off by 1 line, as in the attached example.

I have come up with some code to report line numbers correctly - and it seems to work.

Have I found a bug in _FileCountLines?

If so, is my solution correct for all cases?

I would be very happy if the gurus can optimize the code.

_FileCountLines returns 22 that is number of lines from test.txt if you open the file with notepad

Edited September 3, 2008 by Andreik

[MrCreatoR]

Yep, _FileCountLines() return incorrect results.

I think there is two ways of calculate lines in file reliably:

expandcollapse popup

;1)

ConsoleWrite(_FileCountLines("test.txt"))

Func _FileCountLines($sFilePath)
    Local $sFRead = FileRead($sFilePath)
    Local $iCountLines = 0
    
    $sFRead = StringReplace($sFRead, @CRLF, "")
    $iCountLines += @extended
    
    $sFRead = StringReplace($sFRead, @CR, "")
    $iCountLines += @extended
    
    StringReplace($sFRead, @LF, "")
    $iCountLines += @extended
    
    Return $iCountLines + 1
EndFunc


;2) - Slower due to line by line file reading

ConsoleWrite(_FileCountLines2("test.txt"))

Func _FileCountLines2($sFilePath)
    Local $hFOpen = FileOpen($sFilePath, 0)
    Local $iCountLines = 0
    
    If $hFOpen = -1 Then Return SetError(1, 0, 0)
    
    While 1
        FileReadLine($hFOpen)
        If @error = -1 Then ExitLoop
        
        $iCountLines += 1
    WEnd
    
    FileClose($hFOpen)
    
    Return $iCountLines
EndFunc

Edited September 3, 2008 by MrCreatoR

[MrCreatoR]

Btw, It's has been reported as bug a month ago.

[PoorLuzer]

_FileCountLines returns 22 that is number of lines from test.txt if you open the file with notepad

Andreik,

You are correct! Notepad too has a bug then !

Ha ha ha ha...

Makes me wonder if I am wrong

Am I ?

[PoorLuzer]

Btw, It's has been reported as bug a month ago.

Oops.. I should have searched the bug database :-(

Anyways.

I tried both the functions suggested (that is you first function - not the second), and Xenobiologist's suggested fix.

Both seem to be buggy.

So far only my function seems to report the number of lines in a file correctly.

I have attached sample test files and au3 code to test the functions out.

It seems counting lines in a file is hard!

If I wrote C++ code to do this, what are the chances that the code reflects in AutoIt?

_LineCount.au3

test.txt

trailing.txt

[MrCreatoR]

Both seem to be buggy.

What so buggy in these funcs? It's return correctly the number of lines, with the text file that you attached in the first post.

P.S

How do you check the real number of lines?

Edited September 4, 2008 by MrCreatoR

[MrCreatoR]

I check the example in your last post, and all ok, my functions return correct results, it's just needs a little errors checking, that's it

$iCountLines = _FileCountLinesEx("Test.txt")
$sCL_Results = StringFormat("Lines = %i\n@error = %i", $iCountLines, @error)

ConsoleWrite($sCL_Results)

Func _FileCountLinesEx($sFilePath)
    Local $iCountLines = 0
    Local $sFRead = FileRead($sFilePath)
    
    If @error = -1 Then Return SetError(1, 0, 0) ;The file is empty
    If @error <> 0 Then Return SetError(2, 0, 0) ;File not exists or other "Read" error.
    
    $sFRead = StringReplace($sFRead, @CRLF, "")
    $iCountLines += @extended
    
    $sFRead = StringReplace($sFRead, @CR, "")
    $iCountLines += @extended
    
    StringReplace($sFRead, @LF, "")
    $iCountLines += @extended
    
    Return $iCountLines + 1
EndFunc

[PoorLuzer]

I check the example in your last post, and all ok, my functions return correct results, it's just needs a little errors checking, that's it

EXACTLY!

He he ;-)

But you still can make the function better - can't you?

Why don't we have two functions in AutoIt:

1. Return the count of certain specified characters in a string

2. File positioning (like the real Win32 ReadFile?)

Also, were I to code this function in C++, what are the chances it would be integrated into AutoIt?

[Xenobiologist]

Hi,

try your luck by sending the code to Jos or Gary or maybe open the bug report and post your code and your problem with the fix there.

Mega

P.S.: text.txt has 23 lines but you count 24 ?

[PoorLuzer]

P.S.: text.txt has 23 lines but you count 24 ?

Hi

No.. it's reporting as 23. I think the file got changed on your end.

MrCreator's and mine are both calculating line counts correctly for all possible cases - would love to stand corrected though.

The cool thing is, MrCreator's code will work on test files from Windows, UNIX and MacOS whereas mine will only work for the first two types.

Your function is the one ending with X.

I am attaching a screenshot of a run on the test file to clear up any confusions.

Edited September 4, 2008 by PoorLuzer

[Xenobiologist]

Hi,

das this help?

; #FUNCTION# ====================================================================================================================
; Name...........: _FileCountLines
; Description ...: Returns the number of lines in the specified file.
; Syntax.........: _FileCountLines($sFilePath)
; Parameters ....: $sFilePath - Path and filename of the file to be read
; Return values .: Success - Returns number of lines in the file.
;                  Failure - Returns a 0
;                  @Error  - 0 = No error.
;                  |1 = File cannot be opened or found.
;                  |2 = Unable to Split the file
; Author ........: Tylo <tylo at start dot no>
; Modified.......: Xenobiologist
; Remarks .......: It does not count a final @LF as a line.
; Related .......:
; Link ..........;
; Example .......; Yes
; ===============================================================================================================================

Func _FileCountLinesX($sFilePath)
    Local $hFile, $sFileContent, $aTmp
    $hFile = FileOpen($sFilePath, 0)
    If $hFile = -1 Then Return SetError(1, 0, 0)
    $sFileContent = FileRead($hFile, FileGetSize($sFilePath))
    FileClose($hFile)
    If StringInStr($sFileContent, @LF) Then
        $aTmp = StringSplit(StringStripCR($sFileContent), @LF)
    ElseIf StringInStr($sFileContent, @CR) Then
        $aTmp = StringSplit($sFileContent, @CR)
    Else
        If StringLen($sFileContent) > 0 Then Return 1
        Return SetError(2, 0, 0)
    EndIf
    Return $aTmp[0]
EndFunc   ;==>__FileCountLines

Mega

[PoorLuzer]

Hi,

das this help?

Nopes - same output as previous screenshot.

You can try it out yourself.

In case of any file transfer errors. try the test.txt from the zip file in the first post.

[Xenobiologist]

Hi,

changed the script to easily show the results

expandcollapse popup

#include <File.au3>
#include <Array.au3>

ConsoleWrite('!--- expected 0' & @CRLF)
ConsoleWrite("Number of lines - null" & @LF & " LineCount - " & LineCount("null") & @LF & "_FileCountLines - " & _FileCountLines("null") & @LF & "_FileCountLinesMrCreatoR - " & _FileCountLinesMrCreatoR("null") & @LF & "_FileCountLinesX - " & _FileCountLinesX("null") & @CRLF);
ConsoleWrite('!--- expected 0' & @CRLF)
ConsoleWrite("Number of lines - zero.txt" & @LF & " LineCount - " & LineCount("zero.txt") & @LF & "_FileCountLines - " & _FileCountLines("zero.txt") & @LF & "_FileCountLinesMrCreatoR - " & _FileCountLinesMrCreatoR("zero.txt") & @LF & "_FileCountLinesX - " & _FileCountLinesX("zero.txt") & @CRLF);
ConsoleWrite('!--- expected 23' & @CRLF)
ConsoleWrite("Number of lines - test.txt " & @LF & "LineCount - " & LineCount("test.txt") & @LF & "_FileCountLines - " & _FileCountLines("test.txt") & @LF & "_FileCountLinesMrCreatoR - " & _FileCountLinesMrCreatoR("test.txt") & @LF & "_FileCountLinesX - " & _FileCountLinesX("test.txt") & @CRLF);
ConsoleWrite('!--- expected 1' & @CRLF)
ConsoleWrite("Number of lines - trailing.txt" & @LF & "LineCount - " & LineCount("trailing.txt") & @LF & "_FileCountLines - " & _FileCountLines("trailing.txt") & @LF & "_FileCountLinesMrCreatoR - " & _FileCountLinesMrCreatoR("trailing.txt") & @LF & "_FileCountLinesX - " & _FileCountLinesX("trailing.txt") & @CRLF);

; #FUNCTION# ====================================================================================================================
; Name...........: _FileCountLines
; Description ...: Returns the number of lines in the specified file.
; Syntax.........: _FileCountLines($sFilePath)
; Parameters ....: $sFilePath - Path and filename of the file to be read
; Return values .: Success - Returns number of lines in the file.
;                  Failure - Returns a 0
;                  @Error  - 0 = No error.
;                  |1 = File cannot be opened or found.
;                  |2 = Unable to Split the file
; Author ........: Tylo <tylo at start dot no>
; Modified.......: Xenobiologist
; Remarks .......: It does not count a final @LF as a line.
; Related .......:
; Link ..........;
; Example .......; Yes
; ===============================================================================================================================

Func _FileCountLinesX($sFilePath)
    Local $hFile, $sFileContent, $aTmp
    $hFile = FileOpen($sFilePath, 0)
    If $hFile = -1 Then Return SetError(1, 0, 0)
    $sFileContent = FileRead($hFile, FileGetSize($sFilePath))
    FileClose($hFile)
    If StringInStr($sFileContent, @LF) Then
        $aTmp = StringSplit(StringStripCR($sFileContent), @LF)
    ElseIf StringInStr($sFileContent, @CR) Then
        $aTmp = StringSplit($sFileContent, @CR)
    Else
        If StringLen($sFileContent) > 0 Then Return 1
        Return SetError(2, 0, 0)
    EndIf
    Return $aTmp[0]
EndFunc   ;==>_FileCountLinesX

Func _FileCountLinesMrCreatoR($sFilePath)

    ; Copyright MrCreatoR
    ; This function has been created by MrCreatoR
    ; http://www.autoitscript.com/forum/index.php?showtopic=79656&view=findpost&p=574317
    
    Local $sFRead = FileRead($sFilePath)
    Local $iCountLines = 0
    $sFRead = StringReplace($sFRead, @CRLF, "")
    $iCountLines += @extended
    $sFRead = StringReplace($sFRead, @CR, "")
    $iCountLines += @extended
    StringReplace($sFRead, @LF, "")
    $iCountLines += @extended
    Return $iCountLines + 1
EndFunc   ;==>_FileCountLinesMrCreatoR

Func LineCount($sFilePath)
    Local $N = FileGetSize($sFilePath)
    If @error Or 0 == $N Then Return 0
    Return CharCountPlus1(FileRead($sFilePath, $N), @LF);
EndFunc   ;==>LineCount

#cs
    Find all occurences of LF.
    If last two bytes are not CR/LF, return the above count incremented by 1
#ce

Func CharCountPlus1($inputString, $matchString)
    Local $count = 1
    While 0 <> StringInStr($inputString, $matchString, 2, $count)
        $count += 1;
    WEnd
    If 0 == StringInStr(StringRight($inputString, 3), $matchString, 2) Then
        $count += 1;
    EndIf
    Return $count;
EndFunc   ;==>CharCountPlus1oÝ÷ Ù¬º[ljëh×6!--- expected 0
Number of lines - null
 LineCount - 0
_FileCountLines - 0
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 0
!--- expected 0
Number of lines - zero.txt
 LineCount - 0
_FileCountLines - 0
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 0
!--- expected 23
Number of lines - test.txt 
LineCount - 24
_FileCountLines - 23
_FileCountLinesMrCreatoR - 23
_FileCountLinesX - 23
!--- expected 1
Number of lines - trailing.txt
LineCount - 2
_FileCountLines - 1
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 1

Mega

[Xenobiologist]

Hi,

yeah tried the one from the zip. Now I got the prob!

!--- expected 0
Number of lines - null
 LineCount - 0
_FileCountLines - 0
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 0
!--- expected 0
Number of lines - zero.txt
 LineCount - 0
_FileCountLines - 0
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 0
!--- expected 23
Number of lines - test.txt 
LineCount - 23
_FileCountLines - 22
_FileCountLinesMrCreatoR - 23
_FileCountLinesX - 22
!--- expected 1
Number of lines - trailing.txt
LineCount - 2
_FileCountLines - 1
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 1

Mega

[PoorLuzer]

Hi,

changed the script to easily show the results

Number of lines - null
 LineCount - 0
_FileCountLines - 0
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 0
!--- expected 0
Number of lines - zero.txt
 LineCount - 0
_FileCountLines - 0
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 0
!--- expected 23
Number of lines - test.txt 
LineCount - 24
_FileCountLines - 23
_FileCountLinesMrCreatoR - 23
_FileCountLinesX - 23
!--- expected 1
Number of lines - trailing.txt
LineCount - 2
_FileCountLines - 1
_FileCountLinesMrCreatoR - 1
_FileCountLinesX - 1

The code for _FileCountLinesMrCreatoR that you have is outdated. I have updated the latest code in the attachment.

As you can see, both MrCreatoR's and mine are conforming - but I would love to stand corrected.

There are two things I am assuming:

1. A trailing linefeed should be counted as a line (albeit a blank one). Is this assumption wrong?

2. The AutoIt functions used by me are locale independent. I use EN-US.

zipped.zip

x.au3

test.txt

trailing.txt

Edited September 4, 2008 by PoorLuzer

[Subhobroto]

This function is indeed useful, but for some reason, all the code given so far seem to be buggy in some respect!

A correct algorithm to count the number of lines for a DOS/UNIX based system:

1. count all the @CR

2. count all the @LF NOT preceeded immediately by a @CR

3. Increment the sum of above counts by one

4. If the user does not want to count trailing blank line as a line, and if last 2 characters in the file is either of the above characters, decrement the sum above by one.

All this can be done in one pass instead of multiple passes by implementing a state machine.

However, just to demonstate that the above algorithm is workable, I have hacked up a solution and not implemented it as a FSM yet - I will, when it's demonstrated by others tobe correct.

I have mailed you the code.. have a go at it >_<

[youknowwho4eva]

While messing with this, I discovered another mishap. FileReadLine("your file",-1) should read the last line. Instead it reads the last line before the first blank line. In my file I had.

1

2

3

4

5

6

7

8

9

10

and filereadline returned 9 as the end of the file.

Edit: just did a filereadline on line 14 (location of the 10) and it reads nothing.

Edit: I lied helps if I hit save

Edited September 24, 2008 by youknowwho4eva

[PoorLuzer]

@Subhobroto,

I got your solution - I have coded a C++ version, but did not have time to post it.

I will make modifications to the code you sent me and post it here.

Oh - your file line counting works for Windows, UNIX and MacOS

About your suggestion for a state machine - it's novel - if enough demand is there, and if there are chances that this function gets into core AutoIt, I will definitely implement it.

Keep up updated on the bugs!

@youknowwho4eva,

What you mentioned - was that your oversight or indeed a bug?

[youknowwho4eva]

Oversight, I Thought I had saved it with the 10 at the bottom, but apparently I only had the 1-9 when I saved.