Remove Blank Lines in a file

[DemonWareXT]

Hey everyone,

I'm back again with more problems ^^'

I have an application that opens a file to read the stuff in there.

What I want it to do beforehand is to remove blank lines from the File.

Here is what I got

$read=FileRead($listFull)
     FileDelete($listFull)
     Do
         $read=StringReplace($read,@CRLF&@CRLF,@CRLF)
     Until @extended=0
     FileWrite($listFull,$read)

It actually works quite well, but it runs into an error if the first line of the document is a blank line

[Melba23]

DemonWareXT,

Perhaps like this:

#include <File.au3>
#include <Array.au3>

Global $aLines

_FileReadToArray("Test.txt", $aLines)

For $i = $aLines[0] To 1 Step -1
    If $aLines[$i] = "" Then
        _ArrayDelete($aLines, $i)
    EndIf
Next

_FileWriteFromArray("NewTest.txt", $aLines, 1)

Note you have to run the delete code from the end of the array or you run into errors with the count when you delete lines.

M23

[Xenobiologist]

#include<Array.au3>
#include<File.au3>
Global $pfad = @ScriptDir & '\blank.txt', $lines_A
_FileReadToArray($pfad, $lines_A)
For $i = UBound($lines_A) - 1 To 1 Step -1
If Not StringRegExp($lines_A[$i], '[^\s]', 0) Then _ArrayDelete($lines_A, $i)
Next
_FileWriteFromArray($pfad, $lines_A, 1)

[hannes08]

This will be faster for big files:

#include <File.au3>
#include <Array.au3>

Global $aLines

_FileReadToArray("Test.txt", $aLines)

$sLines = _ArrayToString($aLines, "|", 1)
$sLines = StringReplace($sLines, "||", "|")
$aLines = StringSplit($sLines, "|")

_FileWriteFromArray("NewTest.txt", $aLines, 1)

There's no need to call _ArrayDelete which redims the array on every call (and therefor is slow).

Or you could use StringRegExpReplace:

$sContent = FileRead("Test.txt")
$sContent = StringRegExpReplace($sContent, "^\s*$", "")
FileWrite("NewTest.txt", $sContent)

But I'm not very much into RegularExpressions, so that needs to be tested.

Edited September 27, 2011 by hannes08

[DemonWareXT]

Thanks a lot M23 works like a charm

[Melba23]

hannes08,

Unfortunately your solution will not work if there are multiple blank lines - it only repalces pairs with singles.

However, your "No ReDim" point is very valid and so I have come up with this:

$sText = FileRead("Test.txt")

$sText = StringRegExpReplace(StringRegExpReplace($sText, "(\v)+", @CRLF), "\A\v|\v\Z", "")

$hFile = FileOpen("NewText.txt", 2)
FileWrite($hFile, $sText)
FileClose($hFile)

which seems to fit the bill. The inner SRE repaces all multiple EOLs with a single EOL, while the outer removes any remaining leading and trailing EOLs (if they were multiple they have already been reduced to a single by the first SRE).

M23

Edit: See below for a corrected version of the SRE.

Edited September 27, 2011 by Melba23

[DemonWareXT]

@M23

your second version doesn't seem to work either, I get the original problem again, if there is a blank on the first line then it will crash... or maybe I'm to daft to implement it xD

[Melba23]

DemonWareXT,

As you might expect it works fine for me or I would not have posted it!

And I cannot see why a leading EOL would crash the code - or do you just mean that it does not remove the blank?

Could you please post the code you are using and example of a file which crashes it - I will take a look.

M23

[DemonWareXT]

@M23

it doesn't leave the first line blank, it enters a number in there?

well if you really wanna see my whole code xD

(I did put your first code in there again tho)

Edit: I hope you can read it, somehow the code shows up crazy on ma PC, so I atached the script too

expandcollapse popup

#include <GuiConstants.au3>
#include <Array.au3>
#include <file.au3>
#include <ButtonConstants.au3>
#include <GUIListBox.au3>
;include <GUIConstantsEx.au3>
;#include <WindowsConstants.au3>
;#Include <GuiScrollBars.au3>
;~ HotKeySet("^w", "tag")
HotKeySet("^q", "lTag")
HotKeySet("{ESC}", "errorH")
;;;;HotKeySet("{DEL}", "del")
 
;**************************
;Global Variables
;**************************
$listFull = @ScriptDir & 'list.csv'
;$listShort = @ScriptDir & 'list short names.txt'
;$listWhole = @ScriptDir & 'people.csv'
$error = 0
Do
;clean up the lists
;~   $read=FileRead($listFull)
;~   FileDelete($listFull)
;~   Do
;~  $read=StringReplace($read,@CRLF&@CRLF,@CRLF)
;~   Until @extended=0
;~   FileWrite($listFull,$read)
;call function to clear blank lines in text
clearBlank()
 
 
;**************************
;read array from text file
;**************************
;read the whole array
Dim $Whole
If Not _FileReadToArray($listFull,$Whole) Then
    MsgBox(4096,"Error", " Error reading Array  error:" & @error)
    Exit
   EndIf
   ;sort the array
  _ArraySort($Whole)
;write array back to csv
  _FileWriteFromArray($listFull,$Whole, 1)
  ;For $i = 1 to UBound($Whole)-1
  ;Next
Dim $helper[2] = [0,0]
Dim $PeopleFull[1]
Dim $PeopleShort[1]
For $i = 1 to UBound($Whole)-1
  $helper = StringSplit($Whole[$i],';', 1)
  ReDim $PeopleFull[UBound($PeopleFull)+1]
  ReDim $PeopleShort[UBound($PeopleShort)+1]
  $PeopleFull[$i] = $helper[1]
  $PeopleShort[$i] = $helper[2]
  ;MsgBox(0,"",$helper[1])
  ;MsgBox(0,"",$helper[2])
Next
 
 
;**************************
;Create the GUI
;**************************
;Do while loop so we can rebuild the GUI if need be.
;main GUI window
$Form1 = GUICreate("Auswählen",570,550,192,114)
;some textlables to eplain whats going on
GUICtrlCreateLabel("1. Select people, use shift and ctrl", 10, 480)
GUICtrlCreateLabel("2. Go to G+ get cursor in input dialog", 10, 500)
GUICtrlCreateLabel("3. CTRL+Q to start Tagging", 10, 520)
;Button that adds ponies to the list
$add = GUICtrlCreateButton("Add", 350, 520,50)
;Input Box
$Input1 = GUICtrlCreateInput("", 200, 520, 150, 25)
;some textlables to explain whats going on
GUICtrlCreateLabel("| Fomrate like this: Name;Shortname", 200, 480)
GUICtrlCreateLabel("| Use with caution", 200, 500)
;Button to delete people
$del = GUICtrlCreateButton("Delete Selected", 400, 520)
 
; Create an array to hold the checkbox ControlIds - make it the same size as the $PeopleFull array
;Global $Person[UBound($PeopleFull)]
;Global $del[UBound($PeopleFull)]
;Some variables to count stuff
$count = 0
$counter = 0
;if retry = 1 the do while loop will end
$retry = 1
;List of people
$ListPeople = GUICtrlCreateList("", 10, 10, 550, 450, BitOR($LBS_STANDARD, $LBS_EXTENDEDSEL))
;$ListPeople = GUICtrlCreateList("", 10, 10, 300, 450, BitOR($GUI_SS_DEFAULT_LIST,$LBS_MULTIPLESEL))
For $i = 1 to UBound($PeopleFull)-1
  GUICtrlSetData($ListPeople, $PeopleFull[$i])
Next
;**************************
;Create GUI Actions
;**************************
;Set the GUI to be shown
GUISetState(@SW_SHOW)
  Do
   $msg = GUIGetMsg()
   Switch $msg
    Case $add
    If GUICtrlRead ($Input1) = "" Then
      MsgBox(0,'Niet','Please... Input!')
     Else
      add()
     EndIf
    Case $del
     del()
   EndSwitch
 
  Until $msg = $GUI_EVENT_CLOSE
Until $retry=1
;Functions
;~ Func tag()
;~  ;First sleep 5seconds so the user can place the cursor to the right place
;~  ;Sleep(4000)
;~  $error = 0
;~  For $i = 1 to UBound($PeopleFull)-1
;~   If $error = 1 Then
;~   ElseIf GuiCTRLRead ($Person[$i]) = $GUI_CHECKED Then
;~  ClipPut($PeopleFull[$i])
;~  ;Send("{+}")
;~  Send("^v")
;~  Sleep(1000)
;~  Send("{ENTER}")
;~  Sleep(500)
;~   EndIf
;~  Next
;~ EndFunc
Func lTag()
$items = _GUICtrlListBox_GetSelItems($ListPeople)
Opt("SendKeyDelay",40)
For $i = 1 to UBound($items)-1
  If $error = 1 Then
  Else
   $item = $items[$i]+1
   Send("{+}");
   Send($PeopleShort[$item])
   Sleep(1000)
   Send("{ENTER}")
   Sleep(500)
  EndIf
Next
Opt("SendKeyDelay",5)
Send("{CTRLUP}")
;Send("{SHIFTUP}")
;Send("{ALTUP}")
$error = 0
EndFunc
Func clearBlank()
  Global $aLines
_FileReadToArray($listFull, $aLines)
For $i = $aLines[0] To 1 Step -1
  If $aLines[$i] = "" Then
   _ArrayDelete($aLines, $i)
  EndIf
Next
_FileWriteFromArray($listFull, $aLines, 1)
EndFunc
Func add()
;$counter = $counter
ReDim $Whole[UBound($Whole)+1]
$Whole[$counter] = GUICtrlRead ($Input1)
_ArraySort($Whole)
_FileWriteFromArray($listFull,$Whole, 1)
$helper = StringSplit(GUICtrlRead ($Input1),';', 1)
ReDim $PeopleFull[UBound($PeopleFull)+1]
ReDim $PeopleShort[UBound($PeopleShort)+1]
$PeopleFull[$i] = $helper[1]
$PeopleShort[$i] = $helper[2]
GUICtrlSetData($ListPeople, $helper[1])
_ArraySort($PeopleFull)
_ArraySort($PeopleShort)
EndFunc
Func del()
$items = _GUICtrlListBox_GetSelItems($ListPeople)
For $i = 1 to UBound($items)-1
  $item = $items[$i]+1
  $PeopleFull[$item] = ""
  $PeopleShort[$item] = ""
  $Whole[$item] = ""
  ;MsgBox(0,"",$PeopleShort[1])
Next
 
_ArraySort($PeopleFull)
_ArraySort($PeopleShort)
_ArraySort($Whole)
_FileWriteFromArray($listFull,$Whole, 1)
clearBlank()
GUICtrlSetData($ListPeople, "")
For $i = 1 to UBound($PeopleFull)-1
  GUICtrlSetData($ListPeople, $PeopleFull[$i])
Next
EndFunc
;debuging
Func errorH()
$error=1
;Exit
EndFunc

The list behind it is called list.csv and is built up like this:

name;name2

autoTag_V2.5.au3

Edited September 27, 2011 by DemonWareXT

[Melba23]

DemonWareXT,

That post did not work.

M23

[DemonWareXT]

@M23

just realised that xD

Attached the script

the list is built up like this:

list.csv

blablabla;blablabla

autoTag_V2.5.au3

[Melba23]

DemonWareXT,

Works just fine for me (but then I would say that! ):

(list.csv) - not in the file but the forum software does not show the leading blank line if I do not put it in!

blablabla;blablabla2
blablabla;blablabla3



blablabla;blablabla7
blablabla;blablabla8
blablabla;blablabla9
blablabla;blablabla10



blablabla;blablabla14
blablabla;blablabla15
blablabla;blablabla16
blablabla;blablabla17


blablabla;blablabla20
blablabla;blablabla21

blablabla;blablabla23
blablabla;blablabla24



blablabla;blablabla28
blablabla;blablabla29

becomes

(list.csv)
blablabla;blablabla2
blablabla;blablabla3
blablabla;blablabla7
blablabla;blablabla8
blablabla;blablabla9
blablabla;blablabla10
blablabla;blablabla14
blablabla;blablabla15
blablabla;blablabla16
blablabla;blablabla17
blablabla;blablabla20
blablabla;blablabla21
blablabla;blablabla23
blablabla;blablabla24
blablabla;blablabla28
blablabla;blablabla29

And the code does not crash in that function. Are you sure that is where the error occurs?

M23

[DemonWareXT]

@M23

I sent you the code where I implemented your first try which worked perfectly^^

either you implement your second version or I'll send the file again with error xD

Edit: attached it again

autoTag_V2.5.au3

Edited September 27, 2011 by DemonWareXT

[Akarillon]

Just to ask, does StringStripWS() work for the same problem or does it only work on spaces?

[DemonWareXT]

Just to ask, does StringStripWS() work for the same problem or does it only work on spaces?

StringStripWS() is only used to strip of white spaces, so I don't think it would work

[Melba23]

DemonWareXT,

Try this SRE - I think it solves the problem:

$sText = StringRegExpReplace(StringRegExpReplace($sText, "(\v)+", @CRLF), "(^\v*)|(\v*\Z)", "")

I was only removing the CR of a CRLF. When I opened the file in NotePad to check it did not honour the remaining 0A with a new line - it was only when I looked with a hex editor I saw the problem.

As for the added number I think it happens when you sort your array - try this code:

_ArraySort($Whole, 0, 1)

Must rush to the dentist.

Try those out and I will look again when I return.

M23

[DemonWareXT]

@M23

Now it works, you're an hero

about the number problem, I think this only happened because there was some array size confusion, which now doesn't happen any more since the line removing works correctly now

[Melba23]

DemonWareXT,

Excellent news.

And I only have to take some antibiotics for a while, so a result for both of us!

M23

[GEOSoft]

Just at a quick glance this may also work.

$sText = StringRegExpReplace($sText, "(?m:^)\h*(\r\n|\r|\n)", "")

[UEZ]

Is this working for you?

$sNew = StringReplace($text, Chr(10) & Chr(10), "")
If Not @extended Then $sNew = StringReplace($text, Chr(13) & Chr(10) & Chr(13) & Chr(10), "")
ConsoleWrite($sNew & @LF)

Br,

UEZ