File read and array script

[BlueLord]

Hy all,

I have a file with 100 entries, each 10 entries are the same:

entries.txt

1

1

1

1

1

1

1

1

1

1

2

2

2

2

2

2

2

2

2

2

3

3

etc until 100 entries.

I need to make a script which read 10 lines check if value is the same and write only an entry on new file

new.txt

1

2

3

4

5

6

etc until 10 entries

Some hints exm

Thanks

[Melba23]

BlueLord,

Some hints

Some effort on your part!

However, try looking at FileReadLine within a loop - with a counter up to 10 to check that the lines are the same. Then FileWriteLine to create the new file. Give it a go - you might surprise yourself!

M23

[water]

Quick and dirty:

Do a _FileReadToArray and then loop through the array and write to the output file if $aArray[$iIndex] <> $aArray[$iIndex+1]

[sahsanu]

I have a file with 100 entries, each 10 entries are the same:

In that scenario, I would use _FileReadToArray, _ArrayUnique and _FileWriteFromArray.

[BlueLord]

Hy,

I have tries the version with arrayunique

#include <file.au3>
#include <Array.au3>

Dim $aRecords

$input = "input.txt"
$output = FileOpen("output.txt", 1)
 _FileReadToArray($input, $aRecords)

 $aNewArray = _ArrayUnique($aRecords)

_FileWriteFromArray($output,$aNewArray)
FileClose($output)

But it put only a uniq value from line.

This is the script i have tried to build, but my programing skills are not so good

$file1 = FileOpen ( "input.txt", 0 )
$file2 = FileOpen ( "output.txt", 1 )

While 1
    For $_I = 1 To 5
            $line = FileReadLine ( $file1 )
            If @error = -1 Then ExitLoop
                ;how to compare lines
            
    Next
            FileWrite ( $file2, $line & @Crlf )
            
Wend

FileClose ( $file1 )
FileClose ( $file2 )
ShellExecute ( "output.txt")

In atachament i put the input file and output file so i want to be.

input.txt

output.txt

[sahsanu]

I have tries the version with arrayunique

_ArrayUnique won't work because your series are repeating

I did it in a hurry but maybe it could be useful to you (check it twice because as I said I did it too fast) :

#include <file.au3>

$file1 = @ScriptDir & "\input.txt"
$file2 = @ScriptDir & "\output.txt"
Global $afile1
_FileReadToArray($file1,$afile1)
$hfile2 = FileOpen($file2, 2) ;this will overwrite the output file

For $i = 1 To $afile1[0]
    If $i < $afile1[0] Then
        If $afile1[$i] = $afile1[$i+1] Then
            ContinueLoop
        Else
            FileWrite ($hfile2,$afile1[$i] & @CRLF)
        EndIf
    ElseIf $afile1[$i] = $afile1[$i-1] Then
        FileWrite ($hfile2,$afile1[$i] & @CRLF)
    EndIf
Next

FileClose($hfile2)
ShellExecute($file2)

[UEZ]

Your output file has double lines:

1,2,3 <<<<
2,3,4 <<<<
3,4,5
2,3,4 <<<<
1,2,3 <<<<

Is that intentional?

If not and you want a unique output then try this:

expandcollapse popup

#include <Array.au3>
$sInput = FileRead("input.txt")
$aInput = StringSplitW($sInput, ",")
$aOutput = ArrayUnique($aInput)
_ArrayDisplay($aOutput)

; #FUNCTION# ======================================================================================
; Name ................:    StringSplitW()
; Version .............:    v0.80 build 2011-05-05 beta
; Description .......:    Splits  a string into columns instead of rows as it is done by SplitString(), like a csv file to a 2d array ;-)
; Syntax ..............:    StringSplitW($string, $delimiter, $max_width)
; Parameters ........:    $string - string to split
;                                $delimiter - the delimter how to split the string
; Return values ....:    Success - 2d array
;                                 Failure 1 - either $string or $delimter is not set
;                                 Failure 2 - array width exceeded
;                                 Failure 3 - 2D array memory exceeded -> width + height > 0x5EFA
; Author ..............:    UEZ
; Modified ............:
; Remarks ............:
; Related ..............:    StringSplit()
; =================================================================================================
Func StringSplitW($string, $delimiter)
    If $string = "" Or $delimiter = "" Then Return SetError(1, 0, 0)
    Local $chk, $width, $i, $j, $k, $len, $max = 1, $max_width
    Local $aPos[1], $l = 0
    Local $aSplit =  StringSplit($string, @LF)
    $max_width = 0x5EFA - $aSplit[0]
    If $max_width < 1 Then Return SetError(3, 0, 0)
    Local $aVertical[$aSplit[0]][$max_width]
    For $k = 1 To $aSplit[0]
        $len = StringLen($aSplit[$k])
        If $len > 1 Then
            $chk = StringReplace($aSplit[$k], $delimiter, $delimiter)
            $width = @extended
            If $width > $max_width Then Return SetError(2, 0, 0)
            If $width > $max Then $max = $width + 1
            Switch $width
                Case 0
                    $aVertical[$l][0] = $aSplit[$k]
                Case Else
                    Dim $aPos[$width * 2 + 2]
                    $j = 1
                    $aPos[0] = 1
                    For $i = 0 To $width - 1
                        $aPos[$j] = StringInStr($aSplit[$k], $delimiter, 0, $i + 1) - 1
                        $aPos[$j + 1] = $aPos[$j] + 2
                        $j += 2
                    Next
                    $aPos[UBound($aPos) - 1] = StringLen($aSplit[$k])
                    $j = 0
                    For $i = 0 To UBound($aPos) - 1 Step 2
                        $aVertical[$l][$j] = StringMid($aSplit[$k], $aPos[$i], $aPos[$i + 1] - $aPos[$i] + 1)
                        $j += 1
                    Next
                EndSwitch
                $l += 1
        EndIf
    Next
    ReDim $aVertical[$l][$max]
    Return $aVertical
EndFunc

; #FUNCTION# ============================================================================
; Name.............:    ArrayUnique
; Description ...:  Returns the Unique Elements of a 1-dimensional or 2-dimensional array.
; Syntax...........:    _ArrayUnique($aArray[, $iBase = 0, oBase = 0])
; Parameters ...:   $aArray - The Array to use
;                           $iBase  - [optional] Is the input Array 0-base or 1-base index.  0-base by default
;                           $oBase  - [optional] Is the output Array 0-base or 1-base index.  0-base by default
; Return values:    Success - Returns a 1-dimensional or 2-dimensional array containing only the unique elements
;                           Failure - Returns 0 and Sets @Error:
;                           0 - No error.
;                           1 - Returns 0 if parameter is not an array.
;                           2 - Array has more than 2 dimensions
;                           3 - Array is already unique
;                           4 - when source array is selected as one base but UBound(array) - 1 <> array[0] / array[0][0]
;                           5 - Scripting.Dictionary cannot be created for 1D array unique code
; Author .........:     UEZ 2010 for 2D-array, Yashied for 1D-array (modified by UEZ)
; Version ........:     0.96 Build 2010-11-20 Beta
; Remark ........:  check is always case sensitive
; =======================================================================================
Func ArrayUnique($aArray, $iBase = 0, $oBase = 0)
    If Not IsArray($aArray) Then Return SetError(1, 0, 0) ;not an array
    If UBound($aArray, 0) > 2 Then Return SetError(2, 0, 0) ;array is greater than a 2D array
    If UBound($aArray) = $iBase + 1 Then Return SetError(3, 0, $aArray) ;array is already unique because of only 1 element
    Local $dim = UBound($aArray, 2), $i
    If $dim Then ;2D array
        If $iBase And UBound($aArray) - 1 <> $aArray[0][0] Then Return SetError(4, 0, 0)
        Local $oD = ObjCreate('Scripting.Dictionary')
        If @error Then Return SetError(5, 0, 0)
        Local $i, $j, $k = $oBase, $l, $s, $aTmp, $flag, $sSep = Chr(01)
        Local $aUnique[UBound($aArray)][$dim]
        If Not $oBase Then $flag = 2
        For $i =  $iBase To UBound($aArray) - 1
            For $j = 0 To $dim - 1
                $s &= $aArray[$i][$j] & $sSep
            Next
            If Not $oD.Exists($s) And StringLen($s) > 3 Then
                $oD.Add($s, $i)
                $aTmp = StringSplit(StringTrimRight($s, 1), $sSep, 2)
                For $l = 0 To $dim - 1
                    $aUnique[$k][$l] = $aTmp[$l]
                Next
                $k += 1
            EndIf
            $s = ""
        Next
        $oD.RemoveAll
        $oD = ""
        If $k > 0 Then
            If $oBase Then $aUnique[0][0] = $k - 1
            ReDim $aUnique[$k][$dim]
        Else
            ReDim $aUnique[1][$dim]
        EndIf
    Else ;1D array
        If $iBase And UBound($aArray) - 1 <> $aArray[0] Then Return SetError(4, 0, 0)
        Local $sData = '', $sSep = ChrW(160), $flag
        For $i = $iBase To UBound($aArray) - 1
            If Not IsDeclared($aArray[$i] & '$') Then
                Assign($aArray[$i] & '$', 0, 1)
                $sData &= $aArray[$i] & $sSep
            EndIf
        Next
        If Not $oBase Then $flag = 2
        Local $aUnique = StringSplit(StringTrimRight($sData, 1), $sSep, $flag)
    EndIf
    Return $aUnique
EndFunc   ;==>ArrayUnique

Br,

UEZ

Edited May 28, 2011 by UEZ

[sahsanu]

Your output file has double lines:

1,2,3
2,3,4 <<<<
3,4,5
2,3,4 <<<<
1,2,3

Is that intentional?

Hello UEZ,

Yes, it is. Seems that OP wants it in that way as shown in his

[UEZ]

Ok then try this:

For 1D arrays:

#include <array.au3>
#include <file.au3>

Global $aInput
_FileReadToArray("input.txt", $aInput)
_ArrayDelete($aInput, 0)
$aOutput = Compact1DArray($aInput)
_ArrayDisplay($aOutput)

Func Compact1DArray($array)
    Local $ao
    If Not IsArray($array) Then Return SetError(1, 0, 0)
    If UBound($array, 0) <> 1 Then Return SetError(2, 0, 0)
    If UBound($array) < 2 Then Return SetError(3, 0, 0)
    Local $i = 0, $j = 1
    Local $s1, $s2, $sOutput
    While $j < UBound($array)
        $s1 = StringStripCR($array[$i])
        $s2 = StringStripCR($array[$j])
        If $s1 <> $s2 Then
            $sOutput &= $s1 & @LF
            $i = $j
            $j += 1
        Else
            $j += 1
        EndIf
    WEnd
    If Not $i Then
        $sOutput &= $s1 & @LF
    Else
        $sOutput &= $s2 & @LF
    EndIf
    $ao = StringSplit(StringTrimRight($sOutput, 1), @LF, 2)
    Return $ao
EndFunc

For 2D arrays:

expandcollapse popup

#include <array.au3>
Global $sInput = FileRead("input.txt")
Global $aInput = StringSplitW($sInput, ",")
Global $aOutput = Compact2DArray($aInput)
_ArrayDisplay($aOutput)

Func Compact2DArray($array)
    Local $ao
    If Not IsArray($array) Then Return SetError(1, 0, 0)
    If UBound($array, 0) <> 2 Then Return SetError(2, 0, 0)
    If UBound($array) < 2 Then Return SetError(3, 0, 0)
    Local $i = 0, $j = 1
    Local $s1, $s2, $sOutput
    While $j < UBound($array)
        $s1 = ArrayLine2String($array, $i)
        $s2 = ArrayLine2String($array, $j)
        If $s1 <> $s2 Then
            $sOutput &= $s1 & @LF
            $i = $j
            $j += 1
        Else
            $j += 1
        EndIf
    WEnd
    If Not $i Then
        $sOutput &= $s1 & @LF
    Else
        $sOutput &= $s2 & @LF
    EndIf
    $ao = StringSplitW($sOutput, ",")
    Return $ao
EndFunc

Func ArrayLine2String($array, $index, $delimiter = ",")
    Local $string
    If Not IsArray($array) Then Return SetError(1, 0, 0)
    If UBound($array, 0) <> 2 Then Return SetError(2, 0, 0)
    If $index < 0 Or $index > UBound($array) - 1 Then Return SetError(3, 0, 0)
    For $i = 0 To UBound($array, 2) - 1
        $string &= StringStripCR($array[$index][$i]) & $delimiter
    Next
    Return StringTrimRight($string, 1)
EndFunc

; #FUNCTION# ========================================================================================================================================
; Name .................:   StringSplitW()
; Description ..........:   Splits  a string into columns instead of rows as it is done by SplitString(), like a csv file to a 2d array ;-)
; Syntax ...............:   StringSplitW($sString, $sDelimiter, $iWidthLen)
; Parameters ...........:   $sString - string to split
;                           $sDelimiter - [optional] the delimter how to split the string
;                           $iWidthLen - [optional] length of the row (amount of columns - default is 256)
; Return values .......:    Success - 2d array
;                           Error 1 - either $sString or $delimter is not set
;                           Error 2 - array width exceeded
;                           Error 3 - error splitting string
;
; Version .............:    v0.96 build 2015-01-20 beta
; Author ..............:    UEZ
; Modified ............:
; Remarks .............:    RegEx take from http://stackoverflow.com/questions/4476812/regular-expressions-how-to-replace-a-character-within-quotes
; Related .............:    StringSplit, StringReplace, StringRegExpReplace, StringLen, StringStripCR
; ===================================================================================================================================================
Func StringSplitW($sString, $sDelimiter = ";", $sQuotationMark = '"', $sDummy = "¦", $iWidthLen = 256)
    If $sString = "" Or $sDelimiter = "" Then Return SetError(1, 0, 0)
    Local $chk, $iWidth, $i, $j, $k, $iLen, $iMax = 1, $iMaxWidth
    Local $aPos[1], $l = 0
    Local $aSplit =  StringSplit(StringStripCR($sString), @LF)
    If @error Then Return SetError(3, 0, 0)
    Local $aVertical[$aSplit[0]][$iWidthLen], $iDelimiterLen = StringLen($sDelimiter) - 1, $sLine
    For $k = 1 To $aSplit[0]
        $iLen = StringLen($aSplit[$k])
        If $iLen > 1 Then
            $sLine = StringRegExpReplace($aSplit[$k], '(?m)\' & $sDelimiter & '(?=[^' & $sQuotationMark & ']*' & $sQuotationMark & '(?:[^' & $sQuotationMark & '\r\n]*' & $sQuotationMark & '[^' & $sQuotationMark & ']*' & $sQuotationMark & ')*[^' & $sQuotationMark & '\r\n]*$)', $sDummy)
            $chk = StringReplace($sLine, $sDelimiter, $sDelimiter)
            $iWidth = @extended
            If $iWidth > $iWidthLen Then Return SetError(2, 0, 0)
            If $iWidth >= $iMax Then $iMax = $iWidth + 1
            Switch $iWidth
                Case 0
                    $aVertical[$l][0] = $sLine
                Case Else
                    Dim $aPos[$iWidth * 2 + 2]
                    $j = 1
                    $aPos[0] = 1
                    For $i = 0 To $iWidth - 1
                        $aPos[$j] = StringInStr($sLine, $sDelimiter, 0, $i + 1) - 1
                        $aPos[$j + 1] = $aPos[$j] + 2 + $iDelimiterLen
                        $j += 2
                    Next
                    $aPos[UBound($aPos) - 1] = StringLen($sLine)
                    $j = 0
                    For $i = 0 To UBound($aPos) - 1 Step 2
                        $aVertical[$l][$j] = StringMid(StringReplace($sLine, $sDummy, $sDelimiter), $aPos[$i], $aPos[$i + 1] - $aPos[$i] + 1)
                        $j += 1
                    Next
                EndSwitch
                $l += 1
        EndIf
    Next
    ReDim $aVertical[$l][$iMax]
    Return $aVertical
EndFunc

Not fully tested!

Br,

UEZ

Edited January 19, 2015 by UEZ