UDF unique array

[djek]

I needed one, so I stole one

Array needs to be sorted first, no checks!

expandcollapse popup

; ----------------------------------------------------------------------------
;
; AutoIt Version: 3.0
; Language:    English
; Platform:    Win9x / NT
; Converter:         djek
;UDF shamelessly stolen from Glenn Barnas/FRIT-EROC (http://www.kixhelp.com/udfs/udf/83390.htm)
;
; Script Function:
;   Returns an array with unique elements
;
; ----------------------------------------------------------------------------
;!Requires the array to be sorted first like _ArraySort
;Parameters:  <dimensional array> ,  <likestringsplit>
;   likestringsplit: 0= return normal array 1=strip first element
;
;EXAMPLES:
;--
; 1
;--
;#include <Array.au3>
;dim $nonuniqstring,$result
;$nonuniqstring=StringSplit("a|b|b|c|d|e|f","|")
;
;$result = _ArrayUniq($nonuniqstring,1)
;_ArrayDisplay($result, "Uniq")
;
;--
; 2
;--
;#include <Array.au3>
;dim $nonuniqstring,$result
;Dim $nonuniqstring[8]
;$nonuniqstring[0] = "a"
;$nonuniqstring[1] = "b"
;$nonuniqstring[2] = "b"
;$nonuniqstring[3] = "c"
;$nonuniqstring[4] = "d"
;$nonuniqstring[5] = "e"
;$nonuniqstring[6] = "f"
;$nonuniqstring[7] = "f"

;$result = _ArrayUniq($nonuniqstring,0)
;_ArrayDisplay($result, "Uniq")

; ----------------------------------------------------------------------------
; Script Start - Add your code below here
; ----------------------------------------------------------------------------

Func _ArrayUniq($a, $isstringsplit = 0)
;$stringissplit is optional, converts an stringsplitted array to a 'normal' array. iaw [0] has the first value, not the number of strings
; $TOP is last entry, $X is source pointer, $Y is destination pointer
    Dim $Top, $X, $Y, $firstelement
    $Top = UBound($a)         ; Find size of original array
    Dim $WA[$Top]               ; Create working array

; output array pointer
    If $isstringsplit = 0 Then
        $WA[0] = $a[0]       ; copy first element
        $firstelement = 1
    Else
        $WA[0] = $a[1]       ; copy first element
        $firstelement = 2
    EndIf
    $Y = 1

    For $X = $firstelement To $Top - 1
        If $WA[$Y - 1] <> $a[$X] Then; is current value different from last?
            $WA[$Y] = $a[$X]   ; add it to output
            $Y = $Y + 1           ; increment output pointer
        EndIf
    Next
    ReDim $WA[$Y]   ; resize output array
    Return $WA                ; return the array of unique elements
EndFunc  ;==>_ArrayUniq

[Wolvereness]

This will work for none-sorted arrays. Note, values are in same order AND [0] = number of values.

Func _Uniques(ByRef $av_Array)
    If Not IsArray($av_Array) Then Return $av_Array
    Ubound($av_Array, 2)
    If Not @error Then Return $av_Array
    Local $i_Bound = Ubound($av_Array), $i_Count_A, $i_Count_B, $av_NewArray[$i_Bound + 1], $i_Number
    $av_NewArray[0] = 0
    For $i_Count_A = 0 To $i_Bound - 1
        For $i_Count_B = 0 To $i_Bound - 1
            If $av_Array[$i_Count_A] == $av_Array[$i_Count_B] And $i_Count_B <> $i_Count_A Then ContinueLoop(2)
        Next
        $av_NewArray[0] = $av_NewArray[0] + 1
        $av_NewArray[$av_NewArray[0]] = $av_Array[$i_Count_A]
    Next
    $i_Number = $av_NewArray[0]
    Redim $av_NewArray[$i_Number + 1]
    Return $av_NewArray
EndFunc

Edited January 21, 2005 by Wolvereness

[djek]

This will work for none-sorted arrays. Note, values are in same order AND [0] = number of values.

ah yes, but it returns only the unique values,

not an array of values without duplicates

with example 2 (a,b,b,c,d,e,f,f):

Your code returns 4,a,c,d,e

my(stolen) code returns a,b,c,d,e,f

should be fixable

I'd better call it 'returns array without duplicates' (sounds like an indian)

thanks!

[Wolvereness]

This is an updated version that for some reason doesn't need the flag...

Func _ArrayUniques($av_Array)
    If Not IsArray($av_Array) And Abs(Int($i_Flag)) <> $i_Flag Then Return $av_Array
    UBound($av_Array, 2)
    If Not @error Then Return $av_Array
    Local $i_Bound = UBound($av_Array), $i_Count_A, $i_Count_B, $av_NewArray[$i_Bound + 1], $i_Number, $av_Temporary
    $av_NewArray[0] = 0
    For $i_Count_A = 0 To $i_Bound - 1
        For $i_Count_B = 0 To $i_Bound - 1
            If $av_Array[$i_Count_A] == $av_Array[$i_Count_B] And $i_Count_B > $i_Count_A Then ContinueLoop (2)
        Next
        $av_NewArray[0] = $av_NewArray[0] + 1
        $av_NewArray[$av_NewArray[0]] = $av_Array[$i_Count_A]
    Next
    $i_Number = $av_NewArray[0]
    ReDim $av_NewArray[$i_Number + 1]
    Return $av_NewArray
EndFunc  ;==>_ArrayUniques

Edited January 22, 2005 by Wolvereness

[djek]

This is an updated version with a flag... declare $i_Flag =0 to not check for triples...

Wow, that's great.

I suggest that you suggest to put this in the default array include file.

Better call it 'nodupes' to avoid confusion

Edited January 22, 2005 by djek

[groucho]

I think Uniqueness can be solved easily (or?):

1. String-based (thanks to Larry)

$x
If Not StringinStr($unique,$x,0) Then _
        $unique= $unique & "*" & $x
; where * will be replaced later by @LF to create a list

2. array-based (cut-and-paste from many others)

_ArraySort( $array2, "Updated Array")
_ArrayDisplay( $array2, "Updated Array")
$find = "something"
_ArraySearch($array2, $find)
if @error = 1 then
    _ArrayAdd( $array2,$find)
endif
_ArrayDisplay( $array2, "Updated Array")
_ArraySearch($array2, $find)
MsgBox(0, "after", @error); for testing

Func _ArraySearch($avArray, $vWhat2Find, $iStart = -1, $iEnd = -1)
    Local $iCurrentPos, $iUBound
    If Not IsArray($avArray) Then
        SetError(-1)
    EndIf
    $iUBound = UBound($avArray) - 1
    For $iCurrentPos = 1 To $iUBound
    If StringInStr($avArray[$iCurrentPos], $vWhat2Find,0) > 0 then 
                    SetError(0); Entry has been found
        ExitLoop
    Else
        SetError(1); Entry still not found
               EndIf
    Next
EndFunc;==>_ArraySearch

Edited July 5, 2005 by groucho

[nitro322]

I just thought I'd add my own function here. It seems to run significantly faster than the others, and doesn't require sorting. The only potential drawback is the need for a unique delimeter, but in my particular case I know that | will never exist in an array element.

func arrayUnique($arr)
    local $i
    local $seen = ""
    for $i = 0 to ubound($arr)-1
        if NOT stringinstr($seen, $arr[$i]) then $seen = $seen & $arr[$i] & "|"
    next
    $seen = stringtrimright($seen, 1)
    return stringsplit($seen, '|')
endfunc

Feedback welcome, especially if it doesn't work.

Edit: Fixed a logic error, as pointed out by SmOke_N below. I also slightly optimized it (shorter if statement).

Edited October 23, 2006 by nitro322

[SmOke_N]

I just thought I'd add my own function here. It seems to run significantly faster than the others, and doesn't require sorting. The only potential drawback is the need for a unique delimeter, but in my particular case I know that | will never exist in an array element.

func ArrayUnique($arr)
    local $i
    local $seen = ""
    for $i = 0 to ubound($arr)-1
        if stringinstr($seen, $arr[$i]) then
            continueloop
        else
            $seen = $seen & $arr[$i] & "|"
        endif
    next
    return stringsplit($seen, '|')
endfunc

Feedback welcome, especially if it doesn't work.

Your StringInString() has a slight logic issue, and your return of StringSplit() doesn't take out the unneeded delimeter on the end on the return, so it returns 1 more element than is actually there.

I've showed this somewhere before...

Func _ArrayUnique(ByRef $aArray, $vDelim = '', $iBase = 1, $iUnique = 1)
    If $vDelim = '' Then $vDelim = Chr(01)
    Local $sHold
    For $iCC = $iBase To UBound($aArray) - 1
        If Not StringInStr($vDelim & $sHold, $vDelim & $aArray[$iCC] & $vDelim, $iUnique) Then _
            $sHold &= $aArray[$iCC] & $vDelim
    Next
    Return StringSplit(StringTrimRight($sHold, StringLen($vDelim)), $vDelim)
EndFunc

Edited October 3, 2006 by SmOke_N

[nitro322]

Your StringInString() has a slight logic issue, and your return of StringSplit() doesn't take out the unneeded delimeter on the end on the return, so it returns 1 more element than is actually there.

Yeah, I just realized that and was coming back to fix it. Good eyes!

I updated my code above to include the change.

[nitro322]

Hot damn, I just reread your code, SmOke_N, and noticed the &= operator. I've been wanting this for quite a long time! I kept trying to use .= though, which I'm familiar with from PHP and Perl. D'oh!

[SmOke_N]

Hot damn, I just reread your code, SmOke_N, and noticed the &= operator. I've been wanting this for quite a long time! I kept trying to use .= though, which I'm familiar with from PHP and Perl. D'oh!

Yeah, those operators are great, and actually are faster ... You still have a logic issue on your Return with StringSplit() and StringInStr() though.... look at my example, I think that's exactly what you were going for. Edited September 5, 2006 by SmOke_N

[nitro322]

Where's the error? It looks like the only real difference is that mine uses a hardcoded delimiter of '|', whereas yours can be defined. By always using '|' I know that I only need to worry about stripping off the last character.

Of course, that's not to say that hardcoding the delimeter is better. I actually prefer your method. But, I don't see the logic error in the code as it stands. Can you please enlighten me?

Thanks.

[SmOke_N]

Where's the error? It looks like the only real difference is that mine uses a hardcoded delimiter of '|', whereas yours can be defined. By always using '|' I know that I only need to worry about stripping off the last character.

Of course, that's not to say that hardcoding the delimeter is better. I actually prefer your method. But, I don't see the logic error in the code as it stands. Can you please enlighten me?

Thanks.

Ok... to get real unique strings, you have to use the delimeter you are using in the StrInString() to seperate them, note how I did that, otherwise all strings are passed.

Also, in the StringSplit... once you're done adding everything to the string, you have one delimeter too many on the end, so you have an unbalanced array, you have to use StringTrimRight() to get rid of that delimeter, also note where I did that.

[nitro322]

Ok... to get real unique strings, you have to use the delimeter you are using in the StrInString() to seperate them, note how I did that, otherwise all strings are passed.

Ok, that makes sense. In this case since I knew the delimeter was unique I didn't feel there was a need split up the string when comparing, but I see how this is not technically correct.

Also, in the StringSplit... once you're done adding everything to the string, you have one delimeter too many on the end, so you have an unbalanced array, you have to use StringTrimRight() to get rid of that delimeter, also note where I did that.

I fixed that in my revised copy. The if statement before the return should strip off the extra pipe at the end. Is that what you're talking about, or am I still missing something?

[SmOke_N]

Sorry, I didn't see the "If" statement, but there's no need for it, because it will always be true, so just put it in like StringSplit(StringTrimRight($seen, 1), '|') and you'll be fine.

[nitro322]

Yeah, that's true. I added the if statement just to be cautious, but I agree that it can be removed.

Thanks for the feedback.

[Burthold]

Just to add to the mix

This is my function to dedupe a single demention array it removes all dupes and only brings back an array.

Func _deleteduplicate($arrDupes)
    $count = ubound($arrDupes)
    For $x = 0 To $count - 2
        if ubound($arrDupes) > $x then
            $Itemtxt1 = $arrDupes[$x]
            For $y = $count - 1 To $x + 1 Step - 1
                $Itemtxt2 = $arrDupes[$y]
                If StringUpper($Itemtxt1) = StringUpper($Itemtxt2) Then
                    _ArrayDelete($arrDupes,$y)
                    $count = ubound($arrDupes)
                EndIf
            Next
        else
            ExitLoop
        endif
    Next
    Return $arrDupes
EndFunc

I tried to get away from using any functions inside the function but using the arraydelete was just too easy.

Wes

Edited October 3, 2006 by Burthold

[SmOke_N]

Just to add to the mix

This is my function to dedupe a single demention array it removes all dupes and only brings back an array.

Func _deleteduplicate($arrDupes)
    $count = ubound($arrDupes)
    For $x = 0 To $count - 2
        if ubound($arrDupes) > $x then
            $Itemtxt1 = $arrDupes[$x]
            For $y = $count - 1 To $x + 1 Step - 1
                $Itemtxt2 = $arrDupes[$y]
                If StringUpper($Itemtxt1) = StringUpper($Itemtxt2) Then
                    _ArrayDelete($arrDupes,$y)
                    $count = ubound($arrDupes)
                EndIf
            Next
        else
            ExitLoop
        endif
    Next
    Return $arrDupes
EndFunc

I tried to get away from using any functions inside the function but using the arraydelete was just too easy.

Wes

@Burt, that's going to be alot slower then the method I described (Had to edit one of mine above to work correctly)... To test the 2 (yours and mine), do a TimerInit()/TimerDiff()/ConsoleWrite() to see the difference.

Func _ArrayUnique(ByRef $aArray, $vDelim = '', $iBase = 1, $iUnique = 1)
    If $vDelim = '' Then $vDelim = Chr(01)
    Local $sHold
    For $iCC = $iBase To UBound($aArray) - 1
        If Not StringInStr($vDelim & $sHold, $vDelim & $aArray[$iCC] & $vDelim, $iUnique) Then _
            $sHold &= $aArray[$iCC] & $vDelim
    Next
    Return StringSplit(StringTrimRight($sHold, StringLen($vDelim)), $vDelim)
EndFuncoÝ÷ ØÚ0ü¨¹ªÞë-¶¼¢h­ën®{ayªëk,"¶.µ×ªâ­j¢*.r¥uÚ,¢g­)à)¶¬jëh×6#include <array.au3>;Only using this for ArrayDisplay
Dim $Array1[4] = ['apple', 'orange', 'Apple', 'Orange']
_ArrayUnique($Array1, Chr(1), 0);Case sensitive, should return all 4
Dim $Array2[4] = ['apple', 'orange', 'Apple', 'Orange']
_ArrayUnique($Array2, Chr(1), 0, 0);Not Case sensitive, should only return 2 of the 4
_ArrayDisplay($Array1, 'Test 1')
_ArrayDisplay($Array2, 'Test 2')

Func _ArrayUnique(ByRef $aArray, $vDelim = '', $iBase = 1, $iCase = 1)
    If Not IsArray($aArray) Then Return SetError(1, 0, 0)
    If $vDelim = '' Then $vDelim = Chr(01)
    Local $sHold
    For $iCC = $iBase To UBound($aArray) - 1
        If Not StringInStr($vDelim & $sHold, $vDelim & $aArray[$iCC] & $vDelim, $iCase) Then _
            $sHold &= $aArray[$iCC] & $vDelim
    Next
    If $sHold Then
        $aArray = StringSplit(StringTrimRight($sHold, StringLen($vDelim)), $vDelim)
        Return SetError(0, 0, 0)
    EndIf
    Return SetError(2, 0, 0)
EndFunc

No need for all the extra UDF's to be used

[Andrew Peacock]

Hi all,

I've been trying to get this to work, but it's not de-duplicating.. then I ran an example script below and it does de-dupe. So, here's my code - can anyone see why $arr does still contain dups, but $arr2 doesn't?

#include <Array.au3>
$arr = _arraycreate("so", "so")
$arr2 = _ArrayUnique($arr)
_ArrayDisplay($arr, "arr")
_ArrayDisplay($arr2, "arr2")
Func _ArrayUnique(ByRef $aArray, $vDelim = '', $iBase = 1, $iUnique = 1)
    If $vDelim = '' Then $vDelim = Chr(01)
    Local $sHold
    For $iCC = $iBase To UBound($aArray) - 1
        If Not StringInStr($vDelim & $sHold, $vDelim & $aArray[$iCC] & $vDelim, $iUnique) Then _
                $sHold &= $aArray[$iCC] & $vDelim
    Next
    Return StringSplit(StringTrimRight($sHold, StringLen($vDelim)), $vDelim)
EndFunc

Regards,

Andy