detecting a difference between two arrays

[nullschritt]

I was was wondering if there was a way to quickly spot a difference between two arrays, say I have to arrays and as soon as they mismatch return where.

Now that I think about it I can probably code something that does this, though I don't know if it will be the most efficient way. My idea will be to loop through the arrays and if value from array 1 <> value from array 2 return. Though there might be a better way?

[JohnOne]

As far as I;m aware, that is the only way.

[l3ill]

I recall doing something like this before. I cant find it now but it invlolved ArrayToString and then StringCompare.

[JohnOne]

ArrayToString loops through the array, StringCompare will effectively loop through it again.

[jaberwacky]

Why do you have two of the same arrays?

[mikell]

With 'looping through arrays' it will be very long if the arrays are large

As on the forum I could only find funcs checking if arrays are the same or not, for personal use I made this to return the differences (with the line #)

Very fast even with arrays having several thousands lines

expandcollapse popup

#include <SQLite.au3>
#include <SQLite.dll.au3>
#include <Array.au3>

; example WindowsConstants.au3 
$file = FileRead(StringRegExpReplace(@Autoitexe, '(.+)\\[^\\]+', "$1") & "\Include\WindowsConstants.au3")
$aLines = StringRegExp($file, '(?m)(^.*)\R?', 3)   ; array 1

$aLines2 = $aLines   ; array 2
For $i = 15 to UBound($aLines)-1 step 100    ; introduce differences
    _ArrayInsert($aLines, $i, "Global Const $WS_FAKE = extra line*" & Ceiling($i/100))
    _ArrayInsert($aLines2, $i+3, "Global Const $WS_EX_FAKE = extra line 2*" & Ceiling($i/100))
    _ArrayInsert($aLines2, $i+6, "Global Const $WM_FAKE = extra line 2*" & Ceiling($i/100) & "bis")
Next
; _ArrayDisplay($aLines)


$res = _ArrayCompareAndGetResults($aLines, $aLines2, 0)
 _ArrayDisplay($res)


Func _ArrayCompareAndGetResults($array1, $array2, $flag = 0)
  If $flag < 0 OR $flag > 2 Then $flag = 0
  Local $array, $aTemp, $iRows, $iColumns
  _SQLite_Startup()
  _SQLite_Open()   ; ':memory:'
  _SQLite_Exec (-1, "CREATE TABLE table1 (id, items1); CREATE TABLE table2 (id, items2);") 
  _SQLite_Exec(-1, "Begin;")
  For $i = 0 to UBound($array1)-1
        _SQLite_Exec(-1, "INSERT INTO table1 VALUES (" & $i & ", " & _SQLite_FastEscape($array1[$i]) & ");")
  Next
  For $i = 0 to UBound($array2)-1
        _SQLite_Exec(-1, "INSERT INTO table2 VALUES (" & $i & ", " & _SQLite_FastEscape($array2[$i]) & ");")
  Next
  _SQLite_Exec(-1, "Commit;")

 Switch $flag
   Case 1
     _SQLite_GetTable2d(-1, "SELECT * FROM table1 " & _
        "WHERE items1 NOT IN (SELECT items2 FROM table2) ;", $array, $iRows, $iColumns)   
   Case 2
     _SQLite_GetTable2d(-1, "SELECT * FROM table2 " & _
        "WHERE items2 NOT IN (SELECT items1 FROM table1) ;", $array, $iRows, $iColumns)  
   Case 0
     _SQLite_GetTable2d(-1, "SELECT * FROM table1 " & _
        "WHERE items1 NOT IN (SELECT items2 FROM table2) ;", $array, $iRows, $iColumns)   
     _SQLite_GetTable2d(-1, "SELECT * FROM table2 " & _ 
        "WHERE items2 NOT IN (SELECT items1 FROM table1) ;", $aTemp, $iRows, $iColumns)  
      Local $n = UBound($array)-1, $m = UBound($aTemp)-1
      Local $s = ($n > $m) ? $n : $m
      Redim $array[$s+1][4]
      For $i = 0 to $m
        $array[$i][2] = $aTemp[$i][0]
        $array[$i][3] = $aTemp[$i][1]
      Next
      $array[0][0] = $n
      $array[0][1] = ""
      $array[0][2] = $m
      $array[0][3] = ""
  EndSwitch
  _SQLite_Close()
  _SQLite_Shutdown()
    Return $array
EndFunc

[nullschritt]

Why do you have two of the same arrays?

They arn't always the same. One holds current data, the other holds data after it's been modified. I need to retrieve the point at which the data changed, so that I can update a database, without clearing it out and re-inserting all values. 

 

 

With 'looping through arrays' it will be very long if the arrays are large

As on the forum I could only find funcs checking if arrays are the same or not, for personal use I made this to return the differences (with the line #)

Very fast even with arrays having several thousands lines

 

I'll check it out soon. One array is 1d and the other is 2d, but I can easily scale the 2d down for comparison. Though your function seems redundant. You are still looping through every value of both arrays, in order to place them in the database In fact you're using two separate loops to do it even.

This can be accomplished with 1 loop using if logic, perhaps the way I thought up would be the most efficient.

Edited February 7, 2014 by nullschritt

[nullschritt]

Func _arraygetdifference($array1, $array2)
local $pos= 'Null'
  For $i = 0 to UBound($array1)-1
        if $array1[$i] <> $array2[$i]
            $pos = $i
             exitloop
        endif
  Next

  return $pos
EndFunc

Edited February 7, 2014 by nullschritt

[orbs]

if you control the creation of the arrays, the fastest way would be to do a comparison against the old data as you add a value to the current data.

[spudw2k]

I'm not sure how you are using the arrays exactly, but do you really only want to find the "first" difference and skip the rest?  By your design can there only be one changed value?

My thought on the approach would be to have a two dim array with one dim holding the old values and the other holding the new arrays.  This would allow you to scrub through the whole array and see every change.

Just my two cents, but like I said...I'm not sure the scope so it may not suit your purpose.

[nullschritt]

I'm not sure how you are using the arrays exactly, but do you really only want to find the "first" difference and skip the rest?  By your design can there only be one changed value?

My thought on the approach would be to have a two dim array with one dim holding the old values and the other holding the new arrays.  This would allow you to scrub through the whole array and see every change.

Just my two cents, but like I said...I'm not sure the scope so it may not suit your purpose.

By design my application only allows one change to occur at a time. Also the second array of new data, is returned when an event occurs. The maximum number of rows returned possible is also limited to 25,000 by design, which takes a maximum of about 0.1 seconds to enumerate through. The returned data doesn't always get routed to the same function either, each function keeps track of the previous data for itself, which is what the new returned data is compared against.

[mikell]

nullschritt,

I made this func to grab the content and the # of ALL different lines between array1/array2 AND array2/array1

For this the SQlite engine is a lot faster and efficient than the usual ways

Of course if all you need is to know if the arrays are different or not, there is a bunch of simple ArrayCompare example codes on the forum

[spudw2k]

I have to imagine there is more elegant way to handle the change of one item/entity/element (what have you) without having to process an entire array, but without knowing the inner workings I can't really recommend anything.  It just seems inefficient to me, especially when you could have 25,000 results to filter through (even if it does so quickly).  It will always be faster to deal with the single change than have to dig for it. 

[nullschritt]

I have to imagine there is more elegant way to handle the change of one item/entity/element (what have you) without having to process an entire array, but without knowing the inner workings I can't really recommend anything.  It just seems inefficient to me, especially when you could have 25,000 results to filter through (even if it does so quickly).  It will always be faster to deal with the single change than have to dig for it. 

there's no way of locating any point within an array without enumerating it, at some point., even for the function getting the changed data(from a gui control), I would have to enumerate all the values for the control, to check what has changed.

[spudw2k]

there's no way of locating any point within an array without enumerating it, at some point., even for the function getting the changed data(from a gui control), I would have to enumerate all the values for the control, to check what has changed.

Understood. What I was eluding to is there may be an alternative method to track individual changes without having to utilize an array.

edit:

I don't disagree that an array is a good way to store data for a series of controls, but rather than having to search through an array a mechanism could be made to tie a control to an index of an array and go straight to a desired value. It just seems overkill and inefficient to me to have to scrub and array for a single change.

Edited February 7, 2014 by spudw2k

[guinness]

Func _arraygetdifference($array1, $array2)
local $pos=0
  For $i = 0 to UBound($array1)-1
        if $array1[$i] <> $array2[$i]
            $pos = $i
             exitloop
        endif
  Next

  return $pos
EndFunc

That assumes the arrays are the same size. Edited February 7, 2014 by guinness

[JohnOne]

In order to update an element in an array, there must be code to do that, either do the check on the fly, or iterate it in its entirety.

[nullschritt]

That assumes the arrays are the same size.

Again by design the two sets of data are always identical in size, it's only data in the values which can change.

[nullschritt]

Understood. What I was eluding to is there may be an alternative method to track individual changes without having to utilize an array.

edit:

I don't disagree that an array is a good way to store data for a series of controls, but rather than having to search through an array a mechanism could be made to tie a control to an index of an array and go straight to a desired value. It just seems overkill and inefficient to me to have to scrub and array for a single change.

Care to share an example?

[mikell]

That assumes the arrays are the same size.

 

And even if so, if one insert is done at index 0 in array2, you will get UBound($array) differences whereas only the index changes