UnitTesting: Difference between revisions

From AutoIt Wiki
Jump to navigation Jump to search
mNo edit summary
m (→‎The Simplest Framework Possible: Wikilink now points to Tricks #Add Your Own Include Path)
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
{{WIP}}[[Category:Tutorials]]
{{WIP}}


__TOC__
Unit Testing is about caring for the code you write. It is about breaking the code down to testable units. When code is easy to test it is also easy to maintain and improve. Unit testing is about quality assurance and confidence in the code you write. It is about investing a little time upfront to gain on your investment when the project grows or has been put on ice for a while. Unit Testing is basically putting together the simplest "best practises" rules you find about coding and make those rules second nature to you while you code.


==Article in progress==
==What Unit Testing is Not==
'''NOTE:''' Article in progress so constructive input about content is welcome. If you have language comments, want to flame me for spelling or have fine-picking needs please wait until this section is removed. .....
It is not the silver bullet the "inventors" like it to be. Unit testing GUI applications can be a real challenge and sometimes impossible.


==Initial notes==
==Why Should I Adopt Unit Testing?==
This is just a initial page. I have been allowed by Lexholm AS (www.lexholm.no) to improve and release a UnitTesting tool we have made. In the process I have also been allowed to do some documentation to ease adoption of unit testing in the AutoIt community. This page will be about UnitTesting in general and how you can implement your own framework. If you want to use the tool by Lexholm when it is released (it is not documented at it's current state) that is fine. But our main goal with this article is to get you intrested in and caring so much for your code that you start using an automatic approach to quality assurance of your own code. And thereby AutoIt in general.
Every time you make a test you assure that some piece of your code works as intended. By following some simple rules you can make a simple automatic testing engine to make sure your code will work as you intend. If it does not you have to do improvements.


AutoIt has moved from a limited special purpose scripting language to a real contender as an application language for small to medium sized projects. The developers of AutoIt have proven willingness to aggressively improve and extend the language and a core function library. Betas are released frequently and with an astonishing quality. It will surprise me and my colleges at Lexholm a lot if there is no automatic quality assurance philosophy and implementation used by the developers.  
With current and later releases of AutoIt, if it does not then you either have to decide to keep a version of AutoIt where all your tests do pass. Or you will have an incredible easy time locating the areas of your code where you have to do updates.


With the methods we describe here you can improve the quality of your own code and help the developer team assure that each beta release is as good as or better than the previous.
==The Simplest Framework Possible==
Let's create a "unittest.au3" file and save it in an include directory. A good thing to know is that you can have your own [[Tricks #Add Your Own Include Path |include directory]].


==What is Unit Testing==
Unit Testing is about caring for the code you write. It is about breaking the code down to testable units. When code is easy to test it is also easy to maintain and improve. Unit Testing is about quality assurance and confidence in the code you write. It is about investing a littel time upfront to gain on your investment when the project grows or has been put on ice for a while. Unoit Testing is basically putting together the simplest "best practise" rules you find about coding and make those rules second nature to you while you code.
==What Unit Testing is not==
It's not the silver bullet the "inventors" like it to be. Unit testing GUI applications can be a real challenge and sometimes impossible. But as your project evolves and you find hard to automate
==Why should I adopt Unit Testing==
Everytime you make a test you assure that some piece of your code works as intended. By following some simple rules you can make a simple automatic testing engine to make sure your code will work as you intend it to. If it does not you have to do improvements.
With current and later release of AutoIt. If it does not. You either have to decide to keep a version of AutoIt where all your tests do pass. Or you will have an incredible easy time locating the areas of your code where you have to do updates.
==How do I go about to do that==
==The simplest framework posible==
Lets create a unittest.au3 file and save it in a include directory. A good thing to know is that you can have your own [[Tricks Autoit Configuration |include directory]]
=== The UTAssert function ===
=== The UTAssert function ===
Now open your unittest.au3 file and add the simplest possible assert function.
Now open your unittest.au3 file and add the simplest possible assert function.
<source lang=autoit>
<source lang=autoit>
Func UTAssert($bool, $msg="Assert Failure", $erl=@ScriptLineNumber)
Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber)
    If NOT $bool Then  
    If NOT $bool Then  
        ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
        ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
    EndIf
    EndIf
    Return $bool
   
    Return $bool
EndFunc
EndFunc
</source>
</source>


=== A sample project using UTAssert ===
=== A Sample Project Using UTAssert ===
To make good use of our unit testing framework we should make sure our code is nicely divided into well defined function. If you are a hard core unit tester you should even know that the philosophy is to write the test code first and then fill in the code to make the tests work.
To make good use of our unit testing framework we should make sure our code is nicely divided into well defined functions. If you are a hard core unit tester you should even know that the philosophy is to write the test code first and then fill in the code to make the tests work.
 
We need a nice little project showing us what to do. Let's start with a really easy text book example. We invent the function Add. Add can take two arguments. But if the first one is an array then the second one is not needed. If two values are provided they are added and the sum is returned. If an array is provided all elements in the array is added together. So a test setup would look something like this:


So, let's see. We need a nice little project showing us what to do. Let's start with an really easy text book sample. We invent the function Add. Add can take two arguments. But if the first one is an array the second one is not needed. If two values are provided they are added and the sum is returned. If an array is provided all elements in the array is added together.
So a test setup would look something like this.
==== The first tests ====
==== The first tests ====
<source lang=autoit>
<source lang=autoit>
Line 51: Line 39:
UTAssert(Add(-1,-1) = -2)
UTAssert(Add(-1,-1) = -2)
</source>
</source>
That was the easy part. We should add as many tests as we need to make sure the code we write will be working under all expected curcumstances.
That was the easy part. We should add as many tests as we need to make sure the code we write will be working under all expected curcumstances.
Now we have to cover the array requirement for the code
 
Now we have to cover the array requirement for the code.
 
<source lang=autoit>
<source lang=autoit>
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
UTAssert(Add($arg1) = 45)
UTAssert(Add($arg1) = 45)
</source>
</source>
At this point none of the tests will pass because we have not written the function yet.
At this point none of the tests will pass because we have not written the function yet.
==== The function Under Test ====
<source lang=autoit>
<source lang=autoit>
==== The function we are testing ====
Func Add(Const $arg1, Const $arg2 = 0)
Func Add($arg1, $arg2=0)
     Local $ret
     Local $ret
     If IsArray($arg1) OR IsArray($arg2) Then  
   
    Else
     If Not IsArray($arg1) Or Not IsArray($arg2) Then  
         $ret = $arg1 + $arg2
         $ret = $arg1 + $arg2
     EndIf
     EndIf
     
     Return $ret
     Return $ret
EndFunc
EndFunc
</source>
</source>


=== Running our first tests ===
=== Running Our First Tests ===
Ok, So at this point we fire up the script and looks what it gives us
Ok, So at this point we fire up the script and look at what it gives us:
 
<source lang=autoit>
<source lang=autoit>
>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"     
>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"     
Line 77: Line 72:
+>AutoIT3.exe ended.rc:0
+>AutoIT3.exe ended.rc:0
</source>
</source>
Obviously not so strange that the array test failed as we have not added any array code yet.
 
So, let's continue with our quest.
Obviously not so strange that the array test failed as we have not added any array code yet. So, let's continue with our quest.


=== Adding some more code to fix failed tests ===
=== Adding some more code to fix failed tests ===
<source lang=autoit>
<source lang=autoit>
Func Add($arg1, $arg2=0)
Func Add(const $arg1, const $arg2 = 0)
     Local $ret, $i
     Local $ret, $i
     If IsArray($arg1) OR IsArray($arg2) Then  
   
  If IsArray($arg1) Then  
     If IsArray($arg1) Or IsArray($arg2) Then  
For $i = 0 to UBound($arg1) - 1
      If IsArray($arg1) Then  
$ret += Number($arg1[$i])
      For $i = 0 to UBound($arg1) - 1
Next
        $ret += Number($arg1[$i])
  Else
      Next
$ret += Number($arg1)
      Else
  EndIf
      $ret += Number($arg1)
      EndIf
     Else
     Else
         $ret = $arg1 + $arg2
         $ret = $arg1 + $arg2
     EndIf
     EndIf
     
     Return $ret
     Return $ret
EndFunc
EndFunc
</source>
</source>
Another run with the tests we have created and.....Hey, it turnd out to be good. No red lines to jump to. That is really nice.


=== Reconsider our scenarios ===
Another run with the tests we have created and it turned out to be good. No red lines to jump to. That is really nice.
 
=== Reconsider Our Scenarios ===
Now it's time to ask. Did I write a test case for every possible use case I can think of? No, actually I did not. I'm missing several scenarios.  
Now it's time to ask. Did I write a test case for every possible use case I can think of? No, actually I did not. I'm missing several scenarios.  
==== Adding more tests ====
 
==== Adding More Tests ====
<<source lang=autoit>>
<<source lang=autoit>>
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
UTAssert(Add($arg1, 10) = 55)
UTAssert(Add($arg1, 10) = 55)
UTAssert(Add($arg1, $arg1) = 90) ;NOTE*
UTAssert(Add($arg1, $arg1) = 90) ;NOTE*
</source>
</source>
* Now this was not part of the original specification. Should I use it like this or should I return an error? If I return an error how will UTAssert react to it?
* Now this was not part of the original specification. Should I use it like this or should I return an error? If I return an error how will UTAssert react to it?


==== Running with the added tests ====
==== Running With the Added Tests ====
Anyhow, running this will show us that our function needs more work.
Running this will show us that our function needs more work.
 
<source lang=autoit>
<source lang=autoit>
>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"     
>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"     
Line 117: Line 120:
(32) := Assert Failure
(32) := Assert Failure
+>AutoIT3.exe ended.rc:0
+>AutoIT3.exe ended.rc:0
</source>


==== Adding a bit of code again ====
==== Adding a bit of code again ====
</source>
So, we take a look at our code. After considering our code we find that both failures are easy to fix. All we have to do is to duplicate the code
So, we take a look at our code. After considering our code we find that both failures are easy to fix. All we have to do is to duplicate the code
<source lang=autoit>
<source lang=autoit>
If IsArray($arg1) Then  
If IsArray($arg1) Then  
Line 130: Line 134:
EndIf
EndIf
</source>
</source>
And change the variable name $arg1 to $arg2.
And change the variable name $arg1 to $arg2.


==== Identifying duplicate code blocks. ArgSum ====
==== Identifying Duplicate Code Blocks. ====
But hey, any part of the code that is a duplicate like that should be refactored to it's own function. And that function should be tested separately with unit tests.
But hey, any part of the code that is a duplicate like that should be refactored to its own function. That function should be tested separately with unit tests.
So


<source lang=autoit>
<source lang=autoit>
Func ArgSum(Const $arg)
    Local $ret
    If IsArray($arg) Then
      For $i = 0 to UBound($arg) - 1
        $ret += Number($arg[$i])
      Next
    Else
      $ret += Number($arg)
    EndIf


Func ArgSum($arg)
    Return $ret
  Local $ret, $i
  If IsArray($arg) Then
  For $i = 0 to UBound($arg) - 1
$ret += Number($arg[$i])
  Next
  Else
  $ret += Number($arg)
  EndIf
  Return $ret
EndFunc
EndFunc
</source>
</source>


===So Putting this all together===
=== Putting This All Together ===
We have the following code:
We have the following code:


<source lang=autoit>
<source lang=autoit>
;#include-once<unittest.au3>
;#include <unittest.au3>
 
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
UTAssert(Add(39, 40) = 79)
UTAssert(Add(39, 40) = 79)
UTAssert(Add($arg1) = 45)
UTAssert(Add($arg1) = 45)
UTAssert(Add($arg1, 10) = 55)
UTAssert(Add($arg1, 10) = 55)
UTAssert(Add($arg1, $arg1) = 90) ;NOTE* Look above in the WIkI for the note about this test
 
;
UTAssert(Add($arg1, $arg1) = 90) ; NOTE* Look above in the Wiki for the note about this test
;
 
;
; The function under test.
;==== The function we are testing ====
Func Add(Const $arg1, Const$arg2 = 0)
Func Add($arg1, $arg2 = 0)
Return ArgSum($arg1) + ArgSum($arg2)
EndFunc  ;==>Add
 
Func ArgSum(Const $arg1)
Local $ret
Local $ret
$ret = ArgSum($arg1) + ArgSum($arg2)
    
Return $ret
EndFunc   ;==>Add
;
Func ArgSum($arg1)
Local $ret, $x
If IsArray($arg1) Then
If IsArray($arg1) Then
For $x = 0 To UBound($arg1) - 1
For $i = 0 To UBound($arg1) - 1
$ret += Number($arg1[$x])
$ret += Number($arg1[$i])
Next
Next
Else
Else
$ret = Number($arg1)
$ret = Number($arg1)
EndIf
EndIf
 
Return $ret
Return $ret
EndFunc  ;==>ArgSum
EndFunc  ;==>ArgSum
;
 
Func UTAssert($bool, $msg = "Assert Failure", $erl = @ScriptLineNumber)
Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber)
If Not $bool Then
If Not $bool Then
ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
EndIf
EndIf
 
Return $bool
Return $bool
EndFunc  ;==>UTAssert
EndFunc  ;==>UTAssert
</source>
</source>
===How about some automation===
 
AutoIt is all about automation. So a Unit Test tool for AutoIt should be able to help us out. The big question is actually. Can it be simpler than it already is? Probably not much while you write the code. But when you want to try your code against a new release of AutoIt then there are lots of things we can automate.
===How About Some Automation===
====Automatically running the function we are working in====
AutoIt is all about automation. So a unit test tool for AutoIt should be able to help us out. The big question is can it be simpler than it already is? Probably not much while you write the code. But when you want to try your code against a new release of AutoIt then there are lots of things we can automate.
Now, that is a possibility. Say, all you have to do to run the function you are working in is to hit some key combination. To achieve this we would need:
 
====Automatically Running the Function We Are Working In====
Now that is a possibility. Say, all you have to do to run the function you are working in is to hit some key combination. To achieve this we would need:
* A method of identifying the function we are working in.
* A method of identifying the function we are working in.
* A small program to write some code including the module we are working in and call the appropriate function.
* A small program to write some code including the module we are working in and call the appropriate function.


As it is the SciTE4Autoit distribution has a way of identifying the function your caret is located in. You can observe this in the status bare on the left side. This information is also available through a variable.
As it is, the SciTE4Autoit distribution has a way of identifying the function your caret is located in. You can observe this in the status bare on the left side. This information is also available through a variable.


====A simple runner application====
====A Simple Runner Application====
At this point we need a simple runner application. It has to:
At this point we need a simple runner application. It has to:
* Accept function to run and script location.
* Accept function to run and script location.
Line 206: Line 217:
<source lang=autoit>
<source lang=autoit>
;#include <unittest.au3>
;#include <unittest.au3>
;
 
Func UTAssert($bool, $msg="Assert Failure", $erl=@ScriptLineNumber, $error=@error, $extended=@extended)
Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber, Const $error = @error, Const $extended = @extended)
    If NOT $bool Then  
    If NOT $bool Then  
        ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
        ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
    EndIf
    EndIf
If $error <> 0 Then SetError($error, $extended, $error)
     
    Return $bool
    If $error <> 0 Then SetError($error, $extended, $error)
   
    Return $bool
EndFunc
EndFunc
;
 
Func ParseCmdLine(ByRef $arr, ByRef $CmdLine)
Func ParseCmdLine(ByRef $arr, ByRef $CmdLine)
#cs;$arr will be formated according to this
#cs;$arr will be formated according to this
Line 223: Line 236:
   ;This is given by [0] and previous $arr[$arr[0] + 2 = Function names count
   ;This is given by [0] and previous $arr[$arr[0] + 2 = Function names count
#ce   
#ce   
  Local $paths = 1
    Local $paths = 1
   ReDim $arr[$CmdLine[0] + 2]
    
   For $i = 0 to $CmdLine[0]
    ReDim $arr[$CmdLine[0] + 2]
  $arr[$i] = $CmdLine[$i]
    
  Next
    For $i = 0 to $CmdLine[0]
  $arr[$arr[0] + 1] = $paths
      $arr[$i] = $CmdLine[$i]
    Next
 
    $arr[$arr[0] + 1] = $paths
EndFunc
EndFunc
;
 
Func err($msg, $nr, $terminate=0, $erl=@ScriptLineNumber)
Func err(Const $msg, Const $nr, Const $terminate = 0, Const $erl = @ScriptLineNumber)
   dbg($msg, $nr, 0, $erl)
   dbg($msg, $nr, 0, $erl)
   IF $terminate Then Exit
   IF $terminate Then Exit
EndFunc
EndFunc
;
 
Func dbg($msg, $error = @error, $extended = @extended, $erl = @ScriptLineNumber)
Func dbg(Const $msg, Const $error = @error, Const $extended = @extended, Const $erl = @ScriptLineNumber)
ConsoleWrite("(" & $erl & ") : = (" & $error & ")(" & $extended & ") " & $msg & @LF)
ConsoleWrite("(" & $erl & ") : = (" & $error & ")(" & $extended & ") " & $msg & @LF)
 
If $error <> 0 Then SetError($error, $extended, $error)
If $error <> 0 Then SetError($error, $extended, $error)
 
Return $error
Return $error
EndFunc  ;==>dbg
EndFunc  ;==>dbg
;
 
Func testParseCmdLine()
Func testParseCmdLine()
   Local $cmds[4] = [3, "testFunc1", "c:\test", "testFunc2"]
   Local $cmds[4] = [3, "testFunc1", "c:\test", "testFunc2"]
 
   Local $arr[4]
   Local $arr[4]
 
   ParseCmdLine($arr, $cmds)
   ParseCmdLine($arr, $cmds)
 
   UTAssert(IsArray($arr))
   UTAssert(IsArray($arr))
 
   UTAssert($arr[0] = 3)
   UTAssert($arr[0] = 3)
 
   UTAssert($arr[$arr[0] +1] = 1)
   UTAssert($arr[$arr[0] +1] = 1)
 
   UTAssert($arr[0] - $arr[$arr[0] +1] = 2)
   UTAssert($arr[0] - $arr[$arr[0] +1] = 2)
 
   UTAssert($arr[1] = "c:\test")
   UTAssert($arr[1] = "c:\test")
   ;TODO: Should we antisipate sequence of functions to call?
 
   ;TODO: Should we anticipate the sequence of functions to call?
 
   UTAssert($arr[1] = "testFunc1")
   UTAssert($arr[1] = "testFunc1")
 
   UTAssert($arr[1] = "testFunc2")
   UTAssert($arr[1] = "testFunc2")
EndFunc
EndFunc
; =====================================================================
; =====================================================================
; Selftesting part
; Selftesting part
Line 263: Line 292:
</source>
</source>


====Automatically running all test functions in the file we are working in=====
==Creating a UnitTest Runner==
Now here is an interesting topic. What is a test runner? The simplest approach again is just to use SciTE and watch for lines starting with parenthesis wrapped around a number. If done like this we need a script calling all of our test functions. Actually not a bad solution. It does not give you any statistics. And we just love to know how many tests we pass each time.


==Creating a UnitTestRunner==
But as we work on our UDFs we can do something like this.
Now here is an interesting topic. What is a test runner? The simplest approach again is just to use SciTE and watch for lines starting with parenthesis wrapped around a number. If done like this wee need a script calling all of our test functions. Actually not a bad solutions. It does not give you anny statistics. And we just love to know how many tests we pass each time.  


But, as we work on our UDF's we can do something like this.
<source lang=autoit>
<source lang=autoit>
#include-once
#include-once
#inlcude <myFuncLib.au3>
 
#include <UnitTest.au3>
#include "myFuncLib.au3"
 
#include "UnitTest.au3"
 
If StringInStr(@ScriptName, "filename") Then  
If StringInStr(@ScriptName, "filename") Then  
     testMyFunc1()
     testMyFunc1()
EndFunc
EndFunc
 
Func testMyFunc1()
Func testMyFunc1()
     UTAssert(1 = 1, "Now this is a solution")
     UTAssert(1 = 1, "Now this is a solution")
EndFunc
EndFunc
</source>
</source>
In Scite all we have to do now is hit F5 and look for those pesky error lines. Maybe not the cleanst possible but it is simple, and it works.
===How about some automation===
====Automatically running the function we are working in====
====Automatically running all test functions in the file we are working in=====
==Extending the framework==
===Testing advanced functions===
====Using a Setup function====
====Using a Teardown function====
==When our functions are to complex==
===Mock functions===
====The first attempt====
====Automating mock functions====


==Easing the code test code cycle==
In SciTE all we have to do now is press F5 and look for those pesky error lines. Maybe not the cleanest possible but it is simple and it works.
The UTRunner is now ready to be integrated with SciTe4AutoIt.


==Automating all tests==
[[Category:Tutorials]]
===When a new AutoIt version is released===

Latest revision as of 18:50, 8 August 2013

This page is still a work in progress.

Unit Testing is about caring for the code you write. It is about breaking the code down to testable units. When code is easy to test it is also easy to maintain and improve. Unit testing is about quality assurance and confidence in the code you write. It is about investing a little time upfront to gain on your investment when the project grows or has been put on ice for a while. Unit Testing is basically putting together the simplest "best practises" rules you find about coding and make those rules second nature to you while you code.

What Unit Testing is Not

It is not the silver bullet the "inventors" like it to be. Unit testing GUI applications can be a real challenge and sometimes impossible.

Why Should I Adopt Unit Testing?

Every time you make a test you assure that some piece of your code works as intended. By following some simple rules you can make a simple automatic testing engine to make sure your code will work as you intend. If it does not you have to do improvements.

With current and later releases of AutoIt, if it does not then you either have to decide to keep a version of AutoIt where all your tests do pass. Or you will have an incredible easy time locating the areas of your code where you have to do updates.

The Simplest Framework Possible

Let's create a "unittest.au3" file and save it in an include directory. A good thing to know is that you can have your own include directory.

The UTAssert function

Now open your unittest.au3 file and add the simplest possible assert function.

Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber)
    If NOT $bool Then 
        ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
    EndIf
    
    Return $bool
EndFunc

A Sample Project Using UTAssert

To make good use of our unit testing framework we should make sure our code is nicely divided into well defined functions. If you are a hard core unit tester you should even know that the philosophy is to write the test code first and then fill in the code to make the tests work.

We need a nice little project showing us what to do. Let's start with a really easy text book example. We invent the function Add. Add can take two arguments. But if the first one is an array then the second one is not needed. If two values are provided they are added and the sum is returned. If an array is provided all elements in the array is added together. So a test setup would look something like this:

The first tests

UTAssert(Add(1) = 1)
UTAssert(Add(1,1) = 2)
UTAssert(Add(-1,1) = 0)
UTAssert(Add(-1,-1) = -2)

That was the easy part. We should add as many tests as we need to make sure the code we write will be working under all expected curcumstances.

Now we have to cover the array requirement for the code.

Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
UTAssert(Add($arg1) = 45)

At this point none of the tests will pass because we have not written the function yet.

The function Under Test

Func Add(Const $arg1, Const $arg2 = 0)
    Local $ret
    
    If Not IsArray($arg1) Or Not IsArray($arg2) Then 
        $ret = $arg1 + $arg2
    EndIf
      
    Return $ret
EndFunc

Running Our First Tests

Ok, So at this point we fire up the script and look at what it gives us:

>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"    
(23) := Assert Failure
+>AutoIT3.exe ended.rc:0

Obviously not so strange that the array test failed as we have not added any array code yet. So, let's continue with our quest.

Adding some more code to fix failed tests

Func Add(const $arg1, const $arg2 = 0)
    Local $ret, $i
    
    If IsArray($arg1) Or IsArray($arg2) Then 
      If IsArray($arg1) Then 
       For $i = 0 to UBound($arg1) - 1
        $ret += Number($arg1[$i])
       Next
      Else
       $ret += Number($arg1)
      EndIf
    Else
        $ret = $arg1 + $arg2
    EndIf
      
    Return $ret
EndFunc

Another run with the tests we have created and it turned out to be good. No red lines to jump to. That is really nice.

Reconsider Our Scenarios

Now it's time to ask. Did I write a test case for every possible use case I can think of? No, actually I did not. I'm missing several scenarios.

Adding More Tests

<

>
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]

UTAssert(Add($arg1, 10) = 55)

UTAssert(Add($arg1, $arg1) = 90) ;NOTE*
  • Now this was not part of the original specification. Should I use it like this or should I return an error? If I return an error how will UTAssert react to it?

Running With the Added Tests

Running this will show us that our function needs more work.

>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"    
(31) := Assert Failure
(32) := Assert Failure
+>AutoIT3.exe ended.rc:0

Adding a bit of code again

So, we take a look at our code. After considering our code we find that both failures are easy to fix. All we have to do is to duplicate the code

If IsArray($arg1) Then 
   For $i = 0 to UBound($arg1) - 1
	  $ret += Number($arg1[$i])
   Next
Else
   $ret += Number($arg1)
EndIf

And change the variable name $arg1 to $arg2.

Identifying Duplicate Code Blocks.

But hey, any part of the code that is a duplicate like that should be refactored to its own function. That function should be tested separately with unit tests.

Func ArgSum(Const $arg)
    Local $ret

    If IsArray($arg) Then 
      For $i = 0 to UBound($arg) - 1
        $ret += Number($arg[$i])
      Next
    Else
      $ret += Number($arg)
    EndIf

    Return $ret
EndFunc

Putting This All Together

We have the following code:

;#include <unittest.au3>

Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]

UTAssert(Add(39, 40) = 79)

UTAssert(Add($arg1) = 45)

UTAssert(Add($arg1, 10) = 55)

UTAssert(Add($arg1, $arg1) = 90) ; NOTE* Look above in the Wiki for the note about this test

; The function under test.
Func Add(Const $arg1, Const$arg2 = 0)
	Return ArgSum($arg1) + ArgSum($arg2)
EndFunc   ;==>Add

Func ArgSum(Const $arg1)
	Local $ret
  
	If IsArray($arg1) Then
		For $i = 0 To UBound($arg1) - 1
			$ret += Number($arg1[$i])
		Next
	Else
		$ret = Number($arg1)
	EndIf
  
	Return $ret
EndFunc   ;==>ArgSum

Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber)
	If Not $bool Then
		ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
	EndIf
  
	Return $bool
EndFunc   ;==>UTAssert

How About Some Automation

AutoIt is all about automation. So a unit test tool for AutoIt should be able to help us out. The big question is can it be simpler than it already is? Probably not much while you write the code. But when you want to try your code against a new release of AutoIt then there are lots of things we can automate.

Automatically Running the Function We Are Working In

Now that is a possibility. Say, all you have to do to run the function you are working in is to hit some key combination. To achieve this we would need:

  • A method of identifying the function we are working in.
  • A small program to write some code including the module we are working in and call the appropriate function.

As it is, the SciTE4Autoit distribution has a way of identifying the function your caret is located in. You can observe this in the status bare on the left side. This information is also available through a variable.

A Simple Runner Application

At this point we need a simple runner application. It has to:

  • Accept function to run and script location.
  • Wrap up the code to run the test function associated with the function.
;#include <unittest.au3>

Func UTAssert(Const $bool, Const $msg = "Assert Failure", Const $erl = @ScriptLineNumber, Const $error = @error, Const $extended = @extended)
    If NOT $bool Then 
        ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
    EndIf
      
    If $error <> 0 Then SetError($error, $extended, $error)
    
    Return $bool
EndFunc

Func ParseCmdLine(ByRef $arr, ByRef $CmdLine)
#cs;$arr will be formated according to this
   ;$arr[0] = Array items Containing data. On top of that is maintenance items
   ;$arr[1 ... n] = File names
   ;$arr[n .... $arr[0]] = Function names
   ;$arr[$arr[0] + 1 = File names count
   ;This is given by [0] and previous $arr[$arr[0] + 2 = Function names count
#ce   
    Local $paths = 1
   
    ReDim $arr[$CmdLine[0] + 2]
   
    For $i = 0 to $CmdLine[0]
      $arr[$i] = $CmdLine[$i]
    Next
  
    $arr[$arr[0] + 1] = $paths
EndFunc

Func err(Const $msg, Const $nr, Const $terminate = 0, Const $erl = @ScriptLineNumber)
   dbg($msg, $nr, 0, $erl)
   IF $terminate Then Exit
EndFunc

Func dbg(Const $msg, Const $error = @error, Const $extended = @extended, Const $erl = @ScriptLineNumber)
	ConsoleWrite("(" & $erl & ") : = (" & $error & ")(" & $extended & ") " & $msg & @LF)
  
	If $error <> 0 Then SetError($error, $extended, $error)
  
	Return $error
EndFunc   ;==>dbg

Func testParseCmdLine()
   Local $cmds[4] = [3, "testFunc1", "c:\test", "testFunc2"]
   
   Local $arr[4]
   
   ParseCmdLine($arr, $cmds)
   
   UTAssert(IsArray($arr))
   
   UTAssert($arr[0] = 3)
   
   UTAssert($arr[$arr[0] +1] = 1)
   
   UTAssert($arr[0] - $arr[$arr[0] +1] = 2)
   
   UTAssert($arr[1] = "c:\test")
   
   ;TODO: Should we anticipate the sequence of functions to call?
   
   UTAssert($arr[1] = "testFunc1")
   
   UTAssert($arr[1] = "testFunc2")
 EndFunc
 
; =====================================================================
; Selftesting part
; =====================================================================
If StringInStr(@ScriptName, "au3UTRunner.au3") Then 
   testParseCmdLine()
EndIf

Creating a UnitTest Runner

Now here is an interesting topic. What is a test runner? The simplest approach again is just to use SciTE and watch for lines starting with parenthesis wrapped around a number. If done like this we need a script calling all of our test functions. Actually not a bad solution. It does not give you any statistics. And we just love to know how many tests we pass each time.

But as we work on our UDFs we can do something like this.

#include-once

#include "myFuncLib.au3"

#include "UnitTest.au3"

If StringInStr(@ScriptName, "filename") Then 
    testMyFunc1()
EndFunc
  
Func testMyFunc1()
    UTAssert(1 = 1, "Now this is a solution")
EndFunc

In SciTE all we have to do now is press F5 and look for those pesky error lines. Maybe not the cleanest possible but it is simple and it works.