UnitTesting - Revision history

Jaberwocky6669: /* The Simplest Framework Possible */ Wikilink now points to Tricks #Add Your Own Include Path

2013-08-08T18:50:53Z

The Simplest Framework Possible: Wikilink now points to Tricks #Add Your Own Include Path

← Older revision		Revision as of 18:50, 8 August 2013
Line 12:		Line 12:

	==The Simplest Framework Possible==		==The Simplest Framework Possible==
	Let's create a "unittest.au3" file and save it in an include directory. A good thing to know is that you can have your own [[Tricks ~~Autoit Configuration~~ \|include directory]].		Let's create a "unittest.au3" file and save it in an include directory. A good thing to know is that you can have your own [[Tricks #Add Your Own Include Path \|include directory]].

	=== The UTAssert function ===		=== The UTAssert function ===

Jaberwocky6669: Code clean up, typos and grammer.

2013-07-27T05:10:54Z

Code clean up, typos and grammer.

Show changes

Nitekram at 02:40, 10 July 2013

2013-07-10T02:40:12Z

@@ Line 151: / Line 151: @@
 </source>
 ===How about some automation===
 AutoIt is all about automation. So a Unit Test tool for AutoIt should be able to help us out. The big question is actually. Can it be simpler than it already is? Probably not much while you write the code. But when you want to try your code against a new release of AutoIt then there are lots of things we can automate.

Rt01: Fixed typos

2013-05-03T10:24:58Z

Fixed typos

← Older revision		Revision as of 10:24, 3 May 2013
Line 11:		Line 11:
	AutoIt has moved from a limited special purpose scripting language to a real contender as an application language for small to medium sized projects. The developers of AutoIt have proven willingness to aggressively improve and extend the language and a core function library. Betas are released frequently and with an astonishing quality. It will surprise me and my colleges at Lexholm a lot if there is no automatic quality assurance philosophy and implementation used by the developers.		AutoIt has moved from a limited special purpose scripting language to a real contender as an application language for small to medium sized projects. The developers of AutoIt have proven willingness to aggressively improve and extend the language and a core function library. Betas are released frequently and with an astonishing quality. It will surprise me and my colleges at Lexholm a lot if there is no automatic quality assurance philosophy and implementation used by the developers.

	With the ~~methodes~~ we describe here you can improve the quality of your own code and help the developer team assure that each beta release is as good as or better than the previous.		With the methods we describe here you can improve the quality of your own code and help the developer team assure that each beta release is as good as or better than the previous.

	==What is Unit Testing==		==What is Unit Testing==

Rt01: Fixed typos

2013-05-03T10:22:50Z

Fixed typos

← Older revision		Revision as of 10:22, 3 May 2013
Line 7:		Line 7:

	==Initial notes==		==Initial notes==
	This is just a initial page. I have been ~~alowed~~ by Lexholm AS (www.lexholm.no) to improve and release a UnitTesting tool we have made. In the process I have also been ~~alowed~~ to do some documentation to ease adoption of unit testing in the AutoIt ~~comunity~~. This page will be about UnitTesting in general and how you can implement your own framework. If you want to use the tool by Lexholm when it is released (It is not documented at it's current state) that is fine. But our main goal with this article is to get you intrested in and caring so much for your code that you start using a automatic ~~aproach~~ to quality assurance of your own code. And ~~therby~~ AutoIt in general.		This is just a initial page. I have been allowed by Lexholm AS (www.lexholm.no) to improve and release a UnitTesting tool we have made. In the process I have also been allowed to do some documentation to ease adoption of unit testing in the AutoIt community. This page will be about UnitTesting in general and how you can implement your own framework. If you want to use the tool by Lexholm when it is released (it is not documented at it's current state) that is fine. But our main goal with this article is to get you intrested in and caring so much for your code that you start using an automatic approach to quality assurance of your own code. And thereby AutoIt in general.

	AutoIt has moved from a limited special purpose scripting language to a real contender as an application language for small to medium sized projects. The developers of AutoIt ~~has~~ proven willingness to ~~agresively~~ improve and extend the language and a core function library. Betas are released frequently and with an ~~asstonshing~~ quality. It will ~~suprice~~ me and my colleges at Lexholm a lot if there is no automatic quality assurance philosophy and implementation used by the developers.		AutoIt has moved from a limited special purpose scripting language to a real contender as an application language for small to medium sized projects. The developers of AutoIt have proven willingness to aggressively improve and extend the language and a core function library. Betas are released frequently and with an astonishing quality. It will surprise me and my colleges at Lexholm a lot if there is no automatic quality assurance philosophy and implementation used by the developers.

	With the methodes we describe here you can improve the quality of your own code and help the developer team assure that each beta release is as good as or better than the previous.		With the methodes we describe here you can improve the quality of your own code and help the developer team assure that each beta release is as good as or better than the previous.

Rt01: Removed typos

2013-05-03T09:33:30Z

Removed typos

← Older revision		Revision as of 09:33, 3 May 2013
Line 20:		Line 20:

	==Why should I adopt Unit Testing==		==Why should I adopt Unit Testing==
	Everytime you make a test you assure that some ~~pice~~ of your code works as intended. By following some simple rules you can make a simple automatic testing engine to make sure your code will work as you intend it to. If it does not you have to do improvements.		Everytime you make a test you assure that some piece of your code works as intended. By following some simple rules you can make a simple automatic testing engine to make sure your code will work as you intend it to. If it does not you have to do improvements.

	~~with~~ current and later release of AutoIt. If it does not. You either have to ~~decisde~~ to keep a version of AutoIt where all your tests ~~does~~ pass. Or you will have ~~a incredable~~ easy time locating the areas of your code where you have to do updates.		With current and later release of AutoIt. If it does not. You either have to decide to keep a version of AutoIt where all your tests do pass. Or you will have an incredible easy time locating the areas of your code where you have to do updates.

	==How do I go about to do that==		==How do I go about to do that==

IEvKI3gv9Wrkd41u: Additional page tagging. (Category,WIP,etc)

2012-11-13T18:05:49Z

Additional page tagging. (Category,WIP,etc)

← Older revision		Revision as of 18:05, 13 November 2012
Line 1:		Line 1:
			{{WIP}}[[Category:Tutorials]]

			__TOC__

	==Article in progress==		==Article in progress==
	'''NOTE:''' Article in progress so constructive input about content is welcome. If you have language comments, want to flame me for spelling or have fine-picking needs please wait until this section is removed. .....		'''NOTE:''' Article in progress so constructive input about content is welcome. If you have language comments, want to flame me for spelling or have fine-picking needs please wait until this section is removed. .....

JonTest: /* Running our first tests */

2010-05-10T16:14:43Z

Running our first tests

New page

==Article in progress==
'''NOTE:''' Article in progress so constructive input about content is welcome. If you have language comments, want to flame me for spelling or have fine-picking needs please wait until this section is removed. .....

==Initial notes==
This is just a initial page. I have been alowed by Lexholm AS (www.lexholm.no) to improve and release a UnitTesting tool we have made. In the process I have also been alowed to do some documentation to ease adoption of unit testing in the AutoIt comunity. This page will be about UnitTesting in general and how you can implement your own framework. If you want to use the tool by Lexholm when it is released (It is not documented at it's current state) that is fine. But our main goal with this article is to get you intrested in and caring so much for your code that you start using a automatic aproach to quality assurance of your own code. And therby AutoIt in general.

AutoIt has moved from a limited special purpose scripting language to a real contender as an application language for small to medium sized projects. The developers of AutoIt has proven willingness to agresively improve and extend the language and a core function library. Betas are released frequently and with an asstonshing quality. It will suprice me and my colleges at Lexholm a lot if there is no automatic quality assurance philosophy and implementation used by the developers.

With the methodes we describe here you can improve the quality of your own code and help the developer team assure that each beta release is as good as or better than the previous.

==What is Unit Testing==
Unit Testing is about caring for the code you write. It is about breaking the code down to testable units. When code is easy to test it is also easy to maintain and improve. Unit Testing is about quality assurance and confidence in the code you write. It is about investing a littel time upfront to gain on your investment when the project grows or has been put on ice for a while. Unoit Testing is basically putting together the simplest "best practise" rules you find about coding and make those rules second nature to you while you code.

==What Unit Testing is not==
It's not the silver bullet the "inventors" like it to be. Unit testing GUI applications can be a real challenge and sometimes impossible. But as your project evolves and you find hard to automate

==Why should I adopt Unit Testing==
Everytime you make a test you assure that some pice of your code works as intended. By following some simple rules you can make a simple automatic testing engine to make sure your code will work as you intend it to. If it does not you have to do improvements.

with current and later release of AutoIt. If it does not. You either have to decisde to keep a version of AutoIt where all your tests does pass. Or you will have a incredable easy time locating the areas of your code where you have to do updates.

==How do I go about to do that==

==The simplest framework posible==
Lets create a unittest.au3 file and save it in a include directory. A good thing to know is that you can have your own [[Tricks Autoit Configuration |include directory]]
=== The UTAssert function ===
Now open your unittest.au3 file and add the simplest possible assert function.
<source lang=autoit>
Func UTAssert($bool, $msg="Assert Failure", $erl=@ScriptLineNumber)
If NOT $bool Then
ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
EndIf
Return $bool
EndFunc
</source>

=== A sample project using UTAssert ===
To make good use of our unit testing framework we should make sure our code is nicely divided into well defined function. If you are a hard core unit tester you should even know that the philosophy is to write the test code first and then fill in the code to make the tests work.

So, let's see. We need a nice little project showing us what to do. Let's start with an really easy text book sample. We invent the function Add. Add can take two arguments. But if the first one is an array the second one is not needed. If two values are provided they are added and the sum is returned. If an array is provided all elements in the array is added together.
So a test setup would look something like this.
==== The first tests ====
<source lang=autoit>
UTAssert(Add(1) = 1)
UTAssert(Add(1,1) = 2)
UTAssert(Add(-1,1) = 0)
UTAssert(Add(-1,-1) = -2)
</source>
That was the easy part. We should add as many tests as we need to make sure the code we write will be working under all expected curcumstances.
Now we have to cover the array requirement for the code
<source lang=autoit>
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
UTAssert(Add($arg1) = 45)
</source>
At this point none of the tests will pass because we have not written the function yet.
<source lang=autoit>
==== The function we are testing ====
Func Add($arg1, $arg2=0)
Local $ret
If IsArray($arg1) OR IsArray($arg2) Then
Else
$ret = $arg1 + $arg2
EndIf
Return $ret
EndFunc
</source>

=== Running our first tests ===
Ok, So at this point we fire up the script and looks what it gives us
<source lang=autoit>
>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"
(23) := Assert Failure
+>AutoIT3.exe ended.rc:0
</source>
Obviously not so strange that the array test failed as we have not added any array code yet.
So, let's continue with our quest.

=== Adding some more code to fix failed tests ===
<source lang=autoit>
Func Add($arg1, $arg2=0)
Local $ret, $i
If IsArray($arg1) OR IsArray($arg2) Then
If IsArray($arg1) Then
For $i = 0 to UBound($arg1) - 1
$ret += Number($arg1[$i])
Next
Else
$ret += Number($arg1)
EndIf
Else
$ret = $arg1 + $arg2
EndIf
Return $ret
EndFunc
</source>
Another run with the tests we have created and.....Hey, it turnd out to be good. No red lines to jump to. That is really nice.

=== Reconsider our scenarios ===
Now it's time to ask. Did I write a test case for every possible use case I can think of? No, actually I did not. I'm missing several scenarios.
==== Adding more tests ====
<<source lang=autoit>>
Local $arg1[9] = [1, 2, 3, 4, 5, 6, 7, 8, 9]
UTAssert(Add($arg1, 10) = 55)
UTAssert(Add($arg1, $arg1) = 90) ;NOTE*
</source>
* Now this was not part of the original specification. Should I use it like this or should I return an error? If I return an error how will UTAssert react to it?

==== Running with the added tests ====
Anyhow, running this will show us that our function needs more work.
<source lang=autoit>
>Running:(3.2.2.0):E:\scite\..\autoit-v3.2.2.0\autoit3.exe "E:\CodeX\autoit\au3\au3UnitTest.au3"
(31) := Assert Failure
(32) := Assert Failure
+>AutoIT3.exe ended.rc:0

==== Adding a bit of code again ====
</source>
So, we take a look at our code. After considering our code we find that both failures are easy to fix. All we have to do is to duplicate the code
<source lang=autoit>
If IsArray($arg1) Then
For $i = 0 to UBound($arg1) - 1
$ret += Number($arg1[$i])
Next
Else
$ret += Number($arg1)
EndIf
</source>
And change the variable name $arg1 to $arg2.

==== Identifying duplicate code blocks. ArgSum ====
But hey, any part of the code that is a duplicate like that should be refactored to it's own function. And that function should be tested separately with unit tests.
So

<source lang=autoit>

Func ArgSum($arg)
Local $ret, $i
If IsArray($arg) Then
For $i = 0 to UBound($arg) - 1
$ret += Number($arg[$i])
Next
Else
$ret += Number($arg)
EndIf
Return $ret
EndFunc
</source>

===How about some automation===
AutoIt is all about automation. So a Unit Test tool for AutoIt should be able to help us out. The big question is actually. Can it be simpler than it already is? Probably not much while you write the code. But when you want to try your code against a new release of AutoIt then there are lots of things we can automate.
====Automatically running the function we are working in====
Now, that is a possibility. Say, all you have to do to run the function you are working in is to hit some key combination. To achieve this we would need:
* A method of identifying the function we are working in.
* A small program to write some code including the module we are working in and call the appropriate function.

As it is the SciTE4Autoit distribution has a way of identifying the function your caret is located in. You can observe this in the status bare on the left side. This information is also available through a variable.

====A simple runner application====
At this point we need a simple runner application. It has to:
* Accept function to run and script location.
* Wrap up the code to run the test function associated with the function.

<source lang=autoit>
;#include <unittest.au3>
;
Func UTAssert($bool, $msg="Assert Failure", $erl=@ScriptLineNumber, $error=@error, $extended=@extended)
If NOT $bool Then
ConsoleWrite("(" & $erl & ") := " & $msg & @LF)
EndIf
If $error <> 0 Then SetError($error, $extended, $error)
Return $bool
EndFunc
;
Func ParseCmdLine(ByRef $arr, ByRef $CmdLine)
#cs;$arr will be formated according to this
;$arr[0] = Array items Containing data. On top of that is maintenance items
;$arr[1 ... n] = File names
;$arr[n .... $arr[0]] = Function names
;$arr[$arr[0] + 1 = File names count
;This is given by [0] and previous $arr[$arr[0] + 2 = Function names count
#ce
Local $paths = 1
ReDim $arr[$CmdLine[0] + 2]
For $i = 0 to $CmdLine[0]
$arr[$i] = $CmdLine[$i]
Next
$arr[$arr[0] + 1] = $paths
EndFunc
;
Func err($msg, $nr, $terminate=0, $erl=@ScriptLineNumber)
dbg($msg, $nr, 0, $erl)
IF $terminate Then Exit
EndFunc
;
Func dbg($msg, $error = @error, $extended = @extended, $erl = @ScriptLineNumber)
ConsoleWrite("(" & $erl & ") : = (" & $error & ")(" & $extended & ") " & $msg & @LF)
If $error <> 0 Then SetError($error, $extended, $error)
Return $error
EndFunc ;==>dbg
;
Func testParseCmdLine()
Local $cmds[4] = [3, "testFunc1", "c:\test", "testFunc2"]
Local $arr[4]
ParseCmdLine($arr, $cmds)
UTAssert(IsArray($arr))
UTAssert($arr[0] = 3)
UTAssert($arr[$arr[0] +1] = 1)
UTAssert($arr[0] - $arr[$arr[0] +1] = 2)
UTAssert($arr[1] = "c:\test")
;TODO: Should we antisipate sequence of functions to call?
UTAssert($arr[1] = "testFunc1")
UTAssert($arr[1] = "testFunc2")
EndFunc
; =====================================================================
; Selftesting part
; =====================================================================
If StringInStr(@ScriptName, "au3UTRunner.au3") Then
testParseCmdLine()
EndIf
</source>

====Automatically running all test functions in the file we are working in=====

==Creating a UnitTestRunner==
Now here is an interesting topic. What is a test runner? The simplest approach again is just to use SciTE and watch for lines starting with parenthesis wrapped around a number. If done like this wee need a script calling all of our test functions. Actually not a bad solutions. It does not give you anny statistics. And we just love to know how many tests we pass each time.

But, as we work on our UDF's we can do something like this.
<source lang=autoit>
#include-once
#inlcude <myFuncLib.au3>
#include <UnitTest.au3>
If StringInStr(@ScriptName, "filename") Then
testMyFunc1()
EndFunc
Func testMyFunc1()
UTAssert(1 = 1, "Now this is a solution")
EndFunc
</source>
In Scite all we have to do now is hit F5 and look for those pesky error lines. Maybe not the cleanst possible but it is simple, and it works.

===How about some automation===
====Automatically running the function we are working in====
====Automatically running all test functions in the file we are working in=====

==Extending the framework==
===Testing advanced functions===
====Using a Setup function====
====Using a Teardown function====
==When our functions are to complex==
===Mock functions===
====The first attempt====
====Automating mock functions====

==Easing the code test code cycle==
The UTRunner is now ready to be integrated with SciTe4AutoIt.

==Automating all tests==
===When a new AutoIt version is released===