Jump to content

Test Driven Development Framework

jaberwacky

By jaberwacky
in AutoIt Example Scripts

 Share

Recommended Posts

jaberwacky Universalist

jaberwacky

Posted
Posted (edited)

Good morning, (10:15 AM Eastern Time)

This morning I began the seed of a possible TDD framework. At this early stage it isn't exactly production quality. It doesn't have a fancy GUI or code coverage tools. The only way to see of a test failed is that the test will be red in the console. Let's discuss limitations, possibilities, my current code quality or lack thereof if you'd like. I invite any interested party to join with me in developing this code. I'll answer as many questions as possible.

Future addition(s):

1) A GUI displaying the tests and a green or red light indicating pass/fail status of each test. The GUI should make most of the code that is in the example unneccesary.

07-19-2010 -- Latest update makes adding and running tests more straight forward

08-01-2010 -- Latest update adds more assertions to the framework.

This example should have everything you need to hit the ground running.

Example.au3

#region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=n
#AutoIt3Wrapper_Au3Check_Parameters=-w 1 -w 2 -w 3 -w 4 -w 6 -d
#endregion ;**** Directives created by AutoIt3Wrapper_GUI ****

#region Includes
#include <AutoItObject.au3>
#include <TDD Framework.au3>
#endregion Includes

_AutoItObject_Startup()

Global $Assert = Tests()

; run the tests
$Assert.Example()
$Assert.Example1()
$Assert.ShouldBeEqual32()
$Assert.ShouldBeEqual64()

$Assert = ''

_AutoItObject_Shutdown()

#region Tests
Func Tests() ; Add your unit tests to this class...
    ; inherit the class under development in order to have access to private members and methods
    Local $this = _AutoItObject_Create(Calculator())

    _AutoItObject_AddProperty($this, "Result", $ELSCOPE_PRIVATE)

    _AutoItObject_AddProperty($this, "TDD", $ELSCOPE_PRIVATE, TDD())

    ; define the data that the tests may need...
    _AutoItObject_AddProperty($this, "INT32_MAX", $ELSCOPE_PRIVATE, 2147483647)
    _AutoItObject_AddProperty($this, "INT32_MIN", $ELSCOPE_PRIVATE, -2147483648)
    _AutoItObject_AddProperty($this, "INT64_MAX", $ELSCOPE_PRIVATE, 9223372036854775807)
    _AutoItObject_AddProperty($this, "INT64_MIN", $ELSCOPE_PRIVATE, -9223372036854775808)

    ; define your unit tests here...
    _AutoItObject_AddMethod($this, "Example", "Tests_Example")
    _AutoItObject_AddMethod($this, "Example1", "Tests_Example1")
    _AutoItObject_AddMethod($this, "ShouldBeEqual32", "Tests_ShouldBeEqual32")
    _AutoItObject_AddMethod($this, "ShouldBeEqual64", "Tests_ShouldBeEqual64")

    Return $this
EndFunc ;==>Tests

Func Tests_Example($this)
    $this.TDD.AreEqual(1, 0, "Example")
EndFunc ;==>Tests_Example

Func Tests_Example1($this)
    $this.TDD.AreEqual(0, 0, "Example1")
EndFunc ;==>Tests_Example1

Func Tests_ShouldBeEqual32($this)
    $this.Result = $this.Sum($this.INT32_MAX, 1)
    $this.TDD.AreEqual("overflow", $this.Result, "ShouldBeEqual32")
EndFunc ;==>Tests_ShouldBeEqual32

Func Tests_ShouldBeEqual64($this)
    $this.Result = $this.Sum($this.INT64_MAX, 1)
    $this.TDD.AreEqual("overflow", $this.Result, "ShouldBeEqual64")
EndFunc ;==>Tests_ShouldBeEqual64
#endregion Tests

; @@@@@@@@@@@@@@ This is an example class that is being developed with TDD @@@@@@@@@@@@@@@@@@@
#region Calculator Class
Func Calculator()
    Local $this = _AutoItObject_Create()
    _AutoItObject_AddMethod($this, "Sum", "Calculator_Sum")
    Return $this
EndFunc ;==>Calculator

; this is an example method that is being iteratively refactored according to the TDD method
Func Calculator_Sum($this, Const $p, Const $q)
    If ($p >= 0 And $q >= 0) Or ($p < 0 And $q < 0) Then ; possibility of overflow condition
        If $q >= 0 Then
            Local Const $result = $p + $q

            If $result < $p Then ; overfloweth
                Return SetError(1, 0, "overflow")
            Else

                Return $result
            EndIf
        EndIf
    Else ; no possibility of overflow
        Return $p + $q
    EndIf
EndFunc ;==>Calculator_Sum
#endregion Calculator Class

TDD Framework.au3

#AutoIt3Wrapper_UseX64=n
#AutoIt3Wrapper_Au3Check_Parameters=-w 1 -w 2 -w 3 -w 4 -w 6 -d
#AutoIt3Wrapper_Run_After=md "%scriptdir%\Versions\%fileversion%"
#AutoIt3Wrapper_Run_After=copy "%in%" "%scriptdir%\Versions\%fileversion%\%scriptfile%%fileversion%.au3"
#AutoIt3Wrapper_Run_After=copy "%out%" "%scriptdir%\Versions\%fileversion%\%scriptfile%%fileversion%.exe"

;**** Function Name List ****
; TDD
; TDD_Write
; TDD_Equal
; TDD_NotEqual
; TDD_FileExists
; TDD_FileDoesNotExist
; TDD_GreaterThan
; TDD_NotGreaterThan
; TDD_GreaterThanOrEqualTo
; TDD_NotGreaterThanOrEqualTo
; TDD_LessThan
; TDD_NotLessThan
; TDD_LessThanOrEqualTo
; TDD_NotLessThanOrEqualTo
; TDD_Object
; TDD_NotObject
; TDD_ProcessExists
; TDD_ProcessDoesNotExist
; TDD_True
; TDD_NotTrue
; TDD_False
; TDD_NotFalse
; TDD_WindowExists
; TDD_WindowDoesNotExist

#include-once
#include "AutoItObject.au3"

Func TDD()
    Local $This = _AutoItObject_Class()

    $This.AddMethod( "Equal" , "TDD_Equal" )
    $This.AddMethod( "NotEqual" , "TDD_NotEqual" )

    $This.AddMethod( "FileExists" , "TDD_FileExists" )
    $This.AddMethod( "NotFileExists" , "TDD_NotFileExists" )

    $This.AddMethod( "GreaterThan" , "TDD_GreaterThan" )
    $This.AddMethod( "NotGreaterThan" , "TDD_NotGreaterThan" )

    $This.AddMethod( "GreaterThanOrEqualTo" , "TDD_GreaterThanOrEqualTo" )
    $This.AddMethod( "NotGreaterThanOrEqualTo" , "TDD_NotGreaterThanOrEqualTo" )

    $This.AddMethod( "LessThan" , "TDD_LessThan" )
    $This.AddMethod( "NotLessThan" , "TDD_LessThan" )

    $This.AddMethod( "LessThanOrEqualTo" , "TDD_LessThanOrEqualTo" )
    $This.AddMethod( "NotLessThanOrEqualTo" , "TDD_NotLessThanOrEqualTo" )

    $This.AddMethod( "Object" , "TDD_Object" )
    $This.AddMethod( "NotObject" , "TDD_NotObject" )

    $This.AddMethod( "ProcessExists" , "TDD_ProcessExists" )
    $This.AddMethod( "NotProcessExists" , "TDD_NotProcessExists" )

    $This.AddMethod( "True" , "TDD_True" )
    $This.AddMethod( "NotTrue" , "TDD_NotTrue" )

    $This.AddMethod( "False" , "TDD_False" )
    $This.AddMethod( "NotFalse" , "TDD_NotFalse" )

    $This.AddMethod( "WindowExists" , "TDD_WindowExists" )
    $This.AddMethod( "NotWindowExists" , "TDD_NotWindowExists" )

    $This.AddMethod( "Write" , "_TDD_Write" , True )

    Return $This.Object
EndFunc

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_Equal( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Equal" )
    Local Const $info_string = $testname & "'  |  " & "Expected: '" & $var1 & "'  |  Is: '" & $var2 & "'"
    Local Const $Pass = "> '" & $info_string
    Local Const $Fail = "! '" & $info_string

    Select
        Case IsString( $var1 ) And IsString( $var2 )
            Switch $var1 == $var2
                Case 1
                    $This.Write( $Pass )
                Case 0
                    $This.Write( $Fail )
            EndSwitch
        Case IsNumber( $var1 ) And IsNumber( $var2 )
            Switch $var1 = $var2
                Case 1
                    $This.Write( $Pass )
                Case 0
                    $This.Write( $Fail )
            EndSwitch
    EndSelect
EndFunc   ;==>TDD_Equal

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotEqual( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Not Equal" )
    Local Const $info_string = $testname & "'  |  " & "Expected: '" & $var1 & "'  |  Is: '" & $var2 & "'"
    Local Const $Pass = "> '" & $info_string
    Local Const $Fail = "! '" & $info_string

    Select
        Case IsString( $var1 ) And IsString( $var2 )
            Switch Not ( $var1 == $var2 )
                Case 1
                    $This.Write( $Pass )
                Case 0
                    $This.Write( $Fail )
            EndSwitch
        Case IsNumber( $var1 ) And IsNumber( $var2 )
            Switch Not ( $var1 = $var2 )
                Case 1
                    $This.Write( $Pass )
                Case 0
                    $This.Write( $Fail )
            EndSwitch
    EndSelect
EndFunc   ;==>TDD_NotEqual

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_FileExists( ByRef $This , Const $file , Const $testname = "File Exists" )
    Local Const $Pass = "> '" & $testname & "' File: " & $file & " exists."
    Local Const $Fail = "! '" & $testname & "' File: " & $file & " does not exist."

    Switch FileExists( $file )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_FileExists

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotFileExists( ByRef $This , Const $file , Const $testname = "NotFileExists" )
    Local Const $Pass = "> '" & $testname & "' File: " & $file & " does not exist."
    Local Const $Fail = "! '" & $testname & "' File: " & $file & " exists."

    Switch Not FileExists( $file )
        Case True ; pass
            $This.Write( $Pass )
        Case False ; fail
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_FileDoesNotExist

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_GreaterThan( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Greater Than" )
    Local Const $Pass = "> '" & $testname & "' " & $var1 & " is greater than " & $var2
    Local Const $Fail = "! '" & $testname & "' " & $var1 & " is not greater than " & $var2

    Select $var1 > $var2
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSelect
EndFunc   ;==>TDD_GreaterThan

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotGreaterThan( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Not Greater Than" )
    Local Const $Pass = "> '" & $testname & "' " & $var1 & " is not greater than " & $var2
    Local Const $Fail = "! '" & $testname & "' " & $var1 & " is greater than " & $var2

    Select Not $var1 < $var2
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSelect
EndFunc   ;==>TDD_NotGreaterThan

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_GreaterThanOrEqualTo( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Greater Than Or Equal To" )
    Local Const $Pass = "> '" & $testname & "' " & $var1 & " is greater than or equal to " & $var2
    Local Const $Fail = "! '" & $testname & "' " & $var1 & " is not greater than or equal to " & $var2

    Select $var1 >= $var2
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSelect
EndFunc   ;==>TDD_GreaterThanOrEqualTo

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotGreaterThanOrEqualTo( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Not Greater Than Or Equal To" )
    Local Const $Pass = "> '" & $testname & "' " & $var1 & " is not greater than or equal to " & $var2
    Local Const $Fail = "! '" & $testname & "' " & $var1 & " is greater than or equal to " & $var2

    Select Not $var1 <= $var2
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSelect
EndFunc   ;==>TDD_NotGreaterThanOrEqualTo

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_LessThan( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Less Than" )
    Local Const $Pass = "> '" & $testname & "' " & $var1 & " is less than " & $var2
    Local Const $Fail = "! '" & $testname & "' " & $var1 & " is not less than " & $var2

    Switch $var1 < $var2
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_LessThan

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotLessThan( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Not Less Than" )
    Local Const $Pass = "> '" & $testname & "' " & $var1 & " is not less than " & $var2
    Local Const $Fail = "! '" & $testname & "' " & $var1 & " is less than " & $var2

    Switch Not ( $var1 < $var2 )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_NotLessThan

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_LessThanOrEqualTo( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Less Than Or Equal To" )
    Local Const $Pass = "> '" & $testname & "' " & $var1 & " is less than or equal to " & $var2
    Local Const $Fail = "! '" & $testname & "' " & $var1 & " is not less than or equal to " & $var2

    Switch $var1 <= $var2
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_LessThanOrEqualTo

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotLessThanOrEqualTo( ByRef $This , Const $var1 , Const $var2 , Const $testname = "Not Less Than Or Equal To" )
    Local Const $Pass = "> '" & $testname & "' " & $var1 & " is not less than or equal to " & $var2
    Local Const $Fail = "! '" & $testname & "' " & $var1 & " is less than or equal to " & $var2

    Switch Not ( $var1 <= $var2 )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_NotLessThanOrEqualTo

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_Object( ByRef $This , Const $object , Const $testname = "Object" )
    Local Const $Pass = "> '" & $testname & "' " & ObjName( $object ) & " | is an object."
    Local Const $Fail = "! '" & $testname & "' " & $object & " | is not an object."

    Switch IsObj( $object )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_Object
; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotObject( ByRef $This , Const $object , Const $testname = "Not Object" )
    Local Const $Pass = "> '" & $testname & "' " & $object & " | is not an object."
    Local Const $Fail = "! '" & $testname & "' " & ObjName( $object ) & " | is an object."

    Switch Not IsObj( $object )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_NotObject

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_ProcessExists( ByRef $This , Const $Process , Const $testname = "Process Exists" )
    Local Const $Pass = "> '" & $testname & "' Process: " & $Process & " exists."
    Local Const $Fail = "! '" & $testname & "' Process: " & $Process & " does not exist."

    Switch ProcessExists( $Process )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_ProcessExists

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotProcessExists( ByRef $This , Const $Process , Const $testname = "Not Process Exists" )
    Local Const $Pass = "> '" & $testname & "' Process: " & $Process & " does not exist."
    Local Const $Fail = "! '" & $testname & "' Process: " & $Process & " exists."

    Switch Not ProcessExists( $Process )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_ProcessDoesNotExist

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_True( ByRef $This , Const $var , Const $testname = "True" )
    Local Const $Pass = "> '" & $testname & "' " & $var & " is TRUE."
    Local Const $Fail = "! '" & $testname & "' " & $var & " is FALSE."

    If IsInt( $var ) Then
        Switch $var
            Case True
                $This.Write( $Pass )
            Case False
                $This.Write( $Fail )
        EndSwitch
    Else
        Write( "TDD ERROR : " & $var & " is not an integer." )
    EndIf
EndFunc   ;==>TDD_True

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotTrue( ByRef $This , Const $var , Const $testname = "Not True" )
    Local Const $Pass = "> '" & $testname & "' " & $var & " is NOT TRUE."
    Local Const $Fail = "! '" & $testname & "' " & $var & " is FALSE."

    If IsInt( $var ) Then
        Switch Not $var
            Case True
                $This.Write( $Pass )
            Case False
                $This.Write( $Fail )
        EndSwitch
    Else
        Write( "TDD ERROR : " & $var & " is not an integer." )
    EndIf
EndFunc

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_False( ByRef $This , Const $var , Const $testname = "False" )
    Local Const $Pass = "> '" & $testname & "' " & $var & " is FALSE."
    Local Const $Fail = "! '" & $testname & "' " & $var & " is TRUE."

    If IsInt( $var ) Then
        Switch $var
            Case False
                $This.Write( $Pass )
            Case True
                $This.Write( $Fail )
        EndSwitch
    Else
        Write( "TDD ERROR : " & $var & " is not an integer." )
    EndIf
EndFunc   ;==>TDD_False

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotFalse( ByRef $This , Const $var , Const $testname = "Not False" )
    Local Const $Pass = "> '" & $testname & "' " & $var & " is NOT FALSE."
    Local Const $Fail = "! '" & $testname & "' " & $var & " is TRUE."

    If IsInt( $var ) Then
        Switch Not $var
            Case False
                $This.Write( $Pass )
            Case True
                $This.Write( $Fail )
        EndSwitch
    Else
        Write( "TDD ERROR : " & $var & " is not an integer." )
    EndIf
EndFunc

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_WindowExists( ByRef $This , Const $window , Const $testname = "Window Exists" )
    Local Const $Pass = "> '" & $testname & "' " & $window & " does exist."
    Local Const $Fail = "! '" & $testname & "' " & $window & " does not exist."

    Switch WinExists( $window )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_WindowExists

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func TDD_NotWindowExists( ByRef $This , Const $window , Const $testname = "Not Window Exists" )
    Local Const $Pass = "> '" & $testname & "' " & $window & " does not exist."
    Local Const $Fail = "! '" & $testname & "' " & $window & " does exist."

    Switch Not WinExists( $window )
        Case True
            $This.Write( $Pass )
        Case False
            $This.Write( $Fail )
    EndSwitch
EndFunc   ;==>TDD_WindowDoesNotExist

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i

Func _TDD_Write( ByRef $This , Const $message )
    ConsoleWrite( $message & @LF )
EndFunc

; !i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i!i
Edited by LaCastiglione

Helpful Posts and Websites: AutoIt3 Variables and Function Parameters MHz | AutoIt Wiki | Using the GUIToolTip UDF BrewManNH | Can't find what you're looking for on the Forum?

jaberwacky Universalist

jaberwacky

Posted
  • Author
Posted (edited)

I'm a big-picture kinda guy. When I set out to make this framework I had never actually used TDD! I made this so that I could get my feet wet. I read about TDD and spent time thinking it through before I even touched an IDE. So, I'm afraid I do not know the answer. Sorry.

My vision for this script is to learn as much about TDD as possible. Also, I want to see this script evolve into something truly awesome. If over time this script becomes something totally different than what it is today then that would be awesome. I'll be along for the ride.

edit: I did watch a video tut about TDD in which JUnit was used. The presenter extended the JUnit class with his own class containing the unit tests. At least according to my best understanding.

Edited by jaberwocky6669

Helpful Posts and Websites: AutoIt3 Variables and Function Parameters MHz | AutoIt Wiki | Using the GUIToolTip UDF BrewManNH | Can't find what you're looking for on the Forum?

  • 2 months later...
jackHeeley Seeker

jackHeeley

Posted
Posted

Do you know of any test driven development libraries/frameworks for other interpreted languages? I tried to compare this to how we do C# unit testing with NUnit but the whole setup is just too different.

To address your question first. Below is a 'typical' VB .NET unit test for comparison (NUnit/TestDriven). I guess this is the closest language analog we will get to AutoIt3. Take a look there. My impression at the time was that C# supprt was same thing different syntax, but you say not so? Anyway, is this (style) the direction you guys are thinking of going in?

Now to my personal viewpoint...

Actually for my AutoIt3 development, I (and I guess others too?) have written a simple unit test framework, very closely modelled on a post seen elsewhere in this(?) forum, and following this lead, I have built a test runner (for the eclipse IDE) to launch my unit tests (individually or collectively with one click).

I've supplimented this with log4j style debug/trace feature (built over the Debug.au3 UTF). This allows me to effectively instrument my project code (defensive style) so that it is self-checking. Project code reports errors, and/or warnings and/or info, configurable on a script-by-script basis - bitmask configured in environment variables. For my (most) purposes I do not need to dummy out or remove this instrumentation before release (performance hit is localized and acceptable). This has the added benefit that I could advise a client (if I had one!) to switch on the "field support feature" via environment settings.

The combination of these two features ALREADY allows me to take a "test driven" approach to autoIt project development using a comfortable richly featured open source IDE (Eclipse). There are some niggling limitations but nothing I can't live with.

Personally, I've found this to be enough. I dont really miss the richer set of Assert options or a syntactically familiar (TestDriven/NUnit) way to embed them in code.

See the fragments after the VB sample. I'd be interested in your opinions.

(its amusing to see how the last error differs diametrically from the actual cause of the error, but that's windows!)

The BIG issue for me is test coverage. There I think we need help of the AutoIt development team to instrument the interpreter for us (if it isnt there already as a hidden feature) and give us a way to access the script name/line numbers that have been executed since last checkpoint.

SAMPLE NUNIT TEST VISUAL BASIC .NET

===================================

'

' Copyright © Jack Heeley, Chess BV 2007

'

' nunit test for ItaChip.vb

'

' History:

'

' 20070702 jahe initial write

' 20070717 jahe CoverageExclude attribute

'

#If DEBUG Then

Option Explicit On

Option Strict On

Imports System

Imports System.IO

Imports NUnit.Framework

Namespace Chip

<CoverageExclude()> _

Public Interface ITestableItaChip : Inherits IItaChip

WriteOnly Property WRITE_MCDID_STRING() As String

Sub simulateException()

Sub simulateExceptionWithInnerException()

End Interface

<CoverageExclude()> _

Public Interface ITestableMultosCarrierDevice : Inherits IMultosCarrierDevice

WriteOnly Property WRITE_ID_STRING() As String

End Interface

<TestFixture()> _

Public Class UnitTestItaChip : Inherits AbstractUnitTest

'ItaChip object for testing

Private m_itaChip As ITestableItaChip

'test mcdId list file, containing expected test mcdId value

Private Shared ENABLEMENT_REQUEST_FILE As String

' test mcdId string matching a value in test file "enablement_request_0703.dat"

Private Shared TEST_MCDID_STRING As String = "052B13000000C4C7"

' test mcdId bytes matching a value in test file "enablement_reqeuest_0703.dat"

Private Shared TEST_MCD_ID() As Byte = {&H5, &H2B, &H13, &H0, &H0, &H0, &HC4, &HC7}

' format and position from chipId in test file "enablement_requst_0703.dat"

Private TEST_FORMAT_AND_POSITION As String = "F1001_1_1"

<SetUp()> _

Protected Sub SetUp()

'framework invokes this method before each test (populate as required)

ENABLEMENT_REQUEST_FILE = G_PROGRAM_PROPERTIES.PATH_ICECONFIGFILE

m_itaChip = New SimulatedItaChip()

m_itaChip.WRITE_MCDID_STRING() = TEST_MCDID_STRING

End Sub

<Test()> _

Public Sub testHappyFlow()

m_itaChip.runDownloadApplications()

m_itaChip.runFunctionalTest()

m_itaChip.runJtagTest()

End Sub

<Test()> _

Public Sub testGetFormatAndPosition()

Assert.AreEqual(TEST_FORMAT_AND_POSITION, m_itaChip.getFormatAndPosition(ENABLEMENT_REQUEST_FILE))

End Sub

<Test()> _

Public Sub testItaChipExceptions()

Try

m_itaChip.simulateException()

Catch ex As ItaChipException

StringAssert.IsMatch("Test ItaChipException", ex.Message())

End Try

Try

m_itaChip.simulateExceptionWithInnerException()

Catch ex As Exception

Assert.IsInstanceOfType(GetType(ItaChipException), ex)

StringAssert.IsMatch("Test ItaChipException", ex.Message())

Assert.IsInstanceOfType(GetType(ApplicationException), ex.InnerException)

StringAssert.IsMatch("Test InnerException", ex.InnerException.Message())

End Try

End Sub

End Class

End Namespace

#End If

SAMPLE UNIT TEST AUTOIT3

========================

;

; File : testXmldocument.au3

; Type : test file

; Author : Jack Heeley © Chess BV

; Created : 28 Aug 2010

;

; Purpose : testing xml parsing

;

; History :

;

; 1.0 28-Aug-2010 Initial creation;

; this is the unit test for Xmldocument.au3

#include <unitTestFramework.au3>

#include <Xmldocument.au3>

#include <sampleXmlDocumentHandler.au3>

callOnce("runXmlDocumentTestSuite")

Func runXmlDocumentTestSuite()

UTStartUnitTestSuite("testXmldocument.au3")

testXmlLoadHappyFlow()

testXmlLoadUnHappyFlow()

testXmlErrorHandler()

testXmlBooks()

UTEndUnitTestSuite()

EndFunc

Func testXmlLoadHappyFlow()

UTStartUnitTest("testXmlLoadHappyFlow")

Local $returnCode = $UT_UNINITIALIZED_TEST_VALUE

Local $theXmlDom = $UT_UNINITIALIZED_TEST_VALUE

Const $EXPECTED_TEST_FILE = "src\test\resources\au3utils\stocks.xml" ; relative to Working Directory = $(projectloc)

$theXmlDom = XmlDocument_Load($EXPECTED_TEST_FILE)

UTAssert(Not @error)

UTAssert(Not($theXmlDom=0))

XmlDocument_ProcessDoc($theXmlDOM, "mySingleNodeCallback")

UTAssert(Not @error)

UTAssert(Not IsObjectError())

UTEndUnitTest()

DEBUG_INFO("testXmlLoadHappyFlow", "testXmldocument.au3")

EndFunc

... ETC.

SAMPLE LOGFILE CONTENT

======================

Oct 01 16:36:32 unitTestFramework.au3(85) : INFO : UTStartUnitTest(testXmlLoadUnhappyFlow)

Oct 01 16:36:32 unitTestFramework.au3(87) : INFO : UTStartUnitTest() OK

Oct 01 16:36:32 testXmldocument.au3(79) : WARN : Check that UnitTest triggers an ERROR log entry...

Oct 01 16:36:32 Xmldocument.au3(99) : INFO : XmlDocument_Load(src\test\resources\au3utils\testXmlDocumentFileNotExist.xml)

Oct 01 16:36:32 Xmldocument.au3(133) : ERROR : XmlDocument_Load, DOMdocument.load raised parse error=System error: -2146697210.

Oct 01 16:36:32 Xmldocument.au3(136) : INFO : Filename=src\test\resources\au3utils\testXmlDocumentFileNotExist.xml

Oct 01 16:36:32 Xmldocument.au3(138) : INFO : Last Error=Cannot create a file when that file already exists.

Oct 01 16:36:32 testXmldocument.au3(81) : WARN : ...Check that UnitTest triggered an ERROR log entry above

Oct 01 16:36:32 unitTestFramework.au3(108) : INFO : UTAssert(-2146697210, Assert Failure, 82)

Oct 01 16:36:32 unitTestFramework.au3(113) : INFO : UTAssert() OK

Oct 01 16:36:32 unitTestFramework.au3(108) : INFO : UTAssert(True, Assert Failure, 83)

Oct 01 16:36:32 unitTestFramework.au3(113) : INFO : UTAssert() OK

Oct 01 16:36:32 unitTestFramework.au3(94) : INFO : UTEndUnitTest()

SAMPLE CONSOLE OUTPUT FROM UNIT TEST FRAMEWORK...

=================================================

Running test suite: D:\eclipse_helios_workspace\TestCockpit\src\test\fullUnitTestSuite.au3

D:\eclipse_helios_workspace\TestCockpit\src\test\fullUnitTestSuite.au3(40) := Assert Failure

D:\eclipse_helios_workspace\TestCockpit\src\test\fullUnitTestSuite.au3(41) := Assert Failure

D:\eclipse_helios_workspace\TestCockpit\src\test\fullUnitTestSuite.au3(45) := Assert Failure

"fullUnitTestSuite.au3 TestSuites run(8), UnitTests run(34), passed(33), failed(1)

jaberwacky Universalist

jaberwacky

Posted
  • Author
Posted

I did a google search of these forums and found no other TDD "framework" other than this one.

But really, my code here probably doesn't really yet deserve the title framework, lawlz. =)

But hey, if you want to either develop your own framework or expand on this one then I'll help out no problems! That would be great.

Helpful Posts and Websites: AutoIt3 Variables and Function Parameters MHz | AutoIt Wiki | Using the GUIToolTip UDF BrewManNH | Can't find what you're looking for on the Forum?

jackHeeley Seeker

jackHeeley

Posted
Posted

I did a google search of these forums and found no other TDD "framework" other than this one.

But really, my code here probably doesn't really yet deserve the title framework, lawlz. =)

But hey, if you want to either develop your own framework or expand on this one then I'll help out no problems! That would be great.

I followed the lead given in the Wiki...

http://www.autoitscript.com/wiki/UnitTesting

and made my own <unitTestFramework.au3> to suit my personal situation.

Likewise I read up on the Debug.au3 UTF and made my own <DebugEx.au3> to wrap that in a way that suited

my peronal development/debug style.

If I read thw Wiki correctly we might expect all my stuff to become a quirky dialect (as it seems that

something is slated for official release ...

"The UTRunner is now ready to be integrated with SciTe4AutoIt."

I have to say I strongly prefer Eclipse IDE (mainly because it integrates version management system)

My development process is: (edit/run unitTest/check logfile), with a batch build process to compile

and deploy at the end. I only use SciTE to launch the compiler for a deeper parse check once in a

while and always before building/deploying. These are very personal preferences, I don't expect

everybody to line up with my way of doing things.

The stage I'm at is that what I have now, does more or less all that I want it to do (now). I could

post it and feed it back into the community, but I think it would be better directed to the author of

the Wiki, for him to consider and coordinate rather than propagating a lot of competing variants of

the same sort of thing.

Who knows maybe I have a couple of nice extras in my version, and he has a dozen more from other

contributors. But so far I haven't discovered who the author is. Any ideas?

Now the extended debug stuff I developed is another matter. That seems to be very close to being

exactly what the doctor ordered. If somebody could write a configurer program for it (something

like java's lumbermill) I think it would be a real benefit to developers.

jackHeeley Seeker

jackHeeley

Posted
Posted

I've added a ticket to request some test coverage instrumentation in the interpreter.

We should wait and see if anything comes of that.

I also found what looks like an ice cool Au3 debugger (from another contributor) He

makes it available in source.

I haven't evaluated this yet, but at least in principle a test coverage plug-in could

be hosted there. I place that as my second preference.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...