Inline C Source In AutoIt

[Ward]

TCC is a small and fast C compiler. And the LibTCC use TCC as a backend for dynamic code generation.

With LibTCC, we can add some C source to speed up the time-consuming subroutines. For example:

 

 

Func Fib($n)
    Local $C
    $C  = 'int fib(int n)                           ' & @LF
    $C &= '{                                        ' & @LF
    $C &= '    if (n <= 2)                          ' & @LF
    $C &= '        return 1;                        ' & @LF
    $C &= '    else                                 ' & @LF
    $C &= '        return fib(n-1) + fib(n-2);      ' & @LF
    $C &= '}                                        '

    Local $TCC = TCC_New()
    TCC_Compile_String($TCC, $C)

    Local $Size = TCC_Relocate($TCC, 0)
    Local $CodeBuffer = _MemVirtualAlloc(0, $Size, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE)
    TCC_Relocate($TCC, $CodeBuffer)

    Local $Ptr = TCC_Get_Symbol($TCC, "fib")
    Local $Ret = MemoryFuncCall("int:cdecl", $Ptr, "int", $n)

    _MemVirtualFree($CodeBuffer, 0, $MEM_RELEASE)
    TCC_Delete($TCC)

    Return $Ret[0]
EndFunc

 

There is also an example using the C source (MD5.C) to count the MD5 in AutoIt (Include in the ZIP).

Just like run the C source as script.

 

For detail usage of the LibTCC functions. Please see libtcc.h in the TCC source.

(The examples already demonstrate most functions)

 

AutoIt x64 it not supported now.

 

LibTCC.zip

 

2011/08/03 Update Note:

 

• Fix the bug in TCC_Run, thanks rchockxm.
• Add example for TCC_Run

LibTCC.zip

 

 

2011/08/29 Update Note:

 

• Add some function to generated binary code that can be used in other script without LibTCC.

   

  If the C source using Windows API, then the code can't run on other system.

  A solution is getting the API address in AutoIt and pass it to C code by function pointer.

   

  NOTICE: These functions and examples need my

LibTCC_Binary.zip

Edited November 23, 2015 by Ward

[UEZ]

Very impressive work again - after ASM now inline C!

Br,

UEZ

Edited August 2, 2011 by UEZ

[rchockxm]

Nice job!

I found some bug in TCC_Run function...

Func TCC_Run($Argc = 0, $Argv = 0)

Local $Ret = MemoryDllCall($LIBTCC_DLL, "int:cdecl", "tcc_run", "ptr", $TCC, "int", $Argc, "ptr", $Argv)

Return $Ret[0]

EndFunc

thanks nice udf

[twitchyliquid64]

omg LIKE LIKE LIKE!

do you think it is possible to produce pure binary code org'ed/startpos to a specific address?

[taietel]

Thanks Ward!

taietel

[Ward]

omg LIKE LIKE LIKE!

do you think it is possible to produce pure binary code org'ed/startpos to a specific address?

TCC_Relocate do copy and relocate the binary code to specific address.

Call TCC_Relocate twice with different address, compare the result, and then you can get the binary code with some relocation data.

To run the binary code, you must alloc the memory space and then do relocate again by yourself.

But this binary code can only run on your computer if you use some Windows API or DLL.

Because for different computer, the API address may be different.

[twitchyliquid64]

i think i have not explained very well.

i do operating systems development, and that means you do not have anything like a 'relocation table' to make things easy. all addresses are actual addresses, not offsets from a base address.

consider a floppy booting model. the first 512 bytes of the first floppy sector and track are loaded into memory address 0x000000. it them executes from that point.

all code that the processor executes must be pure addresses as valid operations.

so i need a compiler that can generate position dependent code. my current solution is a gcc cross compiler but i hate the relative complexity of using it.

[Ward]

If the code is executed from the same address every time, you can do relocate by this address, then you get the binary code for this address. This binary code can be executed only from this address.

If the code may be executed from different address, you must relocate every time before executing.

Some well-designed code can be executed from different address without relocation, is called shellcode. You can search "shellcode" on the internet for detail. I use the similar method for my Machine Code UDFs. But this is a big topic and not related to this post.

[Andreik]

Man you're crazy, that's very very impressive and I'm sure it will be useful also. Thanks for sharing.

[smartee]

Excellent stuff as always Ward , Tiny C has always been a favourite of mine.

[KaFu]

Another unique addition to the community, much appreciated as always ! The MD5 example raises the question for me, whether this one or your is faster.. I assume the OPCode is faster, but I'm not sure.

[Ward]

Another unique addition to the community, much appreciated as always ! The MD5 example raises the question for me, whether this one or your is faster.. I assume the OPCode is faster, but I'm not sure.

I think machine code version is faster. (I not test yet.)

TCC compiles fast, but not generates fast machine code.

So the MD5 example for TCC is just for demo, not for use.

[Stilez]

AutoIt x64 it not supported now.

Out of interest what's the issue with x64?

Can it generate x32 code which can still run on an x64 machine?

Edited August 6, 2011 by Stilez

[trancexx]

Out of interest what's the issue with x64?

Can it generate x32 code which can still run on an x64 machine?

x64 executable can't use x86 dlls nor x86 opcode. That's all there is to it.

Machine type is irrelevant.

[Stilez]

I'm not a machine code programmer, nor a Windows coder,. I accept it but I don't know enough to understand. Presumably TCC compiler and compiled code is x86, but AutoIt x64 can still compile code x86. At what point would x64 code try to call or link x86 code and be unable? If one compiled the program with AutoIt on an x86 machine would the executable be able to run on x64? What about if one ran the x86 version of AutoIt on an x64 machine, or installed/ran AutoIt in a 32 bit compatibility mode?

Some clarification would be interesting Thanks!

Edited August 6, 2011 by Stilez

[Ward]

AutoIt x64 is not a compiler, it can't compile x64 nor x86 code. AutoIt (x86/x64) only run the scripts.

64 bit windows can run both x64 and x86 exe in different mode.

When run x64 exe (like AutoIt x64), it need x64 code, x64 dll, etc.

If you force x64 exe to run x86 code, the code look like nonsense, so the result is crash.

TCC can generate x64 code for now, but it seems not support "microsoft x64 calling convention".

If we want LibTCC for AutoIt x64, we need TCC x64 version with correct calling convention.

This needs a lot of modification on source code. And is not what I want to do.

[arcker]

Hi ward,

As usual, your scripts are so awesome to learn more from code machine.

I've tried to make another example, it works ... half. For some reason the code is executed twice.

expandcollapse popup

; =============================================================================
;  LibTCC UDF Example (2011.8.1)
;  Purpose: Speed Comparison, AutoIt vs LibTCC
;  Author: Ward, Arcker
; =============================================================================

#include <Memory.au3>
#include "MemoryDll.au3"
#include "LibTCC.au3"
#AutoIt3Wrapper_UseX64=n
; Method 1, Count by AutoIt
;$Timer = TimerInit()
;ConsoleWrite("Result1: " & Fib(30) & ", spent " & Round(TimerDiff($Timer), 2) & " ms" & @CRLF)

; Method 2, Count by C, The spent time include UDF loading and C source compiling
$Timer = TimerInit()
ConsoleWrite("Result2: " & AfficherListeFichiers("c:") & ", spent " & Round(TimerDiff($Timer), 2) & " ms" & @CRLF)

Exit

Func Fib($n)
    If $n <= 2 Then
        Return 1
    Else
        Return Fib($n - 1) + Fib($n - 2)
    EndIf
EndFunc   ;==>Fib

Func AfficherListeFichiers($n)
    Local $C
    $C = '#include <windows.h>' & @LF
    $C &= '__stdcall int test(char Dossier[2])' & @LF
    $C &= '{ ' & @LF
    $C &= 'TCHAR szError[100]; ' & @LF
    $C &= 'HANDLE hFind; ' & @LF
    $C &= ' DWORD dwError=0;' & @LF
    $C &= 'WIN32_FIND_DATA FindData; ' & @LF
    $C &= 'MessageBox(0,  Dossier, "AutoIt vs TCC", MB_ICONINFORMATION); ' & @LF
    $C &= '// Change de dossier ' & @LF
    $C &= 'SetCurrentDirectory (Dossier); ' & @LF
    $C &= '// Début de la recherche ' & @LF
    $C &= 'hFind=FindFirstFile ("*.*"  , &FindData); ' & @LF
    $C &= 'if (hFind!=INVALID_HANDLE_VALUE) ' & @LF
    $C &= '{' & @LF

    $C &= 'MessageBox (NULL, FindData.cFileName, "Fichier first", MB_ICONINFORMATION);  ' & @LF

    $C &= '// Fichiers suivants' & @LF
    $C &= 'while (FindNextFile (hFind, &FindData) != 0 ) ' & @LF
    $C &= '{ ' & @LF

    $C &= ' dwError = GetLastError();' & @LF
    $C &= 'if (dwError == ERROR_NO_MORE_FILES) ' & @LF
    $C &= '{' & @LF
    $C &= 'wsprintf (szError, TEXT("Error ! %ld\r\n"),dwError);' & @LF
    $C &= 'MessageBox (NULL, szError, "Error", MB_ICONINFORMATION);' & @LF
    ;$C &=  'break;' &@lf
    $C &= '}' & @LF

    $C &= 'if ((FindData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)) ' & @LF
    $C &= '{ ' & @LF
    $C &= 'MessageBox (NULL, FindData.cFileName, "Dossier", MB_ICONINFORMATION); ' & @LF
    $C &= '} ' & @LF
    $C &= '} ' & @LF
    $C &= '} ' & @LF
    $C &= '// Fin de la recherche ' & @LF
    $C &= 'FindClose (hFind); ' & @LF
    $C &= '//return 1; ' & @LF

    $C &= '}' & @LF
    ;$c&= FileRead("displayerror.c")
    ConsoleWrite($C & @CRLF)


    Local $TCC = TCC_New()
    TCC_Add_Include_Path($TCC, @ScriptDir & "\include")
    TCC_Add_Include_Path($TCC, @ScriptDir & "\include\winapi")
    TCC_Add_Library_Path($TCC, @ScriptDir & "\lib")
    TCC_Add_Library($TCC, "user32")
    ;TCC_Add_Library($TCC, "kernel32")

    TCC_Set_Error_Func($TCC, 0, DllCallbackGetPtr(DllCallbackRegister("TCC_Error", "none:cdecl", "ptr;str")))

    TCC_Compile_String($TCC, $C)

    Local $Size = TCC_Relocate($TCC, 0)
    Local $CodeBuffer = _MemVirtualAlloc(0, $Size, $MEM_COMMIT, $PAGE_EXECUTE_READWRITE)
    TCC_Relocate($TCC, $CodeBuffer)

    Local $Ptr = TCC_Get_Symbol($TCC, "_test@4")
    ConsoleWrite("CodeBuffer=>" & $Size & @CRLF)
    $sPath = "c:\"
    $tPath = DllStructCreate("char[" & StringLen($sPath) & "]")
    DllStructSetData($tPath, 1, $sPath)
    Local $Ret = DllCall("user32.dll", "uint", "CallWindowProc", "ptr", $Ptr, _
            "ptr", DllStructGetPtr($tPath) _
            )

    ;   Local $Ret = DllCall("user32.dll", "uint", "CallWindowProc", "ptr", $Ptr, _
    ;                                               "str", $sPath _
    ;                                               )

    ConsoleWrite("fin")

    _MemVirtualFree($CodeBuffer, 0, $MEM_RELEASE)
    TCC_Delete($TCC)
EndFunc   ;==>AfficherListeFichiers

Edited August 9, 2011 by arcker

[Ward]

Hi ward,

As usual, your scripts are so awesome to learn more from code machine.

I've tried to make another example, it works ... half. For some reason the code is executed twice.

CallWindowProc API can only accept 4 parameters.

Your script can use MemoryFuncCall to run. For Example:

MemoryFuncCall("uint", $Ptr, "ptr", DllStructGetPtr($tPath))

[arcker]

Ward,

thx for the explanation.

I've managed to make it work with tcc_run.

I understand why you use messagebox (srt,int,int,int) in one of the example.

For now it works good it I do the same in my function since I know that, or it works without adding "fake" parameters

by using MemoryFuncCall. That's just awesome.

thx again.

So 3 solutions to make it work :

- Use tcc_run with arguments (argc and co )

- Use Dllcall by using 4 parameters in all case with tcc_get_symbol(%function%@16)

- Use MemoryDllCall by using any parameter count, by using tcc_get_symbol("_%function%@(parametercount*4)")

Edited August 9, 2011 by arcker

[UEZ]

Is it possible to use also AutoIt function within C code?

E.g. when I want to speed up this GDI+ code

expandcollapse popup

#include <GDIPlus.au3>
#include <WindowsConstants.au3>

Opt("MustDeclareVars", 1)
Opt("GUIOnEventMode", 1)

Global Const $Width = 200
Global Const $Height = 100

Global $hGUI = GUICreate("B!zarre Arts", $Width, $Height)

GUISetState()

If @OSBuild < 7600 Then WinSetTrans($hGUI,"", 0xFF) ;workaround for XP machines when alpha blending is activated on _GDIPlus_GraphicsClear() function to avoid slow drawing

_GDIPlus_Startup()
Global $hBMP = _WinAPI_CreateBitmap($width, $height) ;create a dummy bitmap
Global $hImage = _GDIPlus_BitmapCreateFromHBITMAP($hBMP)
Global $hBitmap = _GDIPlus_BitmapCreateHBITMAPFromBitmap($hImage)

Global $hDC  = _WinAPI_GetDC($hGUI)
Global $hDC_backbuffer  = _WinAPI_CreateCompatibleDC($hDC)
Global $DC_obj = _WinAPI_SelectObject($hDC_backbuffer, $hBitmap)
Global $hGraphic = _GDIPlus_GraphicsCreateFromHDC($hDC_backbuffer)
_GDIPlus_GraphicsClear($hGraphic)
_GDIPlus_GraphicsSetSmoothingMode($hGraphic, 0)

Global $hBrush = _GDIPlus_BrushCreateSolid(0)
Global $t1, $t2, $t3, $s2, $fwX, $fwY, $a, $b, $v, $alpha, $red, $green, $blue
Global $t = 0, $s = 1, $f = 0x1E3, $q = 0xFF, $q2 = 0x100 / 4, $speed = 3


GUISetOnEvent(-3, "Close")

Do
    If $t < 1500 Then
        $t += 2
    Else
        $t = 0
    EndIf
    $t1 = $t / 75
    $t2 = $t1 - 5
    $t3 = $t - 10
    If Not Mod($t, 200) Then
        $s += 1
        $f = 1
    ElseIf  $f > 100 Then
        $f = 0x1E3
    Else
        $f += 2
    EndIf
    If $s > 3 Then
        $s = 1
    EndIf
    $s2 = $s
    For $a = 0 To $Height Step $speed
        If $f < 100 And $f < $a Then
            If $s = 1 Then $s2 = 3
            If $s = 2 Then $s2 = 1
            If $s = 3 Then $s2 = 2
        EndIf
        For $b = 0 To $Width Step $speed
            $fwX = $b / 10 - 20
            $fwY = $a / 10 - 10
            Switch $s2
                Case 1
                    $v = Sin($fwX + $fwY * $t1) - Cos($fwY + $fwX * $t1)
                Case 2
                    $v = Sin($fwX * $t2) + Sin($fwY * $t2) + Sin($t2 * $t2 + $fwX * $fwX + $fwY * $fwY)
                Case 3
                    $v = Sin($t3 * $fwX * $fwY) + Sin($fwY) * Cos($t3) * 0.0010
            EndSwitch
            $red = Max(0, Min($q, $q * $v))
            $green = Max(0, Min($q, $q * ($v - Cos($t1 * $s2))))
            $blue = Max(0, Min($q, $q *($v - Sin($t1 * $s2))))
            $alpha = $q2
            _GDIPlus_BrushSetSolidColor($hBrush, "0x" & Hex($alpha, 2) & Hex($red, 2)  & Hex($green, 2)  & Hex($blue, 2))
            _GDIPlus_GraphicsFillRect($hGraphic, $b, $a, $speed, $speed, $hBrush)
        Next
    Next
    _WinAPI_BitBlt($hDC, 0, 0, $width, $height, $hDC_backbuffer, 0, 0, $SRCCOPY)
Until Not Sleep(30)

Func Min($i, $j)
    If $i < $j Then Return $i
    Return $j
EndFunc

Func Max($i, $j)
    If $i > $j Then Return $i
    Return $j
EndFunc

Func Close()
    _GDIPlus_BrushDispose($hBrush)
    _WinAPI_SelectObject($hDC_backbuffer, $DC_obj)
    _GDIPlus_GraphicsDispose($hGraphic)
    _WinAPI_DeleteObject($hBMP)
    _GDIPlus_BitmapDispose($hImage)
    _WinAPI_DeleteObject($hBitmap)
    _WinAPI_ReleaseDC($hGUI, $hDC)
    _GDIPlus_Shutdown()
    GUIDelete($hGUI)
    Exit
EndFunc   ;==>Close

by transforming only the 2 nested For/Next loops to C which are the performance killer here.

Br,

UEZ

Edited August 9, 2011 by UEZ