Executing raw machine code

[Lazycat]

I'm not recall if this concept was on forums somwhere. Suppose similar technique is used in DllCallBack.au3 by piccaso, but I'm just used it and not dig it deep

Once I'm found on the AHK forums interesting concept for executing raw machine code. Unfortunately Autoit's DllCall can't call function by it's address, so I though "doh" and going out.

But today I found another way: API function CallWindowProc, being used in not documented way can do the same job. This tested only under WinXP SP2, this also should not have a problem under Win9x, I can't say about other OS.

Of course this is not too convenient way and I'm not suggest to use it anywhere, but this trick can be quite useful for time-intensive operations, like checksums calculations, sorting (together with DllCallbackRegister) etc.

I'm attched two examples. First calculate CRC32 with two functions: first is Au3 native, second reuse machine code. On my computer, on 64kb string second function is about 2500 times faster.

Second example calculate CRC32 for file (this give the same value as in SFV format). On my system it take about 2.5 seconds for 150mb file, what I think good enough for interpret language.

fast_crc32.zip

[Alek]

Holy crap that was fast

now i just need to learn machine code xD

[randallc]

Hi,

I'm interested to get a faster sort for autoIt; but I have no idea where to fnd machine code for it!

I presumed it was too hard to do, as noone has done it even with a "C" dll as far as I know, so I have stuck to calling vbscript or jscript via com.

Can you show ;

a., how you would call it and the call-back?

B. Where to get the machine code! -or would I have to write it myself!(not a possibility....)

?

Best, randall

[JRSmile]

And now we only have to find a way to heal cancer in AutoIT.

VERY NICE WORK !!!

[ptrex]

@Lazycat

Realy nice !!

And just in time.

I will PM you about this, because the native AU3 function to read big files makes AU3 crash.

so this way might give a solution to it.

Because I just tried it on a 100 Mb file and it works at the speed of light

thanks

ptrex

[CoderDunn]

Man that is fast! But the only problem is how do we get the machine code to use in scripts? Would it be possible to compile for example C++ code, open it in a hex editor (or disassembler?), and copy the machine code into an autoit string, and then run it using your function? Of course, I really don't know what I'm talking about or if it's even possible xD

Hallman

Edited January 3, 2008 by Hallman

[Zedna]

Awesome technique Lazycat!!!

But I would like to know general way to get machine code too.

[spyrorocks]

Mmmmm au3 2 asm yummy.

[WolfWorld]

This is very cool nice job

[Lazycat]

Man that is fast! But the only problem is how do we get the machine code to use in scripts? Would it be possible to compile for example C++ code, open it in a hex editor (or disassembler?), and copy the machine code into an autoit string, and then run it using your function? Of course, I really don't know what I'm talking about or if it's even possible xD

You right, this is the way.

Also to Alek, randallc, Zedna

You need to write function in Dll with standard export (you may use any language for that), then disassemble it and use you function's code.

Here the link to topic I was inspired and where take a CRC32 code: http://www.autohotkey.com/forum/topic21172.html

I'm tried to compile something myself, but have a troubles with MinGW, since it place too many redundant code and so far I can't find a way how to get rid of that. Maybe VC compiler can do job better, but I haven't it.

Personally, I'm interested to have in this way MD5 function and Base64 encoder/decoder. If you have a time and interest - you welcome.

[mikeytown2]

@Lazycat, If you could post a how to, beginning to end, that would be cool!

Example Template

• Write C Code
• Compile C Code Using ___
• Use Hex Editor ___ to copy binary data
• Insert Machine code into AutoIt
• ...
• BSOD

This could make a very fast base64 encoder/decoder

[ValeryVal]

There are also discussion about this subject at December 13, 2002

here

You can see sense of these hexs:

55 8B EC and so on

There is a more simple way to convert asm code into binary line, I think.

[CoderDunn]

You right, this is the way.

Wow thats surprizing that I guessed something right xD Considering the fact that I know close to nothing on other programming languages

Hallman

[Laszlo]

In another site I described the simplest process I know to get machine code:

- Install the free VS'05 express compiler from MS

- Create an empty console project

- Write C functions, with their names exported (Project/Properties: Linker command line options / Additional options: "/EXPORT:MyFuncName")

- Set the compiler option to list assembly and machine code (C/C++ / Output Files: Assembler Output: Assembly With Machine Code (/FAc))

- Compile the C project

The compiler generates an "fname.cod" file, where you find the machine code in hex and the corresponding assembly instructions. You can use an editor to strip off unwanted information, but I have been using a 10-line script for this and for nicely formatting the hex stream of the machine code.

If you are interested, there are many low level functions already compiled to machine code, like CRC32. The difficulty is to prevent the compiler to link its library functions, which makes the machine code not standing alone. I have been using my own versions (mostly written in assembly) with the prefix "__forceinline", but it is hard to use doubles. int's and __int64's can be handled w/o series complications.

[arcker]

thx Lazlo and welcome.

I've finally understood how to read those hex. I was lost in disasm and others tools.

We're using an high level language so any help on how machine code works is really helpful.

i've passed my last evening to read you whole post on machine code execution with AHK

i've learned two things that can interests other people that read this thread :

- machine code is really powerful for mathematical operations (so really good for encrypting/decrypting)

- C stays more powerful in most of the cases. But it's for AHK.

If i'm wrong, please correct me.

[Laszlo]

(I was told that those three letters representing another scripting language are forbidden in this Forum. It looks like not to be true... Well, some people just seem to want artificial hostility.)

The machine code functions I developed behave exactly like the ones in a Windows dll, like user32.dll, ntdll.dll, msvcrt80.dll, etc. They take parameters from the stack and return a value or a pointer in the eax CPU register. If you succeed calling system functions from these dll's, you can run machine code functions via a dll call to CallWindowProc, which in turn will run the machine code provided in a binary buffer.

There are two common calling conventions, standard calls (when the called function cleans the stack) represented in RTL functions in ntdll, or CDECL, when the caller does this, represented in the C runtime library functions in msvcrt80.dll. You can specify the calling standard in the project settings in the VS compilers. The only thing to remember is that variable number of arguments are not supported by standard call. (There are other, less frequently used calling conventions, like fast-calls, but they just complicate our lives.)

There are two tasks:

- store the machine code in a buffer

- execute it via a dll call to CallWindowProc.

The former can be done in many different ways. For simplicity I converted hex bytes to binary and wrote them to memory. You can do it with a series of dll calls to RtlMoveMemory, faking Unicode strings, writing integer arrays, etc. Speed is not an issue, because this conversion is done only once at initialization.

[Lazycat]

Thanks for the tips Laszlo, and for idea in whole. Hope I'll have time and encourage myself to install VC Express and try.

[passman]

Thanks for the tips Laszlo, and for idea in whole. Hope I'll have time and encourage myself to install VC Express and try.

This may not be relevent.. but you can get the ASM code from the "DEBUG" command. I use to use it back int 80's to add asm code to BASIC.

ASM is not hard to learn for simple functions. I will try to write an AutoIt script to move Debug text to inline code.

[SmOke_N]

Interesting topic.... I was directed by this topic http://www.autoitscript.com/forum/index.ph...mp;#entry511901 and Just yesterday I was using a hash table for this very thing in another language... but put asm code in with it. CRC32 output in 2 milliseconds or a 4mb file was pretty refreshing.

Look forward to looking at the other methods ya'lll used and comparing.

[Skrip]

Is it possible to 'Poke' code, with this or something related?