Fast multi-client TCP server

[Kealper]

Yes, it probably could if it was implemented correctly, I have used this for quite a few projects myself.

EDIT: Spelling.

Edited July 26, 2012 by Kealper

[Attckdog]

Yes, it probably could if it was implemented correctly, I have used this for quotes few projects myself.

Cool thanks for the quick reply I'll test it out and report back if I come up with any problems.

[Shinies]

Great work, i was looking for exactly something like this.

Your comments made it a billion times easier to understand whats needs to be done in a script of this nature!

[Kealper]

Great work, i was looking for exactly something like this.

Your comments made it a billion times easier to understand whats needs to be done in a script of this nature!

Thanks, glad to hear it's as useful to other as it is to me!

PS: This bit of code shown below can be ignored/removed, it was some debug code that sneaked in to one of the changes I had made, and it seems I forgot to remove it before posting the updated version! >_<

All this did is just show the total number of clients currently connected every second in the console, nothing direly important or code-breaking if removed.

AdlibRegister("asdf", 1000)
Func asdf()
ConsoleWrite($Clients[0][0] & @CRLF)
EndFunc

Edited August 23, 2012 by Kealper

[Shinies]

Thanks, glad to hear it's as useful to other as it is to me!

PS: This bit of code shown below can be ignored/removed, it was some debug code that sneaked in to one of the changes I had made, and it seems I forgot to remove it before posting the updated version! >_<

All this did is just show the total number of clients currently connected every second in the console, nothing direly important or code-breaking if removed.

AdlibRegister("asdf", 1000)
Func asdf()
ConsoleWrite($Clients[0][0] & @CRLF)
EndFunc

Figured that was the case.. have been using it to debug also

[Kealper]

I've just made some more changes to it, this time to increase efficiency when large (over 1,000) active clients are connected and one or more clients disconnects. Turns out in this case that it is up to 500x faster, on my computer at least, to build a copy of the original array without the sub-elements you don't want and then overwrite the old array with the new copy than it is to use ReDim to re-size the one array. I would have figured the latter would have been much more efficient, but it seems there are always curve-balls in the game of programming.

I've changed the original Sleep(1) call in the main loop to a custom sleep function (credit goes to monoceres for the basics behind that function) which takes delays in microseconds instead of milliseconds, because it turns out the lower-level function AutoIt's Sleep wraps appears to only be able to go down to ~10ms, so there was a decent gain in response time just by switching that over. I've got it set to 5000us (5ms) sleep delay in the example, but you can safely go down to 1000us (1ms) without much of a resource-usage hike, from my testing that is; Anything lower than 1000us doesn't seem to give much more of a performance increase, so diminishing returns, etc...

I also corrected some more spelling (again... ), tweaked the little example echo server code in that base to better demonstrate my choice of why that line dealing with pulling packets from the buffer that many have said was incorrect is actually exactly how I intended it to function.

[Krsta]

Kealper: Thank you for the Idea. I'm refactored your code a bit:

expandcollapse popup

#cs ----------------------------------------------------------------------------
Fast TCP Server
Author: Petr Krstev
Based on "Fast multi-client TCP server" by Ken Piper
#ce ----------------------------------------------------------------------------
TCPStartup()
Opt("TCPTimeout", 0)

#region ;Safe-to-edit things are below
Global $BindIP = "0.0.0.0" ;Listen on all addresses
Global $BindPort = 8080 ;Listen on port 8080
Global $Timeout = 15000 ;Max idle time is 15 seconds before calling a connection "dead"
Global $PacketSize = 2048 ;Max packet size per-check is 2KB
Global $MaxClients = 50 ;Max simultaneous clients is 50
#endregion ;Stuff you shouldn't touch is below

Global $Listen
Global $Socket[$MaxClients] ; Rows of data
Global $RemoteIP[$MaxClients] ; Rows of data
Global $TimeStamp[$MaxClients] ; Rows of data
Global $Buffer[$MaxClients] ; Rows of data
Global $Stack[$MaxClients+2] ; Stack to hold numbers of the free rows of data
Global $SP = 0
Global $Ws2_32 = DllOpen("Ws2_32.dll") ;Open Ws2_32.dll, it might get used a lot
Global $NTDLL = DllOpen("ntdll.dll") ;Open ntdll.dll, it WILL get used a lot
Global $CleanupTimer = TimerInit() ;This is used to time when things should be cleaned up
Global $NeedExit = False

PreloadStack()

OnAutoItExitRegister("Close") ;Register this function to be called if the server needs to exit

$Listen = TCPListen($BindIP, $BindPort) ;Start listening on the given IP/port
If @error Then Exit 1 ;Exit with return code 1 if something was already bound to that IP and port

While 1
if $NeedExit Then ExitLoop ;Exit demanded from client
USleep(5000, $NTDLL) ;This is needed because TCPTimeout is disabled. Without this it will run one core at ~100%.
;The USleep function takes MICROseconds, not milliseconds, so 1000 = 1ms delay.
;When working with this granularity, you have to take in to account the time it takes to complete USleep().
;1000us (1ms) is about as fast as this should be set. If you need more performance, set this from 5000 to 1000,
;but doing so will make it consume a bit more CPU time to get that extra bit of performance.
Check() ;Check recv buffers...
DoIt() ; ...and do things
If TimerDiff($CleanupTimer) > 1000 Then ;If it has been more than 1000ms since Cleanup() was last called, call it now
$CleanupTimer = TimerInit() ;Reset $CleanupTimer, so it is ready to be called again
Cleanup() ;Clean up the dead connections
EndIf
Local $iSock = TCPAccept($Listen) ;See if anything wants to connect
If $iSock = -1 Then ContinueLoop ;If nothing wants to connect, restart at the top of the loop
If $SP = 0 Then ;If $SP = 0 then the max connection limit has been reached
TCPCloseSocket($iSock) ;It has been reached, close the new connection and continue back at the top of the loop
ContinueLoop
EndIf

Local $FreeSock = Pop() ;Get the next free row in data
$Socket[$FreeSock] = $iSock ;Set the socket ID of the connection
$RemoteIP[$FreeSock] = SocketToIP($iSock, $Ws2_32) ;Set the IP Address the connection is from
$TimeStamp[$FreeSock] = TimerInit() ;Set the timestamp for the last known activity timer
$Buffer[$FreeSock] = "" ;Blank the recv buffer
WEnd

Func Check() ;for incoming data
If $SP >= $MaxClients Then Return ;If there are no clients connected, stop the function right now
For $i = 0 To $MaxClients-1 ;Loop through all rows
If $Socket[$i] > 0 Then ;valid socket in the row
Local $sRecv = TCPRecv($Socket[$i], $PacketSize) ;Read $PacketSize bytes from the current client's TCP buffer
If $sRecv <> "" Then ;If there was data sent from the client
$Buffer[$i] &= $sRecv ;add it to the buffer
$TimeStamp[$i] = TimerInit() ;update the activity timer
EndIf
EndIf
Next
EndFunc

Func DoIt() ;Processing incoming data
For $i = 0 To $MaxClients-1 ;Loop through all rows
If $Buffer[$i] <> "" Then ;Data buffer in the row is not empty
#region ;Example data processing stuff here. This is handling for a simple "echo" server with line handling
Local $sLine = StringSplit($Buffer[$i], @CRLF, 1) ;Split up the data in to an array,...
For $j = 1 To $sLine[0]-1 ; ...so it is easy to loop through each line
If $sLine[$j] = "terminate" Then $NeedExit = True ;Client want terminate the server itself (not just connection)
TCPSend($Socket[$i], "Echoing line: '" & $sLine[$j] & "'" & @CRLF) ;Echo back the line the client sent
Next
$Buffer[$i] = $sLine[$sLine[0]] ;Put last incomplete line back in the buffer
#endregion ;Example
EndIf
Next
EndFunc

Func Cleanup() ;Clean up any disconnected clients to regain resources
If $SP >= $MaxClients Then Return ;If no clients are connected then return
For $i = 0 To $MaxClients-1 ;Loop through all rows
If $Socket[$i] > 0 Then ;valid socket in the row
$Buffer[$i] &= TCPRecv($Socket[$i], $PacketSize) ;Dump any data not-yet-seen in to their recv buffer
If @error Or TimerDiff($TimeStamp[$i]) > $Timeout Then ;Check to see if the connection has been inactive for a while or if there was an error
TCPCloseSocket($Socket[$i]) ;If yes, close the connection
$Socket[$i] = -1 ;Set the socket ID to an invalid socket
Push($i) ;Free this row
EndIf
EndIf
Next
EndFunc

Func Close()
DllClose($Ws2_32) ;Close the open handle to Ws2_32.dll
DllClose($NTDLL) ;Close the open handle to ntdll.dll
For $i = 0 To $MaxClients-1 ;Loop through all rows
If $Socket[$i] > 0 Then TCPCloseSocket($Socket[$i]) ;Force the client's connection closed
Next
TCPCloseSocket($Listen) ;Force the server's binded connection closed
TCPShutdown() ;Shut down networking stuff
; MsgBox(0,"","!",1)
EndFunc

Func SocketToIP($iSock, $hDLL = "Ws2_32.dll") ;A rewrite of that _SocketToIP function that has been floating around for ages
Local $structName = DllStructCreate("short;ushort;uint;char[8]")
Local $sRet = DllCall($hDLL, "int", "getpeername", "int", $iSock, "ptr", DllStructGetPtr($structName), "int*", DllStructGetSize($structName))
If Not @error Then
$sRet = DllCall($hDLL, "str", "inet_ntoa", "int", DllStructGetData($structName, 3))
If Not @error Then Return $sRet[0]
EndIf
Return "0.0.0.0" ;Something went wrong, return an invalid IP
EndFunc

Func USleep($iUsec, $hDLL = "ntdll.dll") ;A rewrite of the _HighPrecisionSleep function made by monoceres (Thanks!)
Local $hStruct = DllStructCreate("int64")
DllStructSetData($hStruct, 1, -1 * ($iUsec * 10))
DllCall($hDLL, "dword", "ZwDelayExecution", "int", 0, "ptr", DllStructGetPtr($hStruct))
EndFunc

Func PreloadStack()
Local $i
For $i = 0 to $MaxClients-1
Push($i)
Next
EndFunc

Func Push($Value)
if $SP < $MaxClients Then
$Stack[$SP] = $Value
$SP += 1
Return 1
Else
Return 0
EndIf
EndFunc

Func Pop()
if $SP > 0 Then
$SP -= 1
Return $Stack[$SP]
Else
Return -1
EndIf
EndFunc

My changes:

A/ using 4 1D flat arrays instead of 2D array. Each 1D array represents one column in DB/table holds same type of data. Same Index = same connection.

B/ using a "Stack" data structure for information about unused rows in the DB/table.

C/ dividing the Func "Check" into 2 Funcs: "Check" and "DoIt". One for Checking up for a new data, second for a processing new data.

D/ The Func "DoIt" have another type of line parsing. More simple with the possibily to invoke server termination from client.

E/ The Func "CleanUp" stay for socket disconnection and Timeout detection and lost the massive changes on the 2D array. Just push the free socket index on the Stack.

F/ The Func "Close" - added closing of listen socket too.

G/ There is 3 new Funcs: Push, Pop, PreloadStack. Push and Pop classical Stack working (LiFo data structure). PreloadStack pushes all the free indexes in the stack.

Krst'a

[caleb41610]

Test: create 1000 connections, then disconnect all connections.

Script on the first page: 90 seconds

Krsta's script: 94 seconds

my script: 16 seconds (14 sec without creating/deleting list view items.)

This is my implementation:

This is the code I used to test the time: (i just changed the ports on your scripts to 1337, and max connections to 1000)

TCPStartup()

Global Const $number_of_connections = 1000 ; Total number of connections to perform
Global Const $ip = @IPAddress1 ; IP address
Global Const $port = 1337 ; Port
Global $connection_array[$number_of_connections + 1]
$errors = 0
$timer = TimerInit()

For $i = 1 To $number_of_connections
$connection_array[$i] = TCPConnect($ip, $port)
If @error Then
$errors += 1
EndIf
Next
MsgBox(0, "", $errors & " errors" & @CRLF & $number_of_connections & " connections in " & Round(TimerDIff($timer) / 1000) & " seconds.")
$timer = TimerInit()

For $i = 1 To $number_of_connections
TCPCloseSocket($connection_array[$i])
If @error Then
$errors += 1
EndIf
Next
MsgBox(0, "", $errors & " errors" & @CRLF & $number_of_connections & " disconnects in " & Round(TimerDIff($timer) / 1000) & " seconds.")
$timer = TimerInit()

TCPShutdown()

Edited October 29, 2012 by caleb41610

[Krsta]

caleb41610: Thank you for pointing out performance-related problem. It is always important.

Tests that I did proved similar results. After 6 hours of unsuccessful experiments, I was very desperate.

Finally I found why the results are so different. All that magic is in command "TCPListen" where we (Me and Kealper too) use default value for the third parameter.

I get similar results to yours (without list items stuf), when I change my script (and Kealper too) like this:

$Listen = TCPListen($BindIP, $BindPort, $MaxClients) ;Start listening on the given IP/port

Finally, I made some additional changes to my script (see attachment) and now the result time is about 5s and there is still room to do further optimizations.

Thank you once more and Good luck

Petr

Test: create 1000 connections, then disconnect all connections.

Script on the first page: 90 seconds

Krsta's script: 94 seconds

my script: 16 seconds (14 sec without creating/deleting list view items.)

This is my implementation:

This is the code I used to test the time: (i just changed the ports on your scripts to 1337, and max connections to 1000)

ST.au3

[caleb41610]

Glad it helped. i may fix mine some day soon.

[Kealper]

Hmm.. Seems one of the forum updates or something cleared the followers list, so I wasn't getting notifications of people posting here or I would have responded sooner...

I didn't even catch that I didn't have the third parameter in this on the post or my local copy of that code... I have it in any of the stuff I do, it must have just slipped my mind to add it to the base while I was working on it, sorry about that!

Using caleb41610's test script, on the code in the original post (after correcting that simple mistake!) will do 1000 connections in 10 seconds, where as Krsta's final code will do 1000 connections in 5 seconds... The reason for this is how accepting connections is handled. Mine makes processing client data a priority by always checking if any clients have sent something before checking the new connection queue and getting new clients in, where as Krsta's makes accepting new clients a priority, by only checking current client data when no new connections are queued. If I change mine around a bit and make it so it only calls Check() when $iSock = -1, I see similar performance, down to 1000 clients in 5 seconds (mine has rounded up to 6 seconds a few times, so its average is slightly slower than Krsta's). If I had to guess, mine is still slower because it uses a dynamic array size for the main client array instead of Krsta's fixed array size, but for the things I do with this code the slight performance hit it incurs is worth the benefit of a flexible max client limit. So it seems mine is still being bit in the butt by overhead in ReDim, and since Krsta isn't using ReDim, no butt-biting goes on there. This has shown me what a difference it can make to switch to fixed-size for the clients array if I ever have a need for a client that can accept connections more consistently though.

Also I chose to do the dynamically re-sized 2d array for simplicity's sake, and for more flexibility, as max clients can be adjusted on-the-fly without requiring a restart (by just changing $MaxClients), and it would be much less work while developing off of this (as it was intended) to change a 2d array than it would be to create another 1d array, in my personal opinion.

(Updated first post with the quick-fix I overlooked before)

EDIT: I should also point out another down-side I hadn't thought of while initially writing this... With a fixed-size array, it tears through CPU cycles walking through the entire array each time, even if there isn't many clients connected. So if you've got it set to 10,000 total connections, it will have to walk through those 10,000 elements each time.

With a dynamically-sized array, the number of elements will always only be the number of active clients, so if the limit is 10,000 but there are only 10 clients connected, the array will only be around 10 elements large(11 if you use one as an index). Just thought I'd throw that out there too for a bit more understanding of why I made things the way I did.

Edited December 15, 2012 by Kealper

[vuxat]

ahoy

i have taken the liberty to transcribe a part of your code into a usable function

acts partly like adlibregister but does only check and run on call (so it does not interrupt current processes)

original code:

If TimerDiff($CleanupTimer) > 1000 Then ;If it has been more than 1000ms since Cleanup() was last called, call it now
        $CleanupTimer = TimerInit() ;Reset $CleanupTimer, so it is ready to be called again
        Cleanup() ;Clean up the dead connections
    EndIf

my function:

Func _starttimer($sttimr = 'Cleanup', $sttimrdiff = 1000)
    $sttimrn = $sttimr & 'Timer'
    If IsDeclared($sttimrn) = 0 Then Assign($sttimrn, _Timer_Init(), 2)
    If TimerDiff(Eval($sttimrn)) > $sttimrdiff Then
        Assign($sttimrn, TimerInit())
        Return Execute($sttimr & '()')
    EndIf
EndFunc

you can call it like this in your loop and it will start the given function after X msec

_starttimer('Cleanup', 1000)

best regards,

stefan

[Jimtags]

Since I'm using this tcp server script..
Here's some good additional, A packet dynamic encryption

Func PacketEnc($EncData,$EncKey)
Local $EncOut1 = Random(111111,999999,1)
Local $EncOut2 = _Crypt_EncryptData($EncOut1&$EncData, $EncKey, $CALG_AES_256)
Return BinaryToString($EncOut2)
EndFunc

Func PacketDec($DecData,$DecKey)
Local $DecOut1 = _Crypt_DecryptData($DecData, $DecKey, $CALG_AES_256)
Local $DecOut2 = BinaryToString($DecOut1)
Return StringTrimLeft($DecOut2, 6)
EndFunc

-edit: removed deprecated _StringEncrypt()

Edited May 8, 2014 by Jimtags

[guinness]

_StringEncrypt() is deprecated in the latest version of AutoIt.

[Jimtags]

_StringEncrypt() is deprecated in the latest version of AutoIt.

 

But still usable.. anyway that's just a example for how to make the packet not static.. It can help someone maybe

[guinness]

But still usable.. anyway that's just a example for how to make the packet not static.. It can help someone maybe

I should have removed it when it was deprecated, but I was being too nice for those who needed time to upgrade their scripts (don't I regret that decision now). You're welcome to use outdated code in your own scripts, just don't let you bad habits leak onto the Forums in the future please.

To all,

I provided an alternative StringEncrypt() in _Crypt_EncryptData(), though it's not backwards compatible with _StringEncrypt().

[jchd]

To add horror to what guinness said, old _StringEncrypt() is less reliable and robust than ROT13.

Edit: made clear I was talking about the legacy version.

Edited April 22, 2014 by jchd

[guinness]

To add horror to what guinness said, StringEncrypt() is less reliable and robust than ROT13.

I guess you meant the old version? Yeah I second that, even ROT5 is more secure than that old function.

[jchd]

Exactly. Post edited for clarity.

[JFC14]

Something is broken with newest Autoit version and Win7 64bit.

Running the server script with no modifications, it allows telnet connections but immediately kills the connection when you type anything (less than 1 second).

Disabling the "cleanup" function causes it to reply the text properly.

FOUND THE PROBLEM:
The newest version of AutoIt changes the behavior of TCPRecv:

'?do=embed' frameborder='0' data-embedContent>>

You need to change line 90 from

 

If @error Or TimerDiff($Clients[$i][2]) > $Timeout Then ;Check to see if the connection has been inactive for a while or if there was an error

 

To:

 

 

If (@error And @error <> -1 ) Or TimerDiff($Clients[$i][2]) > $Timeout Then ;Check to see if the connection has been inactive for a while or if there was an error

 

@error = -1 is not actually an error... it just means there is no data received at the moment.  

 

 

Also, another problem I found is that if there is anything in the recv buffer, it will never timeout.  So in your telnet example if you type something but dont push enter, it will never timeout.  This is because when there is anything in the recv buffer, it keeps resetting the timeout timer.  This means a client could be connected forever when they really shouldn't be.  Perhaps instead of just checking to see if the buffer is empty, see if it has changed as well.

 

 

 

 

Edited October 11, 2014 by JFC14