issue with TCPRecv() in 3.3.10.2

[garbb]

I had been using some tcp functions to communicate with an IRC server with autoit v3.3.8.1 and it was working fine. But now when using the same script with v3.3.10.2 I keep getting @error set from TCPRecv() unexpectedly...

Here's the test code:

expandcollapse popup

opt('trayicondebug',1)
Opt("TrayOnEventMode", 1)
Opt("Traymenumode", 2)
Opt("Trayautopause", 0)

Global $sock

OnAutoItExitRegister('exitfunc')
func exitfunc()
    ConsoleWrite('TCPShutdown()='&TCPShutdown()&@CR)
EndFunc

TrayCreateItem("Send command")
TrayItemSetOnEvent(-1, "input")
func Input()
    local $cmd=InputBox('Send command','type a command to send to the IRC server')
    if $cmd='disconnect' then
        $ret=TCPCloseSocket($sock)
        $error=@error
        msgbox(0,0,$ret&' '&$error)
    EndIf
    ConsoleWrite('SENDING "' & $cmd & '"'&@CRLF)
    TCPSend($sock, $cmd & @CRLF)
    If @error Then
        MsgBox(1, "IRC.au3", "Server has disconnected.")
        Return -1
    EndIf
    Return 1
EndFunc

Global $server = "irc.irchighway.net"
Global $port = 6667
;~ Global $server = "chat.freenode.net"
;~ Global $port = 6665
Global $nick = "au3bot"
Global $channel = "#AutoIt"

TCPStartup()
$sock = _IRCConnect($server, $port, $nick); Connects to IRC and Identifies its Nickname

While 1
    $recv = TCPRecv($sock, 8192); Recieve things from server
    If @error Then Exit MsgBox(1, "IRC Example", "Server has errored or disconnected"&@CR&'@error='&@error); If you can't recieve then the server must have died.
    Local $sData = StringSplit($recv, @CRLF); Splits the messages, sometimes the server groups them
    For $i = 1 To $sData[0] Step 1; Does each message seperately
        if $sData[$i]<>'' then ConsoleWrite($sData[$i]&@CRLF)
        Local $sTemp = StringSplit($sData[$i], " "); Splits the message by spaces
        If $sTemp[1] = "" Then ContinueLoop; If its empty, Continue!
        If $sTemp[1] = "PING" Then _IRCPing($sock,$sTemp[2]); Checks for PING replys (There smaller then usual messages so its special!
        If $sTemp[0] <= 2 Then ContinueLoop; Useless messages for the most part
        Switch $sTemp[2]; Splits the command msg
            Case "266"; It's what I use as an indictator to show when its done sending you info.
                _IRCJoinChannel ($sock, $channel)
        EndSwitch
    Next
    sleep(100)
WEnd


Func _IRCConnect ($server, $port, $nick)
    Local $i = TCPConnect(TCPNameToIP($server), $port)
    If $i = -1 Then Exit MsgBox(1, "IRC.au3 Error", "Server " & $server & " is not responding.")
    TCPSend($i, "NICK " & $nick & @CRLF)
    ConsoleWrite('SENDING "NICK ' & $nick & '"'&@CRLF)
    TCPSend($i, "USER " & $nick & " 0 0 " & $nick &@CRLF)
    ConsoleWrite('SENDING "USER ' & $nick & ' 0 0 ' & $nick &'"'&@CRLF)
    Return $i
EndFunc

Func _IRCJoinChannel ($irc, $chan)
    If $irc = -1 Then Return 0
    ConsoleWrite('SENDING "JOIN ' & $chan & '"'&@CRLF)
    TCPSend($irc, "JOIN " & $chan & @CRLF)
    If @error Then
        MsgBox(1, "", "Server has disconnected.")
        Return -1
    EndIf
    Return 1
EndFunc

Func _IRCPing($irc, $ret)
    If $ret = "" Then Return -1
    ConsoleWrite('SENDING "'&'PONG ' & $ret & '"'&@CRLF)
    TCPSend($irc, 'PONG ' & $ret & @CRLF)
        If @error Then
        MsgBox(1, "", "Server has disconnected.")
        Return -1
    EndIf
    Return 1
EndFunc

This script is supposed to just connect to the IRC server with the specified nick and join the specified channel. I added a tray icon menu item to input additional commands to send to the server for testing purposes.

NOTE: You will need to run this in SciTE to be able to see the output from the server in the bottom of the window.

What's strange is that I get different results connecting to different servers. For example, when connecting to irc.irchighway.net:6667 I get a lot of initial response text from the server before I get @error from TCPRecv() and when connecting to chat.freenode.net:6665 I get @error from TCPRecv() almost immediately. Both of these servers work totally fine in autoit v3.3.8.1. 

I looked at the version history and the only changes I see mentioned for TCP functions are:

• Fixed #2384: UDPRecv() and TCPRecv() not setting @error correctly in some instances.
• Fixed #2171: TCPTimeout inconsistent.

 

I tried changing the TCPTimeout option to make it larger but all it does is create a delay before TCPRecv() returns @error.

Is this a bug?

Hmm I found this: http://www.autoitscript.com/trac/autoit/ticket/2596 which seems to indicate the TCPRecv() is setting @error=-1 prematurely? And maybe can just be ignored...

Ok, well I guess I may have solved my own problem. I just changed all the code that that says "if @error" to "if @error>0" after each TCPRecv() call and now it works fine. It seems that the TCPRecv() function sets @error differently now which doesn't seem to have been mentioned in the version history or the new help file...

Edited January 23, 2014 by garbb

[Mat]

Does the fix in the last comment on the trac ticket fix it? Rather than ignoring @error = -1, just sleep for a bit before trying again?

[garbb]

I have my code running in a loop calling TCPRecv() constantly to get data for the server and I changed it to output @error from TCPRecv() and it will be -1 repeatedly until some more data is sent from the server so that means that waiting for a while will not help.

It appears that what is happening is that TCPRecv() is setting @error=-1 when the tcp buffer is empty. So really the only problem is that this change was not documented anywhere. Unless the author did not mean to do this intentionally?

Edited January 23, 2014 by garbb

[FixTCPRecvForNewVersions]

This update to tcprecv messed my server all up and temping me to go back to the previous Autoit version because of it.  Setting @error to -1 because of no data received is not as efficient as the older version's method of ignoring blank returns.  My server now is much much less stable than before since now it can't detect when a client is no longer on the line and now have to force the connection to close (not as effective/stable force closing connections on slower clients)...  My persistent connections are grossly slower than before cause now I have to do all these additional checks to by-pass force closing the connection.  There is no way the Autoit team intended this and I can't wait for a fix!

Bottom line...  

Previous version of TCPRecv could easily detect a closed client side socket by detecting errors. (web clients)

New version of TCPRecv can not detect a closed client side socket because @error -1 is a joke to be ignored. (web clients)

I love Autoit3 just as much as anyone else, and this is the first time i'v been disappointed by some thing they did.

Please fix the @error so that it acts more like the older TCPRecv version ASAP please!

[ekeem]

I am having the exact issue as "FixTCPRecvForNewVersions".  I built a client/server application, very similiar to a chat app.  This update broked it.  Issue: New version set @error to -1 when no data received on an active socket.  Previously, I was able to detected a client disconnect by detecting @error = -1.  Hope this will be fix, I will have to rewrite a lot of code to circumvent this change/bug.

[FireFox]

There are already two tickets opened for this "issue".

I'm not conviced it's an issue, although there may be some code to correct in order to avoid our confusion.

[rcmaehl]

There are already two tickets opened for this "issue".

I'm not conviced it's an issue, although there may be some code to correct in order to avoid our confusion.

 

According to the Documentation for TCPRecv 3.3.10.2

 

@error: -1 Socket error

 

Now, I personally wouldn't consider a lack of data an error, and most Functions, that I know of, don't throw an error when there's no data. (E.g. StringToBinary doesn't throw an Error when no input (Null) is received).

Sure, you can do what Mat suggested and sleep before checking again, but that really doesn't work in variable connections in which you don't know when you're going to receive data, or how much data you're going to receive.

[FixTCPRecvForNewVersions]

Best solution to this problem so far is to downgrade to previous versions of Autoit3 (3.3.8.1) that use the older TcpRecv methods.  

The "sleep" solution did not solve anything as far as I could tell.

I do not care about/notice any bugs/problems in older versions of Autoit3 and would NEVER consider using the newer versions of Autoit in TcpRecv heavy dependent scripts until @error is not set for blank returns.  

(I have been spoiled and took it for granted by how well the older TcpRecv worked for me)

 

I still love Autoit3 and appreciate the work put behind it, must be a monumental amount.

The Autoit team deserves all the donations/credit they can get.

[FixTCPRecvForNewVersions]

Anybody care to explain to me why this keeps returning @error 10053 for both tcp send and tcp recv?

Also if TCPsend is having errors sending data and even the data sent value is ZERO then why does the page keep loading the content?

 

 

Theory; (not tested)

I'm guessing since it is in a loop that the first TCPsend got through, then TCPRecv cant figure out right away that the socket has been closed (Where 3.3.8.1 knows instantly). So the data is trying to be sent a second time on a closed client side socket.  (Tired of switching back and forth between autoit versions to test).

Also i'm certain TCPRecv @error 10053 is not correct error for client closing a browser/socket.  (Not that it matters to me much, I just bring it up because the newer version loves to boast about how accurate their errors are.)

 

Run this server then open a web browser to the host address.

TCPStartup()
$serversock = TCPListen('127.0.0.1', 81)
If @error Then Exit ; we know this one
$data = "data"
While 1
    Do
        $hi = TCPAccept($serversock)
        If @error Then Exit ; we know this one.
    Until $hi <> -1
    MsgBox(0,"Connected","A client has connected...")

    ;$ti = TimerInit()

    While 1
        TCPRecv($hi,1500)
        If @error > 0 Then
            MsgBox(0,"Closed","Closing the connection: " & @error)
            TCPCloseSocket($hi)
            ExitLoop
        EndIf

        ;If TimerDiff($ti) > 99 Then
            $bits = TCPSend($hi,"HTTP/1.0 200 OK" & @CRLF & 'Content-Length: ' & StringLen($data) & @CRLF & @CRLF & $data)
            If @error Then MsgBox(0,"SEND ERROR","@error: " & @error & @CRLF & @CRLF & "Sent:" & $bits & " bytes")
        ;   $ti = TimerInit()
        ;EndIf
    WEnd
WEnd

I'm finding so many irregularities with the new TCPRecv I can't believe anyone can get it to work!

Another thing is that when you comment out the TCPSend then connect with the browser and then close the browser, the socket will stay open FOREVER!

Lets say you have a server and some body connects to it and send no data, then close the browser and turn this into a never ending loop, how many open sockets can you have before your server crashes?

I can only get small script examples to work in new versions of autoit3.  The main server I use is still crippled by the new update and I can't for the life of me or by anyone else's figure out why....  (I'd really like to know why, but its wasting too much of my time)

Back to 3.3.8.1

If you modify this script to support version 3.3.8.1 everything works perfectly. (No errors at all.)

For 3.3.8.1 you got to change the @error > 0 to simply @error.

However if you do not modify the script to support 3.3.8.1 you get similar effects as 3.3.10.2 does.

Very confusing and I can't figure out why people are thinking that version 3.3.10.2's TCPRecv is any kind of better...

Edited April 30, 2014 by FixTCPRecvForNewVersions

[FixTCPRecvForNewVersions]

Does anyone have a working 3.3.10.2 web server that does not rely on TCPSend errors in order to close sockets?

I am convinced over here that TCPRecv is broken when it comes to detecting client side closed web browser sockets, yet it works for Autit3 direct connections.

The above script should prove that TCPRecv is broken when detecting client side web browser closed sockets and MUST rely on TCPSend in order to detect the closed socket which is why it constantly reports 10053.  Which might be the correct error (Protocal error) because TCPSend is trying to send data on a closed client side socket thus creating 10053, somehow TCPRecv is able to pick up on that and report the same, and THEN finally closes the socket.

If you have a working example server that can close sockets without using TCPSend to detect the socket i'd love love love to see it....  

3.3.10.2 TCPRecv is BROKEN and can't detect web browser closed sockets without TCPSend!

What it is doing...

1. A connection is made via web browser to the server.

2. Server tries to receive the data via TCPRecv.

3. Server tries to respond to the data via TCPSend. (Client browser gets the data and then closes the socket and displays the content)

4. Server tries to detect closed sockets or errors with TCPRecv and says there are none, which is WRONG!  (So it acts like it is still open, when it is in fact by now CLOSED)

5. Server tries to respond to the data a SECOND time on a CLOSED client side socket which ERRORS 10053!

6. Server tries to detect closed sockets or errors and this time detects the closed socket and reports error 10053 (Which in WORKING versions would be 10054, and reported before TCPSend)

7. Server closes the socket finally.

3.3.8.1 does not have this problem and works perfectly with Autoit3 directly connected clients and web browsers!

Edited May 1, 2014 by FixTCPRecvForNewVersions

[garbb]

I am using an autoit-based webserver that is based on the "Basic server" from here: '?do=embed' frameborder='0' data-embedContent>>.

If you change the following it works fine for me in 3.3.10.2:

line 54 from 

If @error Then

to 

If @error>0 Then

and line 65 from

_HTTP_SendError($aSocket[$x])

to

_HTTP_SendFileNotFoundError($aSocket[$x])

(i think this is just a mistake because the _HTTP_SendError() function does not exist...)

I think the reason that this server works is because it is always immediately closing the connection/socket after sending the data.

[FixTCPRecvForNewVersions]

Yeah, that script does not use TCPRecv error detection to close sockets.  

I'd like to see a working example in 3.3.10.2 that can successfully demonstrate TCPRecv detecting a closed web browser client like 3.3.8.1 was able to do.  Without involving other things like TCPSend to detect the error for it.

Close the connection immediately after sending data can cause slow clients problems mostly "Server closed the connection before the data could be received" or similar errors.  Mostly a problem when trying to serve requests that have poor internet.

Thank you for that example.

[garbb]

I could be wrong as I am not very knowledgeable on this topic but i believe that the way that most modern browsers work is that they keep a persistent connection open to a server in case they are going to request more data in the near future and it is up to the server to determine when to close the connection; usually after a timeout period.

See the section "Persistent Connections" here: http://odetocode.com/articles/743.aspx

You also might want to have a look at the Session_server in the autoit server I linked above because I believe that it implements such timeouts.

Also if I were you I might try creating a very simple client application in another autoit script and have it connect and disconnect to your server running autoit 3.3.10.2 so that you can see exactly what is going on and if it can detect when the client disconnects properly.

Edited May 1, 2014 by garbb

[FixTCPRecvForNewVersions]

Even if it is a persistent connection, then closing the browser should be detectable by TCPRecv @error 10054...  I bet if you tested this theory it will error out as 10053 if your using the script posted on this forum.

I have tested with client/server software and it works fine, which makes this a confusing problem why a web client connection does not.

I'll run more tests on the persistent connection theory when I get a chance.

[FixTCPRecvForNewVersions]

Persistent connections are made based on header values, in the provided script above, they are not used because of the header value HTTP/1.0 

If you change that to http/1.1 then persistent connections will be made.

If you do use the persistent connections then that test script would need to be modified to stop sending data two times, maybe even put in the timeout like you suggested.  But the basic concept of the test is the same when using either persistent or non persistent connections.  

With persistent connections you have to wait for the timeout or close the browser, either way you still get 10053.

(Oh sorry, closing the browser wont work because TCPRecv can't detect the error, you have to wait for timeout)

With non persistent connections you can use the above script and just navigate to the site in order to trigger the 10053.

The overall goal is to find the 10054 @error that tells TCPRecv that the client side socket is close, as i'm sure you know.  I'm just trying to be redundantly clear.

Thank you for bringing up that information I had no idea persistent connections were so easy to make!

But i'm still convinced TCPRecv in new versions can not detect the @error 10054 from a closed web browser client and the connection type does not seem to matter.  Thanks for the feedback!

Edited May 1, 2014 by FixTCPRecvForNewVersions

[FixTCPRecvForNewVersions]

Autoit3 admits there is a problem with TCPRecv @error detection.  That alone makes me rest easy and now I know i'm not going crazy.  I appreciate the members of Autoit3's hard work and thank them for acknowledging the problem.

http://www.autoitscript.com/trac/autoit/ticket/2596

 

I still love Autoit3 and the Autoit3 team deserves all the donations and credit they get.

[matwachich]

Hi!

I'm using the newest beta (3.3.15.0) and TCPRecv is always buggy! I made a simple multi-clients TCP server, all works fine until the moment where TCPRecv should detect that the other side closed the connection (either by explicitly calling TCPCloseSocket, killing the program, or stopping refresh when I test using web browser).

The new maps are absolutly great for dealing with this kind of TCP servers (I can make it standard, flexible, reusable and the best : you can have multiple servers in one script with the UDF I'm developping!), but this little bug ruins it all

Edit:

here is an example script : run it, and connect with your browser to 127.0.0.1:8080, when the console displays the request, cancel the browsing. The disconnection is never detected!

#NoTrayIcon

TCPStartup()
OnAutoItExitRegister(TCPShutdown)

$hListen = TCPListen("0.0.0.0", 8080)

Do
    $hSock = TCPAccept($hListen)
    Sleep(250)
Until $hSock > 0
ConsoleWrite("Connected!" & @CRLF)

While 1
    $recv = TCPRecv($hSock, 1024, 1)
    If @error Then ExitLoop
    ; ---
    If $recv Then ConsoleWrite(BinaryToString($recv, 4))
    ; ---
    Sleep(100)
WEnd

ConsoleWrite(@CRLF & @CRLF)
ConsoleWrite("Disconnected (error = " & @error & ")" & @CRLF)

TCPCloseSocket($hSock)
TCPCloseSocket($hListen)

 

Edited August 24, 2015 by matwachich