_INetGetSource not working

Seminko · November 22, 2016

Gettin the same error again:

12039 ERROR_INTERNET_HTTP_TO_HTTPS_ON_REDIR
The application is moving from a non-SSL to an SSL connection because of a redirect.

EDIT: hmm, actually no... I just ran below code:

$Source = BinarytoString(InetRead("http://www.hearthpwn.com/?cookieTest=1"))
If @error Then
    MsgBox(1, "Error", @extended)
Else
    Msgbox(0,"", $Source)
EndIf

I received an Error, but the extended said the error... is ... 0? What?

Edited November 22, 2016 by Seminko

j0kky · November 22, 2016

Here is the story: heartpwn.com has implemented a way to dissuade simple script like yours. When you try to download the source code from that site, it begins to send you redirection headers which cause InetRead fail.

Try last test with this script, and report us the console output:

$Source = InetRead("http://www.hearthpwn.com/?cookieTest=1")
If @error Then
    ConsoleWrite("InetRead Error: " & @error & @CRLF & _ 
        "Extended: " & @extended & @CRLF)
    Exit
EndIf
ConsoleWrite("Bytes read: " & @extended & @CRLF)
$Source = BinarytoString($Source)
If @error Then
    ConsoleWrite("BinarytoString Error: " & @error & @CRLF & _ 
        "Extended: " & @extended & @CRLF)
    Exit
EndIf
ConsoleWrite($Source & @CRLF)

Edited November 22, 2016 by j0kky

Seminko · November 22, 2016

9 minutes ago, j0kky said:

Here is the story: heartpwn.com has implemented a way to dissuade simple script like yours. When you try to download the source code from that site, it begins to send you redirection headers which cause InetRead fail.

Well I was thinking the same thing. But. How do you explain the fact that AndyG and mikell are able to get the source code using the same script and I can't?

Here is the console output:

InetRead Error: 13
Extended: 12039

Edited November 22, 2016 by Seminko

j0kky · November 22, 2016

I have no idea, maybe there is something strange relative to some setting on your OS.

Just for curiosity, what does it output?

$Source = InetRead("https://www.hearthpwn.com/?cookieTest=1")
If @error Then
    ConsoleWrite("InetRead Error: " & @error & @CRLF & _ 
        "Extended: " & @extended & @CRLF & _ 
        "Source: " & $Source & @CRLF)
    Exit
EndIf
ConsoleWrite("Bytes read: " & @extended & @CRLF)
$Source = BinarytoString($Source)
If @error Then
    ConsoleWrite("BinarytoString Error: " & @error & @CRLF & _ 
        "Extended: " & @extended & @CRLF)
    Exit
EndIf
ConsoleWrite($Source & @CRLF)

Edited November 22, 2016 by j0kky

Seminko · November 22, 2016

2 minutes ago, j0kky said:

I have no idea, maybe there is something strange relative to some setting on your OS.

Just for curiosity, what does it output?

Output:

InetRead Error: 13
Extended: 12040
Source:

j0kky · November 22, 2016

Well, my suggestion is to parse every data you need from the output of my script

Edited November 22, 2016 by j0kky

Seminko · November 22, 2016

13 minutes ago, j0kky said:

Well, my suggestion is to parse every data you need from the output of my script

Will have to do for now

How do I update your script so it works with with more specific links like: http://www.hearthpwn.com/decks/670319-tempo-aggro-mage-82-ratio-easy-rank-5 ?

j0kky · November 23, 2016

I modified my previous script to allow you to change easily the URL inserted:

TCPStartup()
If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
Local $sIPAddress = TCPNameToIP("www.hearthpwn.com")
If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
Local $iSocket = TCPConnect($sIPAddress, 80)
If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
$sLink = "http://www.hearthpwn.com/decks/670319-tempo-aggro-mage-82-ratio-easy-rank-5"
$sRegEx = StringRegExp($sLink, "hearthpwn\.com/(.*)", 1)
If @error Then _ReportError("Bad link", "", @ScriptLineNumber - 1)
$sLink = $sRegEx[0]
TCPSend($iSocket, "GET /" & $sLink & "?cookieTest=1 HTTP/1.1" & @CRLF & _
    "Host: www.hearthpwn.com" & @CRLF & _
    "Connection: close" & @CRLF & @CRLF)
If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
Local $sData = "", $hTimer = TimerInit()
While 1
    $sRecv = _TCPRecv($iSocket, 2048)
    If $sRecv Then $sData &= $sRecv
    If @error Or @extended Then
        If @error Then _ReportError(@error, @extended, @ScriptLineNumber - 1)
        ExitLoop
    EndIf
    If $sRecv Then $hTimer = TimerInit()
    If TimerDiff($hTimer) > 3000 Then ExitLoop
    Sleep(10)
WEnd
ConsoleWrite("Response received: " & $sData & @CRLF)
TCPCloseSocket($iSocket)
TCPShutdown()

Func _ReportError($iError, $iExtended, $iLine)
    ConsoleWrite("Error: " & $iError & @CRLF & _
        "Extended: " & $iExtended & @CRLF & _
        "Line: " & $iLine & @CRLF)
    TCPShutdown()
    Exit
EndFunc

Func _TCPRecv($iMainsocket, $iMaxLen, $iFlag = 0)
    If IsArray($iMainsocket) And (UBound($iMainsocket, 0) = 1) And (UBound($iMainsocket) > 0) Then $iMainsocket = $iMainsocket[0]
    If $iFlag = Default Then $iFlag = 0
    $iMainsocket = Number($iMainsocket)
    $iMaxLen = Number($iMaxLen)
    $iFlag = Number($iFlag)
    If $iMainsocket < 0 Or _
            $iMaxLen < 1 Or _
            Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(-4, 0, -1) ; invalid parameter

    Local $hWs2 = DllOpen("Ws2_32.dll")
    If @error Then Return SetError(-2, 0, -1) ;missing DLL
    Local $bError = 0, $nCode = 0, $nExtended = 0

    If Not $bError Then
        $aRet = DllCall($hWs2, "int", "ioctlsocket", "uint", $iMainsocket, "long", 0x8004667e, "ulong*", 1) ;FIONBIO
        If @error Then
            $bError = -1
        ElseIf $aRet[0] <> 0 Then ;SOCKET_ERROR
            $bError = 1
        EndIf
    EndIf

    Local $tBuf = DllStructCreate("char[" & $iMaxLen & "]")
    $aRet = DllCall($hWs2, "int", "recv", "uint", $iMainsocket, "ptr", DllStructGetPtr($tBuf), "int", $iMaxLen, "int", 0)
    If @error Then
        $bError = -1
    ElseIf ($aRet[0] = -1) Or ($aRet[0] = 4294967295) Then ;SOCKET_ERROR
        $bError = 1
        $aRet = DllCall($hWs2, "int", "WSAGetLastError")
        If @error Then
            $bError = -1
        ElseIf $aRet[0] = 0 Or $aRet[0] = 10035 Then ;WSAEWOULDBLOCK
            $nCode = -10 ;internal function value, it means no error
        EndIf
    ElseIf $aRet[0] = 0 Then
        $bError = 1
        $nCode = -10
        $nExtended = 1 ;connection closed
    Else
        Local $sResult = DllStructGetData($tBuf, 1) ;data
        If BitAND($iFlag, 2) = 2 Then ;EOT
            If StringRight($sResult, 1) = Chr(3) Then
                $sResult = StringTrimRight($sResult, 1)
                $nExtended = 2 ;End of Text reached
            EndIf
        EndIf
        If BitAND($iFlag, 1) = 1 Then $sResult = Binary($sResult)
    EndIf

    If $bError < 0 Then
        $nCode = -1 ;internal error
        $nReturn = "" ;failure
    ElseIf $bError > 0 Then
        If Not $nCode Then
            $aRet = DllCall($hWs2, "int", "WSAGetLastError")
            If @error Then
                $nCode = -1
            Else
                $nCode = $aRet[0]
            EndIf
            If $nCode = 0 Then $nCode = -3 ;undefined error
        EndIf
        If $nCode = -10 Then $nCode = 0
        $nReturn = ""
    Else
        $nReturn = $sResult
    EndIf
    DllClose($hWs2)
    Return SetError($nCode, $nExtended, $nReturn)
EndFunc   ;==>_TCPRecv

You just need to edit $sLink variable on line 7...

You should follow some recommendation before starting:

- you are sending a GET request to the server, and it is composed by various headers, try to know at least the ones you are using

- server answers to you with a response composed by a status code, some other headers and (maybe) the datas you are searching for

- if you are trying to download source code from a private area of that site (which requires a login to be viewed), you must be able to play with Set-Cookie and Cookie headers

Have fun with your project!

Edited November 23, 2016 by j0kky

Seminko · November 23, 2016

I can't thank you enough j0kky!

However, one more thing, I'm actually grabbing data from three different sites, in the future maybe many more. Other sites than HearthPwn don't work. I believe it is due to:

Local $sIPAddress = TCPNameToIP("www.hearthpwn.com")
$sRegEx = StringRegExp($sLink, "hearthpwn\.com/(.*)", 1)

Is there a way to make it "dynamic" or do you have to "hardcode" it for each site?

j0kky · November 23, 2016

Read about:

TCPNameToIP
StringRegExp

and try yourself, feel free to ask if you run into difficults!

Edited November 23, 2016 by j0kky

_INetGetSource not working

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members