Base64 decoder/encoder, Internet header decoders, email subject decoder, UTF-8, ISO-8859-1, telnet negociations

[noorm]

Hello!

I've been lurking around for a loooong time... and I decided to finally share a little. I do a lot of internet stuff, mostly machine to machine for work (instrumentation) so I have quite a few "RFC" scripts.

Disclaimer these work for me... but I sometime use... "shortcuts" based on my particular requirement. An example, the Base64 encoding snippet might not be too good for binary data. I pad the original data with spaces to avoid the "==" padding of base64.

So... first is the base64 encoding snippet. It is not in a function, it was in a sequential program, used only once! It encode $Graph to $SMTPMessage:

expandcollapse popup

; Create the base64 encoding table
Dim $Base64EncodingTable[0]
For $Cpt = Asc("A") to Asc("Z")
    _ArrayAdd($Base64EncodingTable, Chr($Cpt))
Next
For $Cpt = Asc("a") to Asc("z")
    _ArrayAdd($Base64EncodingTable, Chr($Cpt))
Next
For $Cpt = Asc("0") to Asc("9")
    _ArrayAdd($Base64EncodingTable, Chr($Cpt))
Next
_ArrayAdd($Base64EncodingTable, "+")
_ArrayAdd($Base64EncodingTable, "/")

; Pad the SVG Graph to attach with space(s). Lazy way to avoid base64 == pading
While Mod(StringLen($Graph), 3) <> 0
    $Graph &= " "
WEnd

; Start from the first character
$Cpt = 1

Do
    ; Extract the 3 characters to encode
    $Char1 = Asc(StringMid($Graph, $Cpt, 1))
    $Char2 = Asc(StringMid($Graph, $Cpt+1, 1))
    $Char3 = Asc(StringMid($Graph, $Cpt+2, 1))

    ; Encode them to 4 characters
    $SMTPMessage &= $Base64EncodingTable[BitShift(BitAND($Char1, 252), 2)]
    $SMTPMessage &= $Base64EncodingTable[BitShift(BitAND($Char1, 3), -4) + BitShift(BitAND($Char2, 240), 4)]
    $SMTPMessage &= $Base64EncodingTable[BitShift(BitAND($Char2, 15), -2) + BitShift(BitAND($Char3, 192), 6)]
    $SMTPMessage &= $Base64EncodingTable[BitAND($Char3, 63)]

    ; Increment the counter, and if required, add a @CRLF to split in multiples lines
    $Cpt += 3
    If Mod($Cpt, 57) = 1 Then $SMTPMessage &= @CRLF

; Do this until all the graph has been encoded
Until $Cpt >= StringLen($Graph)

 

Second... I just finished this one and was allready thinking about sharing it... so it's been encapsulated into function a bit more. I use it to decode email subjects in a system where you can update something by email. I separated the Base64Decode function so it can be grabbed more easily. Please note that it return an hex string so you would still need to convert it if it's a string with BinaryToString or whatever suit your needs.

If can be copied as is and runned directly... it include my test strings! (Yes... I'm french!)

expandcollapse popup

; For the $SB_UTF8 and $SB_ANSI Variable
#include <StringConstants.au3>

; For _ArrayAdd and _ArraySearch used in the Base64 decoder
#include <Array.au3>

; Various test sentences...
;$text = "=?UTF-8?Q?Ce=c3=a7i_est_un_autre_test!_h=c3=a9h=c3=a9!?="         ; Normal UTF-8
;$text = "=?UTF-8?Q?Encore_=3d_un_autre_test_=c3=a9_?="                     ; "=" added
;$text = "=?UTF-8?Q?un_autre_test_=5f_=c3=a9?="                             ; "_" added
;$text = "=?UTF-8?B?Q2XDp2kgZXN0IHVuIGF1dHJlIHRlc3QhID0gXyBow6low6kh?="     ; UTF-8 Base64
$text = "=?UTF-8?B?ZcOnaSBlc3QgdW4gYXV0cmUgdGVzdCEgPSBfIGjDqWjDqSE=?="      ; UTF-8 Base64 with padding
;$text = "=?iso-8859-1?Q?Ce=E7i_est_un_test!?="                             ; iso-8859-1
MsgBox(0, "", DecodeHeader($text))

Func DecodeHeader($lString)

    ; Check and store encoding type
    If StringInStr($lString, "?Q?") Then
        ; Quoted printable content
        $lType = "?Q?"
    ElseIf StringInStr($lString, "?B?") Then
        ; Base64 encoding
        $lType = "?B?"
    Else
        ; No encoding (or unknown encoding)
        return($lString)
    EndIf

    ; Start of the charset string
    $lStart = StringInStr($lString, "=?") + 2

    ; End of the charset string
    $lStop = StringInStr($lString, $lType)

    ; Charset variable, storing "UTF-8" or "iso-8859-1"
    $lEncoding = StringMid($lString, $lStart, $lStop-$lStart)

    ; Change encoding type for the BinaryToString flag
    If $lEncoding = "UTF-8" Then
        $lEncoding = $SB_UTF8
    ElseIf $lEncoding = "iso-8859-1" Then
        $lEncoding = $SB_ANSI
    Else
        MsgBox(0, "", "Unknown character set")
        Exit
    EndIf

    ; Start of the actual encoded content
    $lStart = $lStop + 3

    ; End of the actual encoded content
    $lStop = StringInStr($lString, "?=")

    ; Actual content to decode
    $lString = StringMid($lString, $lStart, $lStop-$lStart)

    ; For Quoted printable content
    If $lType == "?Q?" Then

        ; Restore underscore encoded spaces
        $lString = StringReplace($lString, "_", " ")

        ; Starting with the first character of the string
        $lCpt = 1

        ; "=XX" search and convert loop
        While 1

            ; There will be 0 characters to convert in that block unless...
            $lConvertableLenght = 0

            ; That character, and another one 3 bytes over... and the next, and the next...
            For $lCpt2 = 0 to 100

                ; Is equal to "="
                If StringMid($lString, $lCpt+($lCpt2*3), 1) == "=" Then

                    ; In that case, yes, we will have to convert 3 more characters
                    $lConvertableLenght += 3
                Else

                    ; But if we fail to find or reach the end of a block of encoded characters, exit the search
                    ExitLoop
                EndIf
            Next

            ; If we did in fact find some encoded characters
            If $lConvertableLenght > 0 Then

                ; Extract that block of encoded characters
                $lConvertableString = StringMid($lString, $lCpt, $lConvertableLenght)

                ; Convert it
                $lConvertedString = BinaryToString("0x" & StringReplace($lConvertableString, "=", ""), $lEncoding)

                ; Replace it in the original
                $lString = StringReplace($lString, $lConvertableString, $lConvertedString)
            EndIf

            ; Increment the "=XX" search and convert loop counter
            $lCpt += 1

            ; If we reached the end of the string, exit the "=XX" search and convert loop
            If $lCpt >= StringLen($lString) Then ExitLoop

        ; Continue searching in the "=XX" search and convert loop
        WEnd

    ; For Base64 encoded strings
    Else

        ; Use the separate Base64Decode function
        $lString = Base64Decode($lString)
        $lString = BinaryToString($lString, $lEncoding)
    EndIf

    return($lString)
EndFunc

Func Base64Decode($lEncoded)

    ; Create the base64 encoding table
    Dim $Base64EncodingTable[0]
    For $Cpt = Asc("A") to Asc("Z")
        _ArrayAdd($Base64EncodingTable, Chr($Cpt))
    Next
    For $Cpt = Asc("a") to Asc("z")
        _ArrayAdd($Base64EncodingTable, Chr($Cpt))
    Next
    For $Cpt = Asc("0") to Asc("9")
        _ArrayAdd($Base64EncodingTable, Chr($Cpt))
    Next
    _ArrayAdd($Base64EncodingTable, "+")
    _ArrayAdd($Base64EncodingTable, "/")

    ; Start from the first character
    $Cpt = 1
    $Decoded = "0x"

    Do
        ; Extract the 4 characters to encode
        $Char1 = StringMid($lEncoded, $Cpt, 1)
        $Char2 = StringMid($lEncoded, $Cpt+1, 1)
        $Char3 = StringMid($lEncoded, $Cpt+2, 1)
        $Char4 = StringMid($lEncoded, $Cpt+3, 1)

        ; Decode them
        $Decoded &= Hex(BitShift(_ArraySearch($Base64EncodingTable, $Char1, 0, 0, 1), -2) + BitShift(BitAnd(_ArraySearch($Base64EncodingTable, $Char2, 0, 0, 1), 48), 4), 2)
        If $Char3 <> "=" Then $Decoded &= Hex(BitShift(BitAnd(_ArraySearch($Base64EncodingTable, $Char2, 0, 0, 1), 15), -4) + BitShift(BitAnd(_ArraySearch($Base64EncodingTable, $Char3, 0, 0, 1), 60), 2), 2)
        If $Char4 <> "=" Then $Decoded &= Hex(BitShift(BitAnd(_ArraySearch($Base64EncodingTable, $Char3, 0, 0, 1), 3), -6) + _ArraySearch($Base64EncodingTable, $Char4, 0, 0, 1), 2)

        ; Increment the counter
        $Cpt += 4

    ; Do this until all the encoded string has been decoded
    Until $Cpt >= StringLen($lEncoded)

    return($Decoded)
EndFunc

 

Last thing... I may update it into a better format for you, like a standalone telnet program with GUI. It is my telnet options negociations loops. The basic concept is systematically deny all request for special options and keep it "raw".

If server says "Will", I reply "Don't". If it says "Do", I reply "Wont"... unless it's the terminal type subnegociation, in which case I reply xterm!

$Data for now needs to be Global. You still need to know what you're doing, opening sockets and making a basic communication loop or something.

expandcollapse popup

Global $T_Is = Chr(0)
Global $T_Send = Chr(1)
Global $T_TerminalType = Chr(24)
Global $T_SE = Chr(240)
Global $T_SB = Chr(250)
Global $T_Will = Chr(251)
Global $T_Wont = Chr(252)
Global $T_Do = Chr(253)
Global $T_Dont = Chr(254)
Global $T_IAC = Chr(255)

Func NegotiateTelnetOptions()
    $NegotiationCommandsToSendBack = ""
    While StringInStr($Data, $T_IAC)
        $IACPosition = StringInStr($Data, $T_IAC)
        Switch StringMid($Data, $IACPosition+1, 1)
            Case $T_Will
                $NegotiationCommandsToSendBack &= CraftReply_CleanUpData($IACPosition, $T_Dont)
            Case $T_Do
                If StringMid($Data, $IACPosition+2, 1) = $T_TerminalType Then
                    $NegotiationCommandsToSendBack &= CraftReply_CleanUpData($IACPosition, $T_Will)
                Else
                    $NegotiationCommandsToSendBack &= CraftReply_CleanUpData($IACPosition, $T_Wont)
                EndIf
            Case $T_SB
                If StringMid($Data, $IACPosition, 6) = ($T_IAC & $T_SB & $T_TerminalType & $T_Send & $T_IAC & $T_SE) Then
                    $NegotiationCommandsToSendBack &= $T_IAC & $T_SB & $T_TerminalType & $T_Is & "xterm" & $T_IAC & $T_SE
                    $Data = StringReplace($Data, StringMid($Data, $IACPosition, 6), "")
                Else
                    MsgBox(0, "", "Unknown Subnegotiation...") ; Should never happen.
                    Exit
                EndIf
        EndSwitch
    WEnd
    Return $NegotiationCommandsToSendBack
EndFunc

Func CraftReply_CleanUpData($IACPosition, $Reply)
    $PartialCommandToSendBack = $T_IAC & $Reply & StringMid($Data, $IACPosition+2, 1)
    $Data = StringReplace($Data, StringMid($Data, $IACPosition, 3), "")
    Return $PartialCommandToSendBack
EndFunc