Working Gmail (SSL) POP3 script

[Fox2]

This is a working generic set of SSL POP3 functions, ready for use. It's got a lot left to do on it, see below. But anyone should be able to take this, and use this to collect Gmail or other encrypted email via POP3 as needed without extra work.

Full information and .au3 below, and credits to mikeytown (for his original script), catchfish for this post which basically solved the main technical issue surrounding use of the usual OpenSSL.exe, this version of the POP3 protocol, and this and this descriptions of how to analyze emails themselves.

The rest of this post is background info and what I think needs doing next, so that anyone who wants to build this and existing unencrypted UDF's into a full POP3/SMTP UDF, can basically do so. I'll be doing so anyway but others might get there sooner. Please do. I might not get to it!

Fox2

----

Background:

A general POP3/SMTP email UDF requires ability to handle four things: - unencrypted SMTP and POP3, and encrypted ("SSL") versions of each (as used for gmail). To date AutoIt has unencrypted SMTP/POP3, encrypted (gmail) SMTP, but encrypted POP3 has been elusive. There's been a couple of proof of concepts, but no actual, usable, script to handle gmail collection.

What this does:

It creates commands such as these:

• _POPConnect($OpenSSL_Executable, $SmtpServer, $SmtpPort, $EmailUsername, $EmailPassword)
• _POPSendAndGet($Command, $IsMultiline)
• _POPPurgeBuffer() (shouldn't be needed usually)
• _POPCheckStderr() (checks stderr; shouldn't be needed usually)
• _POPDisconnect()

A typical session might be:

_POPConnect($OpenSSL_Executable, $SmtpServer, $SmtpPort, $EmailUsername, $EmailPassword)
If @error then .....

$list = _POPSendAndGet("LIST"); get a list of all emails
If @error then .....

$text = _POPSendAndGet("RETR 1", 1); get email 1
If @error then
    ....
else 
    $Results = _POPAnalyze($text)
    _ArrayDisplay($Results)
endif

_POPSendAndGet("QUIT")

_POPDisconnect()

That simple.

The internal structure of the attached code should be blindingly obvious. I haven't done 100% error checking, and it's still got loads of debug code "DebugReport()", but it's commented and should be simple to understand and seems 100% reliable. It handles both long emails, and emails with "." in the start of lines, hasn't given me an error yet, and has fairly careful checking of server responses to identify when its okay, when to poll more, and so on.

It uses the cygwin compiled version of OpenSSL. It's identical in function to the usual version, but the usual version seems to have a bug that means the first input must be via keyboard, requiring input blocking, an open command window, etc. Messy!

Cygwin OpenSSL

You will need the cygwin DLL files which can be downloaded from their website and then unzipped. The compiled version of OpenSSL from OpenSSL.org (same source code, same usage) somehow has a bug in it, which means that particular version of OpenSSL.exe doesn't respond unless you send something via console/keyboard first... which as mikeytown's script shows, is messy to manage. The above post doesn't tell you completely what to do though. This is what I found out:

• You want to unzip and get hold of the /bin folder for cygwin openssl, using the download system on their website (containing typically openssl.exe, cygssl-0.9.8.dll, cygcrypto-0.9.8.dll, c_rehash). You dont need to actually install cygwin or keep the rest - just download and unzip the openssl part of cygwin, which is currently listed under "libraries" in their system.
• You will also need to find somewhere the file cygwin1.dll (google and download, eg here) and put that in the same folder if not included (openssl.exe uses this).
• You don't need your script in the same folder as these. It just needs a folder containing the above binaries, for use in the Run() command.

No installing is needed for this.

What's to come

A final POP3 UDF should have the following:

• _POPConnect (executable_path, address, port, username, password, is_ssl, OPTIONAL buffertimeout, OPTIONAL maintimeout) -- creates a connection, either normal or SSL. Reports if the connection is successful. Optional parameter = buffer timeout and main timeout for this connection. Returns a connection ID > 0 of the relevant process (allows >1 POP server in use). Buffertimeout is how long to allow to check nothing more's coming through the buffer for an ambiguous or malformed message (typically 1 - 4 seconds). Maintimeout is how long to allow before the server itself is deemed to have timed out, and may be considerably longer (20 - 120 seconds).
• _POPSend(connectionID, command) -- sends a command to the relevant connection. In the background it's remembered whether this is a secure or normal connection.
• _POPGetResponse(connectionID, OPTIONAL canbemultiline) -- gets a response from a connection. If the +OK response can be multiline (LIST, RETR, etc) then add canbemultiline = 1 to specify to expect a multiline response if successful.
• _POPSendAndGet(connectionID, command) -- combines both the above (too useful not to have). 'Canbemultiline' is figured out internally from the command.
• _POPSetTimeout(connectionID, buffertimeout, maintimeout) -- change timeouts. ConnectionID = -1 for all.
• _POPPurgeBuffer() -- waits till there's (eg) 2 clear seconds with no incoming text, to ensure the buffer is genuinely clear and (apparently) nothing more's being sent in the present transaction. Returns any text sent.
• _POPClose(connectionID) -- close a connection. ConnectionID = -1 for all.
• _POPAnalyze(string) -- takes an email raw text as the server provides it, and return an array with predefined arguments in predefined entries ($a[1] is the FROM, $a[2] = TO, ..., $a[19] = main text, $a[20] = number of attachments, $a[21] = first attachment in base 64, etc..... )
• _POPEncode64(text or file,flag) -- takes the raw text (or if flag=1, reads the file whose name is in $text), and uses OpenSSL.exe to encode it to base 64, and returns that as a string.
• _POPDecode64(text or file,flag) -- takes the base 64 text (or if flag=1, reads the file whose name is in $text), and uses OpenSSL.exe to decode it to base 64, and returns that as a string.

If anyone wants to have a start at creating the full version of the above, go for it; Ive done the main "SSL POP3" code which was my need, but it would be nice to have this redone by someone who really knew what they were doing, and made into a proper POP3 UDF.

[AU3 to be posted up shortly. Bug found in my handling of @error in the example scripts - specifically DebugReport() was overwriting @error before it could be tested. "Oops". The core functionality works though, and is reliable. Fixing that first.]

Edited February 11, 2008 by Fox2

[arcker]

seems amazing ! please post asap

[BananaFredSoft]

When are you going to post the script?

[Fox2]

Just polishing up the debug output.

Because POP can fail in interesting ways when its actually used in someone elses code, the one thing I want is good trace/debug ability. It may slow it down a fraction, but the payoff comes with anyone who can see whats gone on, and fix it, either when trying to collect email and something's wrong at a traffic, network, POP or SSL level etc, or if they accidentally misapply it (eg bad FUNC call) in their own code. Better to give people an ability to see what's happening under the hood.

Been away a lot, back on Monday though

Edited February 15, 2008 by Fox2

[Paider]

Hi Fox2,

what about the script?

Did you find the error?

Would be great to see the script.

A really nice idea.

[glangdon]

Hi Fox2,

I am extremely happy that you are pursuing the use of Apzo's POP3 functions in an SSL environment (your post of Jan 9, 2008), and eagerly await your new POP3 UDF!

I attempted to perform the cut and paste you suggested in your post of Feb 9, but, due to my lack of expertise, was unable to do it properly!

I also was unable to locate the downloadable OpenSSL module on the Cygwin site as you suggested in your Feb 10 post, so I did a full install of Cygwin plus the Cygwin OpenSSL library, collected the files you mentioned and would be happy to post a zip (1.4+ MB) of them here for anyone who needs them if only I knew how to do so!!

I wish I were competent enough to assist you in completing the UDF, but unfortunately I am not. Hopefully there are others who are able and willing to help.

Thanks for all your efforts!

[myspacee]

i'm waiting for this for years..

Thank you in advance for future release

m.

[greatbrains]

How is this project coming? I am also very interested in collecting my gmail into autoit.

Keep up the good work!

[Sticky]

Will this ever be released?

[DJKMan]

It seems that there are many people out there who could benefit from this.

When will you upload the script?

[BrewManNH]

The original poster hasn't posted anything on this site in over 3 years, he's probably not still around by this point.