HTTP/HTML Post vs. Get that returns XML

[orange]

I am writing a small tool for work. I need to submit a form to an off-site server that is either POST or GET (depending on query length), and parse through a returned XML file. Here are my problems:

At my company, all browsers except IE 6 are blocked.

Using IE.au3, I can open and properly submit the form via post or get, but I cannot parse the returned XML since IE blocks that.

Using http.au3, I can spoof an IE useragent string, but cannot connect to the server at all. No @errors nor @extended errors returned either. I believe our firewall is blocking me. Following the examples on the http.au3 host thread, I cannot get POST or GET to work.

Here are my questions:

If I use IE, is there a way to dowload the returned XML?

If I use TCP (via http.au3), is there a way to correctly connect with the server that I am missing?

Here is a POST header recorded with Fetch:

POST ******* HTTP/1.1

Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*

Accept-Language: en-us

Content-Type: application/x-www-form-urlencoded

Accept-Encoding: gzip, deflate

User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2; .NET CLR 3.5.21022; MS-RTC LM 8; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

Host: *******

Content-Length: *******

Proxy-Connection: Keep-Alive

Pragma: no-cache

******* query string *******

This is driving me nuts. Please, any input helps.

[Zedna]

Look here

#708691

http://www.autoitscript.com/forum/index....hp?showtopic=19848&hl=XMLDomWrapper&st=0

maybe it will help you

Edited July 21, 2009 by Zedna

[orange]

Look here

#708691

http://www.autoitscript.com/forum/index....hp?showtopic=19848&hl=XMLDomWrapper&st=0

maybe it will help you

Thanks for the reply, but that doesn't answer my question. Parsing the XML isn't the issue - the issue is that by using IE I cannot view, edit, or otherwise RETRIEVE the XML. That's the problem with IE. Try opening XML in IE via viewsource or saveas, and you'll get an error "IE cannot open this file" or somthing like that. Save As also does not work.

This is not unique to my machine either - I'm not asking "whats wrong with this IE?!!?!"

This is a feature (documented on msdn) that I'm trying to get around.

Edited July 21, 2009 by orange

[Zedna]

So try to use this code from above mentioned post

#include <File.au3>

Global $oXML = ObjCreate("Microsoft.XMLHTTP")
$oXML.Open("GET", "http://www.wowarmory.com/character-sheet.xml?r=Anub%27arak&n=Lovepenguin", 0)
$oXML.Send

Global $sFile = _TempFile(@TempDir, '~', '.xml')
FileWrite($sFile, $oXML.responseText)

instead of using IE

[Authenticity]

Try this:

Global $oXML = ObjCreate('Microsoft.XMLHTTP')

$oXML.Open("POST", "http://www.site.com/file.xml", 0, 'username', 'password')
$oXML.Send
ConsoleWrite($oXML.responseText & @CRLF)

Edit: Heh Zedna to the rescue

Edited July 21, 2009 by Authenticity

[orange]

Apologies to the first reply post - I read through too quickly.

I'll try these suggestions out and post back the result. Thanks.

[orange]

Thank you thank you thank you!!!! Everything works great!!!!

Again. apologies to Zedna - I misunderstood your response.

[ManelRodero]

x Authenticity:

Hello,

If I use your code for getting an XML file (using ObjCreate("Microsoft.XMLHTTP") and $oXML.Open("GET", $channel_url, 0)) like this one:

<?xml version="1.0" encoding="ISO-8859-1"?>
<Event
        SUMMARY = "Prueba Manel - No es una reserva real3"
        DESCRIPTION = "Prueba Manel"
        X-PONENT = "Manel Blánquez"
        PUBLISHSERVER = "wmserver"
        PUBLISHINGPOINT = "sala-actes"
/>

I get the following:

X-PONENT = "Manel Bl?uez"

I've lost some chars !! Why? If I use WGET or IE for getting the XML file, then I get all the chars!

Any idea?

Thank you!

[ManelRodero]

Hello again,

I've found some tips ...

If I change the XML to UTF-8 and the encoding to:

<?xml version="1.0" encoding="UTF-8"?>

then, this code works getting all the chars correctly (i.e. X-PONENT = "Manel Blánquez"):

Local $oXML = ObjCreate("Msxml2.XMLHTTP.4.0")
$oXML.Open("GET", $channel_url, 0)
$oXML.Send
ConsoleWrite($oXML.responseText)

but then, I can't parse it using _XMLGetAttrib!. I get -1 errors :-( ... (I suppose XML Parser can't use UTF-8 but only ISO-8859-1).

So, the question is: Can I have AutoIt use/get ISO-8859-1 properly?

Thanks!

[Zedna]

You can use _StringBetween() instead of _XMLGetAttrib() as workaround.

I used it too for my Czech (non English) code page.

[ManelRodero]

You can use _StringBetween() instead of _XMLGetAttrib() as workaround.

I used it too for my Czech (non English) code page.

Hello,

Sorry for this month delay but I've been on holidays.

I've seen your answer about using _StringBetween but this is some kind of low level manipulation, isn't it? And in this manner I would need to use some regular expressions to search for the values of an attribute. How can I get the value for X-PONENT using _StringBetween? And, is there a solution for the original _XMLGetAttrib function?

Thank you very much!

[Zedna]

Something like this:

$value = _StringBetween($oXML.responseText, 'X-PONENT = "', '"')
If @error Then
 $value = ''
Else
 $value = $value[0]
EndIf

[ManelRodero]

Hi everybody,

I've solved the problem with special chars being converted to "?" when using Microsoft.XMLHTTP.

The problem that the XML output was generated using PHP (in ISO-8859-1) and the encoding specified in the XML header (<?xml version="1.0" encoding="iso-8859-1"?> ...

BUT ...

the web server doesn't specify the charset. Now we added the charset next to the content type and all works perfectly. I can get the file using Microsoft libraries and the parse the content using _XML functions ;-)

HTTP/1.x 200 OK
Date: Thu, 03 Sep 2009 14:27:33 GMT
Server: Apache/2.2.6 (Win32) DAV/2 mod_ssl/2.2.6 OpenSSL/0.9.8e mod_autoindex_color PHP/5.2.4
X-Powered-By: PHP/5.2.4
P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
Content-Encoding: gzip
Vary: Accept-Encoding
Content-Length: 304
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: application/xml; charset=ISO-8859-1

Thank you very much!

[sapper1971]

So try to use this code from above mentioned post

#include <File.au3>

Global $oXML = ObjCreate("Microsoft.XMLHTTP")
$oXML.Open("GET", "http://www.wowarmory.com/character-sheet.xml?r=Anub%27arak&n=Lovepenguin", 0)
$oXML.Send

Global $sFile = _TempFile(@TempDir, '~', '.xml')
FileWrite($sFile, $oXML.responseText)

instead of using IE

Hi sorry to dig up an old post but this has solved a problem i had, the only issue I have now is it doesn't overwrite the file if it exists already it just tags it on the end, looking through the help files it appears there is no flag to allow you using the FileWrite function to overwrite a file.

Is there an easy way of using similar code to overwrite the file if it already exists and to create it if it doesn't?

I am using it to control 5 seperate files some of which are checking a live market so are updating / overwriting every second or so and others are simple one off functions that just need to update once before you view the info.

I have thought of maybe adding a delete function before actually writing the file so there is never one but this seems a bit of a messy way of doing it any ideas anyone.

This is my first attempt using autoit and i'm liking it so far my project is nearly finished and Ive managed to solve most things just by looking through the forums. _XMLDOMWRAPPER is hurting my brain though so I may have to ask for guidance at some point but hopefully I can muddle through it :-)

[Zedna]

Is there an easy way of using similar code to overwrite the file if it already exists and to create it if it doesn't?

I have thought of maybe adding a delete function before actually writing the file so there is never one but this seems a bit of a messy way of doing it any ideas anyone.

Yes it's very simple and I reccomend this way for you.

You can also use FileOpen/FileWrite/FileClose and in FileOpen specify overwrite flag but it's more code lines.

[sapper1971]

Yes it's very simple and I reccomend this way for you.

You can also use FileOpen/FileWrite/FileClose and in FileOpen specify overwrite flag but it's more code lines.

Ok thankyou I will go read up on them :-)

[sapper1971]

Yes it's very simple and I reccomend this way for you.

You can also use FileOpen/FileWrite/FileClose and in FileOpen specify overwrite flag but it's more code lines.

Perfect thankyou and like you said simple lol..

[Zedna]

Perfect thankyou and like you said simple lol..

I thought FileDelete + FileWrite (your original way) is simple. I use that too instead of FileOpen/FileWrite/FileClose when I want less lines of code.