INetGet not applying filters in google search

[adamchapman]

I'm trying to download the search results at https://www.google.co.uk/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws

 

My code looks like this:

#include <Array.au3>
#include <Inet.au3>

Local $sKeyword = 'HSBC'
Local $sSource = _INetGetSource('https://www.google.co.uk/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws')

ConsoleWrite ( $sSource)

Now if I copy the output text from ConsoleWrite() and save as an html file, opening that in the browser shows different search results from using the first url directly in the browser.

If anybody can find the issue I'd be very grateful. I'm very inexperienced with AutoIt so apologies if looks silly.

Edited June 19, 2013 by adamchapman

[JohnOne]

Try losing the session stuff, and just using "https://www.google.co.uk/search?q=HSBC"

[adamchapman]

That's the problem, I need the date filter applied:

'&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws'

[bogQ]

winhttp.winhttprequest.5.1 whud not work on google so i guess this can

 

$data = 'https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws'
$XML = ObjCreate("Microsoft.XMLHTTP")
$XML.Open("GET", $data, 0)
$XML.Send()
$sSource = $XML.Responsetext()
ConsoleWrite($sSource)

whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better... Edited June 19, 2013 by bogQ

[FireFox]

whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better...

There must be a google search API somewhere...

Edited June 19, 2013 by FireFox

[bogQ]

probably but if this works i dont see why shud he use google search API, advice is not to open it in browser after it save it to see end result, only coz source from the code will automatically redirect browser to original page instead to his saved source.

<div style="display:block">Please click <a href="https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr:1,cd_min:15/06/2013,cd_max:16/06/2013&tbm=nws&gbv=1&sei=Ba_BUafRBuaI4AT81YDgCQ">here</a> if you are not redirected within a few seconds.

No other reason. Edited June 19, 2013 by bogQ

[FireFox]

Fine.

[adamchapman]

Thanks to all of you.

bogQ, your solution was superb. Very impressive

[adamchapman]

winhttp.winhttprequest.5.1 whud not work on google so i guess this can

 

$data = 'https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws'
$XML = ObjCreate("Microsoft.XMLHTTP")
$XML.Open("GET", $data, 0)
$XML.Send()
$sSource = $XML.Responsetext()
ConsoleWrite($sSource)

whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better...

 

 

Is there a way to use a proxy server with this method?

I've managed to set up an http proxy server today ('?do=embed' frameborder='0' data-embedContent>>), but am having a similar problem as before, where the data returned through code is not the same as the data displayed through a browser. I hope this XML method would fix the issue as it did before

[bogQ]

I would not know if it's possible, I never had any need to use proxy on google before. either way reading from that link I don't like what are you trying to do

I did mention that winhttp.winhttprequest.5.1 will not work on google (but don't know why did you try to use it in that link) and larger automation on google site will lead to image verification that is needed to confirm that your not bot, if you are you talking about that quota limits I would suggest you to read forum rules - Part about: "Bypassing of security measures"

[adamchapman]

Ok thanks for taking the time to have a look anyway.