cURL UDF - a UDF for transferring data with URL syntax

[seangriffin]

got same problem.

EDIT: was reading other examples and find that:

Dlls are 32bit. That means you have to run 32bit AutoIt when using them.

BTW is there any way to get content from a site that generate it from JS?

I'll update the top post to indicate Win 32-bit is a requirement thanks. Yes I only develop under 32-bit sorry, but my guess is if you downloadeded the libcurl 64-bit DLLs then it would work?

I'm no expert in cURL so not sure whether pulling dynamic JS is possible. I know the UDF is only simple right now, mainly as a solution to suit my own needs. Would love to put more effort into it.

[seangriffin]

V0.3 is now available.

The downloads on page 1 been updated.

New features include cURL multi-mode support. cURL_multi can be used in place of cURL_easy to avoid the issue where cURL easy mode blocks certain functions in Windows (such as opening a MS Office document).

[smartee]

...I'm finding easy mode is blocking certain activities in Windows 32-bit when it runs, and slows my computer such that I can't get much work done..

That shouldn't be.

In my experience, using Win7 32 and 64bit versions, the easy interface runs quite reliably (in harmony with MS office too), which OS are you using?

The poor performance is maybe because you download the entire internet resource into memory($curlout) with _WriteFunc() (even when saving to disk) so large files => massive memory usage.

Why not use the stream parameter, and write directly to file(when required), and so maintain a small buffer size, like the examples on libcurl's website?

With respect to the wrapper UDFs I have almost completed the easy interface (largely untested though) I will post it in a new thread in the coming week. (named libcurl UDF maybe? hope there's no confusion ).

I'm sure you'll see many ways to improve your functions using the libcurl UDF.

Now its just a matter of finding time to work on it

Edit: fixed typos

Edited March 31, 2012 by smartee

[seangriffin]

V0.4 is now available.

The downloads on page 1 been updated.

This version includes HTTP Header support.

[seangriffin]

That shouldn't be.

In my experience, using Win7 32 and 64bit versions, the easy interface runs quite reliably (in harmony with MS office too), which OS are you using?

The poor performance is maybe because you download the entire internet resource into memory($curlout) with _WriteFunc() (even when saving to disk) so large files => massive memory usage.

Perhaps, though the HTTP responses I get here in the office aren't big. You may not notice the delay unless you are executing cURL in rapid succession many times over. I am using it to make HTTP requests to a RESTful web service that responds in pages. I have to repeatedly execute the same requests collecting each page one at a time. I need to make over 100 requests each day to collect the complete dataset I need. If I switch to cURL multi-mode I don't experience any blocking effects in Win XP, however the requests take quite a bit longer to finish (from 2 hours using cURL easy to about 5 hours using cURL multi). If you can come up with a better implementation I can trial it for you on my OS.

I'm sure you'll see many ways to improve your functions using the libcurl UDF.

No doubt, I this put together in a tremendous rush as I desparately needed it at work.

Edited April 3, 2012 by seangriffin

[seangriffin]

V0.5 is now available.

The downloads on page one been updated.

Just a minor update to increase the maximum length of URLs to 65,536 characters.

[gcorfielduk]

Hi Sean,

great piece of work. I am using this extensively. Can you add some way to timeout. I know there is a LIBCURL timeout easyopt. I access remote equipment with your curl udf. If that equipment is down the default timeout is very long.

Thanks again

Greg

[stinson6016]

I need a little help, I found a php file that uses curl to pull the warranty info from a dell site. The php file uses a cookie to pass the service tag to dell and pulls the raw data back then processes that data. I'm still at the point of getting the cookie to pass to dell. I have tried several different things for the past several hours and can only seem to get this to work with php on linux; not autoit on windows

If any one has any ideas please let me know; I'm so frustrated with this right now.

#include <cURL.au3>
cURL_initialise()
$url = "http://www.dell.com/support/troubleshooting/us/en/04/TroubleShooting/Display_Warranty_Tab?name=TroubleShooting_WarrantyTab"
$cookie_file = @ScriptDir & "cookie.txt"
$out = cURL_easy($url, $cookie_file, 1, 0, "", "", 0, 1, 1)
; Output the HTML to the console
;~ ConsoleWrite($out)
MsgBox(0, "", $out)
; Clean up cURL
cURL_cleanup()

the cookie in php

$strCookie = "name=OLRProduct; OLRProduct=".$ServiceTag."|;";

[PHPWarner]

Can we use this cURL UDF for HTTP POST requests or only GET available?

[Digisoul]

I am just curious, what is the difference between WinHTTP and cURL ?

you can do almost all HTTP operations with WinHTTP

[Fire]

Very nice and usefull. Thanks a lot Men!

[DeltaRocked]

Hi Sean,

Am getting an error for the code.

#include "cURL.au3"

cURL_initialise()
$url = 'http://www.google.com'
$cookie_file = 'cookie.txt'
$cookie_action = 2
$output_type = 1 ;binary
$output_file = 'output.txt'
$postfields = ''
$ssl_verifypeer = 0
$noprogress = 0
$followlocation = 1
$http_headers = 'Content-Encoding,Content-Language,Content-Length,Content-Location,Content-Disposition,Content-Type'
cURL_easy($url, $cookie_file, $cookie_action, $output_type, $output_file, $postfields, $ssl_verifypeer, $noprogress, $followlocation, $http_headers)
cURL_cleanup()

Error:

>Running:(3.3.8.1):C:\Program Files\AutoIt3\autoit3.exe "C:\a\cURL_test.au3"
C:\a\cURL.au3 (124) : ==> Subscript used with non-Array variable.:
$hCurlHandle = $hCurlHandle[0]
$hCurlHandle = $hCurlHandle^ ERROR
->12:53:10 AutoIT3.exe ended.rc:1

[chakka]

Error!

cURL.au3(411,79) : ERROR: $hCurlHandle: undeclared global variable.

DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlHandle,

[Smiley43210]

Can anyone help? It says

C:\Users\******\AutoIt\UDF\cURL.au3(411,79) : ERROR: $hCurlHandle: undeclared global variable.

The line is

DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlHandle, "uint", $CURLOPT_HTTPHEADER, "ptr", $curl_slist_struct)

but i believe it should be

DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlEasyHandle, "uint", $CURLOPT_HTTPHEADER, "ptr", $curl_slist_struct)

because the previous lines have $hCurlEasyHandle and also since it is doing "curl_easy_setopt"

Also, when I run the example 1 code, it returns nothing but instead exits with exit code -1073741819. The script doesn't even finish, almost as if the dll made it exit. I tried adding a ConsoleWrite("Hi") at the end but it doesn't get printed. Thus, there's no way to check any @error variables or things like it. Any ideas?

Here's the code I ran:

#include 

; Initialise cURL
cURL_initialise()

; Return the HTML from "http://www.autoitscript.com/site"
$out = cURL_easy("http://www.autoitscript.com/site/")

; Output the HTML to the console
ConsoleWrite("-----BEGIN HTML-----"&@CRLF&$out&@CRLF&"-----END HTML-----")

; Clean up cURL
cURL_cleanup()

Output:

>"C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "C:\Users\*****\Downloads\AutoIt\cURL test.au3" /UserParams   
+>14:19:06 Starting AutoIt3Wrapper v.2.1.0.33    Environment(Language:0409  Keyboard:00000409  OS:WIN_7/Service Pack 1  CPU:X64 OS:X64)
>Running AU3Check (1.54.22.0)  from:C:\Program Files (x86)\AutoIt3
+>14:19:06 AU3Check ended.rc:0
>Running:(3.3.8.1):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\Users\*****\Downloads\AutoIt\cURL test.au3"   
--> Press Ctrl+Alt+F5 to Restart or Ctrl+Break to Stop
!>14:19:22 AutoIT3.exe ended.rc:-1073741819
>Exit code: -1073741819    Time: 18.048

Edited December 24, 2012 by Smiley43210

[AHRIMANSEFID]

Reading HTML from a webpage, and writing the reg File.

Hi.

Me Need Help For Read URL.com/text.txt and writing the Windows registry.

InetGet("http://www.test.com/test.txt", @TempDir & "\test.txt", 1, 1)

Thanks A lot.

Edited March 5, 2013 by AHRIMANSEFID

[cyphertron]

Thank you seangriffin, your cURL UDF is very useful.

Could someone let me know what is wrong with the below AutoIt Syntax?

$out = cURL_easy("https://docs.google.com/forms/d/162F4JLfnnU1U61Y_ZhIYb7yV_ZV2y65y6BAcR2N9IeM/formResponse",,,,,"entry.156901506='google@gmail.com'&draftResponse=[]&pageHistory=0")

I've created a sample form in Google docs and passing on email value.

[babullika]

Hey! 

Tring to use UDF like in example

#include <cURL.au3>

; Initialise cURL
cURL_initialise()

; Return the HTML from "http://www.autoitscript.com/site"
$out = cURL_easy("http://www.autoitscript.com/site/")

; Output the HTML to the console
ConsoleWrite($out)

; Clean up cURL
cURL_cleanup()

And getting 

+>21:19:54 Starting AutoIt3Wrapper v.2.1.3.0 SciTE v.3.3.6.0 ;  Keyboard:00000409  OS:WIN_XP/Service Pack 3  CPU:X64 OS:X86    Environment(Language:0419  Keyboard:00000409  OS:WIN_XP/Service Pack 3  CPU:X64 OS:X86)

>Running:(3.3.8.0):C:Program FilesAutoIt3autoit3.exe "C:Documents and SettingsAdminМои документыDropboxsmscurlcurl practice.au3"    

--> Press Ctrl+Alt+F5 to Restart or Ctrl+Break to Stop

!>21:19:56 AutoIt3.exe ended.rc:-1073741819

>Exit code: -1073741819    Time: 3.505

 

Well usng http://fossies.org/windows/www/curl-7.34.0-ssl-sspi-zlib-static-bin-w32.zip/ and after scipt said i need zlib downloaded it from http://www.zlib.net/ version 1.2.8. I guess i need another version of curl lib ? Can some one upload it plz..

Mine are here for tests - https://www.dropbox.com/s/bkrtxssj6qtbr22/curl.rar

 

UPD:

Last one was from ruiing on X, now after try it on Win 7 i got more information - 

Problem signature:

   Problem Event Name: APPCRASH

   Application Name: autoit3.exe

   Application Version: 3.3.10.2

   Application Timestamp: 52c196bd

   Name of the module with the error: StackHash_0a9e

   Version of the module with the error: 0.0.0.0

   The time stamp module with the error: 00000000

   Exception Code: c0000005

   Exception Offset: 00000115

   OS Version: 6.1.7601.2.1.0.256.1

   Language Code: 1049

   Additional Information 1: 0a9e

   Additional information 2: 0a9e372d3b4ad19135b953a78882e789

   Additional Information 3: 0a9e

   Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

 

and in autoit >Exit code: -1073741819  

Edited January 2, 2014 by babullika

[Subzero89]

Thanks for your script, but how i can display headers?

i have to get "Location: http://www.example.com" header.

thanks

Edited April 25, 2014 by Subzero89

[mschol]

i'm trying to use this UDF however openDLL cannot open libcurl.dll and the entire script fails

libcurl.dll/libeay32.dll/sslea32.dll exist in the working dir and in system32

i tried both 32 and 64bit versions of the files...

running windows 7 ultimate x64

[RickB75]

Hey guys. I'm trying to figure out how to use this UDF. Each time I run it using the sample script, I keep getting an error.

>Running AU3Check (1.54.22.0)  from:C:\Program Files\AutoIt3
C:\Documents and Settings\salespc2\My Documents\websiteScripts\addudfs\cURL.au3(411,79) : WARNING: $hCurlHandle: possibly used before declaration.
        DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlHandle,
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\Documents and Settings\salespc2\My Documents\websiteScripts\addudfs\cURL.au3(411,79) : ERROR: $hCurlHandle: undeclared global variable.
        DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlHandle,
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\curl-7.34.0-ssl-sspi-zlib-static-bin-w32\curltest.au3 - 1 error(s), 1 warning(s)
!>11:26:06 AU3Check ended. Press F4 to jump to next error.rc:2
>Exit code: 2    Time: 5.712

Am I doing something wrong? I placed the sample script in the DLL files folder like instructed.