how to pass a multi-line data value or JSON with curl?

[gcue]

Hello.

Instead of trying to pass a multi-line value.  I am trying to save the contents to a txt file and pass it through - but i think the quotes are messing up the --data value in both cases.  i also tried --data-raw, --data-ascii but neither worked - any help is greatly appreciated!!!

expandcollapse popup

$iPID = Run(@ComSpec & ' /c curl.exe --header "accept:application/json" --data "{
    'documentTypes': [
        '=EMAIL='
    ],
    'query': {
        'type': 'AND',
        'constraints': [
            {
                'type': 'FIELD_COMPARISON',
                'field': 'SENT_DATE',
                'operator': 'RANGE',
                'range': {
                    'min': '2023-03-01T00:00:00',
                    'max': '2023-03-17T23:59:59'
                }
            },
            {
                'type': 'AND',
                'constraints': [
                    {
                        'type': 'FIELD_COMPARISON',
                        'customField': 'DOC_TYPE',
                        'field': 'CUSTOM_SORTABLE_STRING',
                        'value': '=EMAIL=',
                        'operator': 'EQUAL_TO'
                    },
                    {
                        'type': 'FIELD_COMPARISON',
                        'customField': 'CONTENT-TYPE',
                        'field': 'CUSTOM_SORTABLE_STRING',
                        'value': 'SM',
                        'operator': 'MATCHES'
                    }
                ]
            }
        ]
    },
    'queryOptions': {
        'source': 'LEGACY',
        'type': 'COUNT',
        'suppressQueryTranslationWarning': true
    },
    'resultColumns': []
}"

$txt_file = "file.txt"
$txt_query = FileRead($txt_file) 
$iPID = Run(@ComSpec & ' /c curl.exe --header "accept:application/json" --data "' & $txt_query & '"', @SystemDir, @SW_HIDE, $STDERR_CHILD + $STDOUT_CHILD)

 

Edited March 23, 2023 by gcue

[mistersquirrle]

I suppose the first question I have to ask is, does this command work manually?

Secondly, have you tried converting the JSON to a single line, with a tool like: https://codebeautify.org/jsonminifier

That site outputs: 

{"documentTypes":["=EMAIL="],"query":{"type":"AND","constraints":[{"type":"FIELD_COMPARISON","field":"SENT_DATE","operator":"RANGE","range":{"min":"2023-03-01T00:00:00","max":"2023-03-17T23:59:59"}},{"type":"AND","constraints":[{"type":"FIELD_COMPARISON","customField":"DOC_TYPE","field":"CUSTOM_SORTABLE_STRING","value":"=EMAIL=","operator":"EQUAL_TO"},{"type":"FIELD_COMPARISON","customField":"CONTENT-TYPE","field":"CUSTOM_SORTABLE_STRING","value":"SM","operator":"MATCHES"}]}]},"queryOptions":{"source":"LEGACY","type":"COUNT","suppressQueryTranslationWarning":true},"resultColumns":[]}

It also mentions that your JSON is not valid (because of the single quotes), which may mean that you need to escape the JSON quotes in your command. Check out this page about escaping the quotes: https://mkyong.com/web/curl-post-json-data-on-windows/

 

Edited March 24, 2023 by mistersquirrle

[gcue]

that worked!  hardcoding the json AND replacing the file contents like this...

$txt_file = "file.txt"
$txt_query = FileRead($txt_file)
$txt_query = StringStripWS($txt_query, 8)
$txt_query = StringReplace($txt_query, '"', '\"')

$iPID = Run(@ComSpec & ' /c curl.exe --header "accept:application/json" --data "' & $txt_query & '"', @SystemDir, @SW_HIDE, $STDERR_CHILD + $STDOUT_CHILD)

thank youuuuu!!!!

[gcue]

hmm thought it was working.  looks like it isn't.  getting wrong value responses (no errors though).  getting 0 for all queries when expecting counts > 0.  queries work when running them manually.  i am properly authenticated bc i am able to go through the process manually with the same parameter values (ie: access token, client id, etc)

As mentioned, I've tried

$txt_query = StringStripWS($txt_query, 8)
$txt_query = StringReplace($txt_query, '"', '\"')

I've also tried putting the query around single quotes and without.

ive also tried the Best JSON minifier to compress the code to 1 line..

nothing works 😩

Edited March 27, 2023 by gcue

[TheXman]

1. That is obviously not an accurate representation of your cURL command line -- at least I hope it isn't.    It doesn't even have a target API URL.  So without an accurate representation (even if you have to redact the URL), how is anyone suppose to know if it is correct.
2. Your initial example is setting the "Accept" header.  Why aren't you setting the "Content-Type" header also.  The "Content-Type" header, which should be "application/json", says that the request data should be interpreted as JSON.  It may not be required, but it is still good practice to define the content type.
3. Your Run(), at least as it is shown, does not require you to prepend @comspec.
4. If you already have valid JSON in a file, then just use cURL's @ syntax to have it read in the file. (see example below)  If you need more information about cURL parameters, you can find them in the online cURL documentation.
5. You haven't provided a complete script.  So, again, there's no way to know what other issues you may have in your script.

The example below, sends a POST request to the postman echo server.  The echo server sends back information about your request as it was received by the server (including the request headers)  The "temp.json" file that is referenced contains your JSON dataset.  As you can see in the echo server's response, the cURL command successfully sent the request.  The "data" field shows the POST data that was received.  The "JSON" field shows JSON because the Content-Type header was sent so it knew that the data was JSON.  Also in the example's command line you will see a "--json" parameter.  You can use that parameter if you are using cURL version 7.82.0 or newer.  It's a shortcut that does 3 things: passes the json data or file, sets the "Content-Type" header to application/json, and sets the "Accept" header to application/json.

You should be able to adapt the example to fit your needs.

Example:

#include <Constants.au3>

Const $CURL_EXE  = "c:\utils\curl\curl64.exe"      ;Modify path as needed
Const $JSON_FILE = "temp.json"                     ;Modify path as needed
Const $API_URL   = "https://postman-echo.com/post" ;Modify URL as needed

example()

Func example()
    Local $sCmdLine = ""
    Local $iPid     = 0

    ;Build command line
    $sCmdLine = StringFormat('"%s" -s --json @"%s" "%s"', $CURL_EXE, $JSON_FILE, $API_URL)
    ConsoleWrite("Cmd Line = " & $sCmdLine & @CRLF)

    ;Execute command line
    $iPid = Run($sCmdLine, "", @SW_HIDE, $STDERR_MERGED)
    If Not $iPid Then Exit MsgBox($MB_ICONERROR, "Run Error", "Check to make sure your command line is valid.")

    ;Wait for the process to close
    ProcessWaitClose($iPid)

    ;Display command output
    ConsoleWrite(@CRLF & "Response:" & @CRLF)
    ConsoleWrite(StdoutRead($iPid) & @CRLF)
EndFunc

Console output:

expandcollapse popup

Cmd Line = "c:\utils\curl\curl64.exe" -s --json @"temp.json" "https://postman-echo.com/post"

Response:
{
  "args": {},
  "data": {
    "documentTypes": [
      "=EMAIL="
    ],
    "query": {
      "type": "AND",
      "constraints": [
        {
          "type": "FIELD_COMPARISON",
          "field": "SENT_DATE",
          "operator": "RANGE",
          "range": {
            "min": "2023-03-01T00:00:00",
            "max": "2023-03-17T23:59:59"
          }
        },
        {
          "type": "AND",
          "constraints": [
            {
              "type": "FIELD_COMPARISON",
              "customField": "DOC_TYPE",
              "field": "CUSTOM_SORTABLE_STRING",
              "value": "=EMAIL=",
              "operator": "EQUAL_TO"
            },
            {
              "type": "FIELD_COMPARISON",
              "customField": "CONTENT-TYPE",
              "field": "CUSTOM_SORTABLE_STRING",
              "value": "SM",
              "operator": "MATCHES"
            }
          ]
        }
      ]
    },
    "queryOptions": {
      "source": "LEGACY",
      "type": "COUNT",
      "suppressQueryTranslationWarning": true
    },
    "resultColumns": []
  },
  "files": {},
  "form": {},
  "headers": {
    "x-forwarded-proto": "https",
    "x-forwarded-port": "443",
    "host": "postman-echo.com",
    "x-amzn-trace-id": "Root=1-6422223a-2c910a151f0d3bd901a5f748",
    "content-length": "1170",
    "user-agent": "curl/8.0.1",
    "content-type": "application/json",
    "accept": "application/json"
  },
  "json": {
    "documentTypes": [
      "=EMAIL="
    ],
    "query": {
      "type": "AND",
      "constraints": [
        {
          "type": "FIELD_COMPARISON",
          "field": "SENT_DATE",
          "operator": "RANGE",
          "range": {
            "min": "2023-03-01T00:00:00",
            "max": "2023-03-17T23:59:59"
          }
        },
        {
          "type": "AND",
          "constraints": [
            {
              "type": "FIELD_COMPARISON",
              "customField": "DOC_TYPE",
              "field": "CUSTOM_SORTABLE_STRING",
              "value": "=EMAIL=",
              "operator": "EQUAL_TO"
            },
            {
              "type": "FIELD_COMPARISON",
              "customField": "CONTENT-TYPE",
              "field": "CUSTOM_SORTABLE_STRING",
              "value": "SM",
              "operator": "MATCHES"
            }
          ]
        }
      ]
    },
    "queryOptions": {
      "source": "LEGACY",
      "type": "COUNT",
      "suppressQueryTranslationWarning": true
    },
    "resultColumns": []
  },
  "url": "https://postman-echo.com/post"
}

 

Edited March 28, 2023 by TheXman

[gcue]

thank you very helpful!

i didnt post all the info because of confidential reasons.  didnt want to post a dummy url for instance

thanks again!