Jump to content

read xpath and WinHttp.WinHttpRequest.5.1

marcoauto
 Share
Go to solution Solved by marcoauto,

Recommended Posts

marcoauto Wayfarer

marcoauto

Posted
Posted

hello, first of all I hope to explain myself well because I don't speak English very well 🙂

I need to monitor a signal and the serve I have has no API to read

So I tried to interface and something I managed to get.
For the moment I can log in and get answered on some calls.

One example of a request I make is the following: 

$oHTTP.Open("GET", $url&"/dynamic_load/0", False)
$oHTTP.Send()
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status
$oUpdates = json_decode($oReceived)
ConsoleWrite("channelUid:"&$oUpdates.channelUid & @crlf)
ecc ecc

I'm just missing the last value to read, but it's also the most important one. The problem is that the system does not return that value to me, so I should go and read directly on the page.
But in the page to read I can't see the table I need, maybe because it's dynamic and inside a div?

Debugging with chrome I can find the full path, but I don't understand how to read it.

I'd also like it not to open the browser to be able to read the value.

This is the script I've made for now: 

Global Const $HTTP_STATUS_OK = 200
Global $user = "myuser"
Global $password = "mypassword"

Global $url = "http://myremoteurl"

Global $oHTTP = ObjCreate("WinHttp.WinHttpRequest.5.1")

# Connect to remote site with user and password. This function works but 
$oHTTP.Open("POST", $url&"/login", False)
$oHTTP.SetRequestHeader("Accept", "application/json")
$oHTTP.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
$oHTTP.Send('username='&$user&'&password='&$password)
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status
Consolewrite($oReceived & @crlf)

And this works, I'm logged

The result is: 

<!DOCTYPE html>
<html>
    <head>
    
        <title>ENCODER</title>
    <link rel="stylesheet" type="text/css" media="screen" href="/assets/stylesheets/main.css">
    <link rel="stylesheet" type="text/css" media="screen" href="/assets/stylesheets/jquery-ui.min.css">
    <link rel="stylesheet" type="text/css" media="screen" href="/assets/stylesheets/jquery-ui-timepicker-addon.css">
        <link rel="shortcut icon" type="image/png" href="/assets/images/favicon.png">
        <script src="/assets/javascripts/jquery-3.1.1.js"></script>
        <script src="/assets/javascripts/jquery-ui.min.js"></script>
        <script src="/assets/javascripts/jquery-ui-timepicker-addon.js"></script>
        <script src="/assets/javascripts/jquery-swapsies.js"</script>
        <script src="/assets/javascripts/dash.all.min.js"></script>
        <script src="/assets/javascripts/main.js"></script>
        <style>
            video { width: 640px; height: 360px; }
        .ui-tabs-vertical { width: 55em; }
        .ui-tabs-vertical .ui-tabs-nav { padding: .2em .1em .2em .2em; float: left; width: 12em; }
        .ui-tabs-vertical .ui-tabs-nav li { clear: left; width: 100%; border-bottom-width: 1px !important; border-right-width: 0 !important; margin: 0 -1px .2em 0; }
        .ui-tabs-vertical .ui-tabs-nav li a { display:block; }
        .ui-tabs-vertical .ui-tabs-nav li.ui-tabs-active { padding-bottom: 0; padding-right: .1em; border-right-width: 1px; }
        .ui-tabs-vertical .ui-tabs-panel { padding: 1em; float: right; width: 40em;}
    </style>
        <script>
            $(function() {
                $( "#pwd-dialog" ).dialog(
                    { modal: true, 
                        autoOpen: false,
                      buttons: {
                          Submit: function() {
                              // validate
                              var pwdcurr = $("#pwd-curr").val().trim();
                              var pwdnew = $("#pwd-new").val().trim();
                              var pwdconfirm = $("#pwd-confirm").val().trim();

                              if (pwdcurr.length == 0 || pwdnew.length == 0)
                                  $("#pwd-msg").html("Password cannot be blank");
                              else if (pwdcurr == pwdnew)
                                  $("#pwd-msg").html("Cannot reuse current password");
                              else if (pwdnew != pwdconfirm)
                                  $("#pwd-msg").html("Password confirmation does not match");
                              else {
                                  $("#pwd-msg").html("");
                                  var send_data = $('#pwdForm').serialize();
                                    $.ajax({
                                        type: "POST",
                                            cache: false,
                                          url: '/password',
                                          data: send_data,
                                          datatype: 'text',
                                          success: function(data) {
                                              if (data == 'success')
                                                    $( "#pwd-dialog" ).dialog( "close" );
                                              else
                                                  $("#pwd-msg").html(data);
                                          }
                                        });     
                              }
                          },
                            Cancel: function() {
                            $( this ).dialog( "close" );
                          }
                        }
                    });
                $( "#pwd-link" ).click(function() {
                $( "#pwd-dialog" ).dialog( "open" );
                });
            });
        </script>
    </head>
  <body>
    <header>

      <dl id="user">
        <dt>admin</dt>
          <dd>
            <a id="pwd-link" href="#">Change Password</a>
            <a href="/logout">Logout</a>
          </dd>
      </dl>
    </header>
    <fullsection id="main">
    <div id="pwd-dialog" title="Change Password">
             
<form action="/password" method="POST" id="pwdForm">
    
        <table>
            <tr>
                <td>Current</td>
                <td><input type="password" id="pwd-curr" name="pwd-curr"/></td>
            </tr>
            <tr>
                <td>New</td>
                <td><input type="password" id="pwd-new" name="pwd-new"/></td>
            </tr>
            <tr>
                <td>Confirm New</td>
                <td><input type="password" id="pwd-confirm" name="pwd-confirm"/></td>
            </tr>
            <tr>
              <td colspan="2">
                    <span id="pwd-msg" style="color: red; font-size: small">
                        &nbsp;
                    </span>
                </td>
            </tr>
        </table>
        
</form>

    </div>
    
    
<script>
    var inputHomeRefreshTask = null;
    var outputHomeRefreshTask = null;
    var encoderRefreshTask = null;
    
    var filearr = [];
  function displayMessage(msgTitle, msgContent) {
      if (msgContent.length > 0) {
          $('<div></div>').dialog({
                modal: true,
                title: msgTitle,
                width: 450,
                height: 300,
                open: function() {
                  $(this).html(msgContent);
                },
                buttons: {
                  Ok: function() {
                    $( this ).dialog( "close" );
                  }
                }   
            });  //end confirm dialog
            return false;
      }
      else
          return true;
  }
  
  function updateElement(elName, elVal) {
    var el = $("#" + elName);
    
    // Update element if it exists
    if (el.length)
        el.html(elVal);
  }
  
  function updateEncoderStatus(faultArr, faultElName) {
        var faultMsg = "";
        
        for (idx in faultArr) {
            var msg = faultArr[idx];
    
            if (msg.length > 0) {
                    faultMsg += (msg + "<br/>");
            }
        }
        
        if (faultMsg.length == 0)
            faultMsg += "No Fault detected";
            
        $("#" + faultElName).html(faultMsg);
  }
    
  function refreshEncoderParams() {
        $.ajax({
            type: "GET",
                cache: false,
              url: "/encoderstatus_load",
              success: function(respObj) {
                updateEncoderStatus(respObj.systemFaultMsg, "encVideoStatus");
                updateEncoderStatus(respObj.inputFaultMsg, "encAudioStatus");
                updateEncoderStatus(respObj.outputFaultMsg, "encOutputStatus");
              }
        });     
  }

    function saveData(form_data, dest_url, cb) {
        //console.log(form_data);
        
        $.ajax({
            type: "POST",
                cache: false,
              url: dest_url,
              data: form_data,
              datatype: 'text',
              success: function(data) {
                  if (data == 'success') {
                        displayMessage("Success", 'Data saved successfully');

                        if (cb && typeof cb === "function")
                        cb(data);
                    }
                  else
                      if(data)
                        displayMessage("Error", data);
                      else  
                        displayMessage("Error", 'Unable to save data');
                       
              }
            });     
    }

    // Send data with callback (cb) after completion
    function sendData(form_data, dest_url, cb) {
        var token =  $('input[name="csrfToken"]').attr('value')
        console.log(form_data);
        $.ajaxSetup({
        beforeSend: function(xhr) {
            xhr.setRequestHeader('Csrf-Token', token);
        }
        });
        $.ajax({
            type: "POST",
                cache: false,
              url: dest_url,
              data: form_data,
              datatype: 'text',
              success: function(data) {
                  if (data != 'success')
                      displayMessage("Error", data);
                      
                    if (cb && typeof cb === "function") {
                    cb(data);
                  }
                }
        });
    }

    // Show or hide Select Options based on requested value
    function updateSelectOption(optElName, optShowIdxArr) {
      if ($("#" + optElName).length) {
        var optAvail = false;
        var selOpt = $("#" + optElName).val();
        
            $("#" + optElName + " option").each(function(idx) {
                if ($.inArray(idx, optShowIdxArr) > -1) {
                    $(this).show();
                    if (selOpt == idx) optAvail = true;
                }
                else
                    $(this).hide();
            });
            
            // Default to first shown one if previously selected
            // option is no longer available
            if ((optShowIdxArr.length > 0) && !optAvail)
                $("#" + optElName)[0].selectedIndex = optShowIdxArr[0];
        }
    }
    
    function initiateVideoRefresh() {
        
            // Do nothing for now
                
    }
    
    function initiateEncoderRefresh() {
        
        refreshEncoderParams();

        // Refresh selective parameters every 1 seconds
        encoderRefreshTask = setInterval(refreshEncoderParams, 1000);
        
    }

    function sleep(ms) {
        return new Promise(resolve => setTimeout(resolve,ms));
    }

    function savePageIndex(pageIndex) {
        $.ajax({
            type: "GET",
                cache: false,
              url: "/save_page_index/" + pageIndex,
              success: function(respObj) {
                // Do Nothing
              }
        });
    }

    $(function() {
        $("#tabs").tabs({

            
                active: 0,
            

            create: function( event, ui ) {
                initiateVideoRefresh();
                if (0 == "4") {
                    initiateEncoderRefresh();
                }
            },
            beforeLoad: function(event, ui) {
              ui.jqXHR.fail(function() {
                ui.panel.html(
                    "Couldn't load this tab." );
              });
            },
            activate: function( event, ui ) {
                if ((ui.oldTab.index() == 0) && ( (inputHomeRefreshTask != null) || (outputHomeRefreshTask != null) )) {
                    // Stop refresh on leaving Home tab
                    clearInterval(inputHomeRefreshTask);
                    clearInterval(outputHomeRefreshTask);
                } else if ((ui.oldTab.index() == 0) && (encoderRefreshTask != null)) 
                    // Stop refresh on leaving Encoder Status tab
                    clearInterval(encoderRefreshTask);
                
                if (ui.newTab.index() == 0)
                    // Start refresh on entering Home tab
                    initiateVideoRefresh();
                else if (ui.newTab.index() == 4)
                    // Start refresh on entering Encoder Status tab
                    initiateEncoderRefresh();

                    savePageIndex(ui.newTab.index());
            },
            'min-height': '600px',
        'overflow': 'auto'
        }).css({
        'min-height': '650px',
        'overflow': 'auto'
        });
        
      
                
        

        $("#tab-encoder").tabs().css({
        'min-height': '570px',
        'overflow': 'auto'
        });
    });
</script>

<div id="selectFilesForm">
</div>       

    <article>
    <div id="tabs">
            <ul>
              <li><a href="/home_load" title="homeTab">Home</a></li>
                
                <li><a href="/output_load" title="tsoipTab">Configuration</a></li>
                <li><a href="/config_load" title="configTab">Save/Load</a></li>
              
              <li><a href="/control_load" title="controlTab">Control/Network</a></li>
              <li><a href="#tab-encoder" title="encoderTab">Status</a></li>
              <li><a href="/admin_load" title="adminTab">Admin</a></li>
            </ul>
            <div id="tab-encoder">
                <ul>
                    <li><a href="#tab2-encoder-current">Current</a></li>
                  <li><a href="/converterstatushistory_load" title="encodeHistoryTab">Historical</a></li>
                </ul>
                <div id="tab2-encoder-current">
                    <table align="center" valign="top" width="100%">
                      <tr>
                            <td><b>System Status</b></td>
                            <td><b>Input Status</b></td>
                            <td><b>Output Status</b></td>
                      </tr>
                      <tr valign="top">
                            <td id="encVideoStatus"></td>
                            <td id="encAudioStatus"></td>
                            <td id="encOutputStatus"></td>
                      </tr>
                    </table>
                </div>
            </div>
        </div>
    </article>



    </fullsection>
  </body>
</html>

but I need a value that I don't see (see image below):XPATH.thumb.png.bd407afe7fd76f9de5b55a9d2226b4d7.png

the references extrapolated from chrome are these: 

element: <td id="home_srtOutputConnectionStatus">Connected</td>
selector: #home_srtOutputConnectionStatus
js path: document.querySelector("#home_srtOutputConnectionStatus")
xpath: //*[@id="home_srtOutputConnectionStatus"]
full xpath: /html/body/fullsection/article/div/div[1]/div/table[1]/tbody/tr/td[2]/div/table/tbody/tr[4]/td[2]

I also tried: 

$strHTLM = _INetGetSource($url)
If @error Then
    ConsoleWrite("_INetGetSource ERR: " & @error & @CRLF)
    Exit
Else
    ConsoleWrite($strHTLM & @CRLF)
EndIf

but it doesn't works because it need autentication and I searched the forum for how to read full xpaths directly but I didn't understand if it can be done without opening the browser.
Some attempts I report here but it is clearly wrong. 

$oHTTP.Open("POST", $url&"/login", False)
$oHTTP.SetRequestHeader("Accept", "application/json")
$oHTTP.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
$oHTTP.Send('username='&$user&'&password='&$password)
$oHTTP.Open("GET", $url&"/html/body/fullsection/article/div/div[1]/div/table[1]/tbody/tr/td[2]/div/table/tbody/tr[4]/td[2]", False)
$oHTTP.SetRequestHeader("Accept", "application/json")
$oHTTP.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
$oHTTP.Send()

I also tried command line curl but I couldn't even log in to the site

Does anyone have any advice for me? Thanks a lot and sorry for the very long post

Marco

Link to comment
Share on other sites
marcoauto Wayfarer

marcoauto

Posted
  • Author
Posted
4 minutes ago, Andreik said:

You are aware that we can't help you much since we don't have the credentials for this API. You can try something like this, maybe it works. 

$sData = InetRead('https://user:password@www.somesite.com', 1)

MsgBox(0, '', BinaryToString($sData))

 

Hi, 

unfortunately the user:password@url method doesn't work with this server. You just have to pass them via http setrequestheader or insert them in the form of the login page

thanks

Marco

Link to comment
Share on other sites
Andreik Universalist

Andreik

Posted
Posted

It's the body in the binary format but I think it's the same as BodyText so it might not be very helpful. How did you get the source code from your image? It's pretty different from what you get with WinHttpRequest. Use Postman or some similar app and tweak your headers / body keys until you get the same source code and then you can send the same request via WinHttpRequest.

When the words fail... music speaks.

Link to comment
Share on other sites
  • Solution
marcoauto Wayfarer

marcoauto

Posted
  • Author
  • Solution
Posted

Solved!

I managed to get all the html it generates returned by changing the "SetRequestHeaders" and therefore also the type of response that the server provides me. Obviously now I have to redo all the response parsing work (it no longer responds in json) but I can see everything and it's what I needed.
I report below the code that I changed in case it is also useful to someone else

Original: 

$oHTTP.Open("GET", $url&"/dynamic_load/0", False)
$oHTTP.SetRequestHeader("Accept", "application/json")
$oHTTP.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
$oHTTP.Send()
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status
Consolewrite($oReceived & @crlf)

Modified: 

$oHTTP.Open("GET", $url&"/dynamic_load/0", False)
$oHTTP.SetRequestHeader("Accept", "text/html, application/xhtml+xml, image/jxr, */*")
$oHTTP.SetRequestHeader("Accept-Encoding", "zip, deflate, text/html, application/xhtml+xml, image/jxr, */*")
$oHTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko")
$oHTTP.SetRequestHeader("Referer", $url)
$oHTTP.Send()
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status
Consolewrite($oReceived & @crlf)

Thanks

Marco

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...