Help scraping data from webpage

[wisem2540]

I am appealing to my fellow autoit'ers for help.  I have tried to parse the sourcecode and the data that I am looking for appears to not be in plain text.  

 

I am looking for all of the relevant data, Hashrate, Last Round Reward and so on,  I apologize for not posting an example code but this is beyond my skill level with autoit.

here is the site.

 

https://eggpool.net/index.php?miner=55713816bf48b85cc1b03db521c0142f99402925d05e726b476a67a4&action=miner&submit=Show

[jdelaney]

I'm not a fan of clicking links.   Send F12 at your browser, select the spy tool, click what you want to grab. ..copy the html, and paste it back here.   Or, wait for someone that doesn't mind clicking links from strangers

[wisem2540]

Thanks for the reply.  As I said, the text I want is not in plain view.  Its a query of some sort.  All the website is, is a Cryptomining stats page.  I have close to 400 posts here.  I am hardly a stranger and  I would not direct this community to some place malicious

[jdelaney]

Not sure what to tell you...for a website to display anything requires html source, and that can be manipulated/read...even if it's just a file's text that is being displayed, you can still grab it, and do a regularexpression to parse out your data..  I'll defer to another that will work with the external links.

[wisem2540]

@jdelaney

Here is a C&P from the Miner Info section I am most interested in.  I hope this helps

 

 

</div>
	<div class="col-lg-3 col-xs-12">
	<div class="box box-info">
	<div class="box-header">
	<h3 class="box-title">Miner info</h3>
	</div>
	<!-- /.box-header -->
	<div class="box-body no-padding">
	<table class="table table-striped">
	<tr><td>Min. Payout</td><td><span id="minp">50</span></td><td>BIS</td></tr>
	<tr><td>Avg HR this round</td><td><span id="hr2">??</span></td><td>GH/s</td></tr>
	<tr><td>Avg HR last round</td><td><span id="lasthr">??</span></td><td>GH/s</td></tr>
	<tr><td>Reward last round</td><td><span id="immature2">??</span></td><td>BIS</td></tr>
	<tr><td>Est. round reward</td><td>(soon)</td><td>BIS</td></tr>
	<tr><td>Est. BIS/Day</td><td>(soon)</td><td>BIS</td></tr>
	<tr><td>Total Paid</td><td><span id="totalpaid">??</span></td><td>BIS</td></tr>
	</table>
	</div>
	<!-- /.box-body -->
	</div>
	<!-- /.box -->
	<!-- /div>

[jdelaney]

Perfect, so it's a table that has lots of data...use this one:

_IETableWriteToArray

Look at the helpfile, try to get it working, and then you can loop through the returned array, and grab your data.

Helpfile example:

; Open a browser with the table example, get a reference to the first table
; on the page (index 0) and read its contents into a 2-D array

#include <Array.au3>
#include <IE.au3>

Local $oIE = _IE_Example("table")
Local $oTable = _IETableGetCollection($oIE, 0)
Local $aTableData = _IETableWriteToArray($oTable)

_ArrayDisplay($aTableData)

_IEQuit($oIE)

Example of looping through all the tables:

#include <Array.au3>
#include <IE.au3>

$oIE = _IECreate("https://eggpool.net/index.php?miner=55713816bf48b85cc1b03db521c0142f99402925d05e726b476a67a4&action=miner&submit=Show")
$oTables = _IETableGetCollection($oIE)
For $oTable In $oTables
    $aTableData = _IETableWriteToArray($oTable)
    _ArrayDisplay($aTableData)
Next

 

Edited February 2, 2018 by jdelaney

[wisem2540]

@jdelaney

 

Holy crap that worked!  I will admit I had my doubts, but I think that is exactly what I needed.

 

Thanks!!

[jdelaney]

A little less generic a script:

$sHTML = '<div class="box-body no-padding">     <table class="table table-striped">     <tr><td>Min. Payout</td><td><span id="minp">50</span></td><td>BIS</td></tr>     <tr><td>Avg HR this round</td><td><span id="hr2">??</span></td><td>GH/s</td></tr>   <tr><td>Avg HR last round</td><td><span id="lasthr">??</span></td><td>GH/s</td></tr>    <tr><td>Reward last round</td><td><span id="immature2">??</span></td><td>BIS</td></tr>  <tr><td>Est. round reward</td><td>(soon)</td><td>BIS</td></tr>  <tr><td>Est. BIS/Day</td><td>(soon)</td><td>BIS</td></tr>   <tr><td>Total Paid</td><td><span id="totalpaid">??</span></td><td>BIS</td></tr>     </table>    </div>'

#include <Array.au3>
#include <IE.au3>

$oIE = _IECreate("")
_IEBodyWriteHTML($oIE,$sHTML)
$oTables = _IETableGetCollection($oIE)
For $oTable In $oTables
    $aTableData = _IETableWriteToArray($oTable,True)
    For $i = 0 To UBound($aTableData)-1
        If $aTableData[$i][0] = "Min. Payout" Then
            ConsoleWrite("Minimum payout = " & $aTableData[$i][1] & " " & $aTableData[$i][2] & @CRLF)
        EndIf
    Next
    _ArrayDisplay($aTableData)
Next

You can add all the if statements you want.  Also, I'm lazy, so didn't do any error handling.

Edited February 2, 2018 by jdelaney