Recursively on a site?

[reida]

Hello. How to go on site pages recursively?

[water]

The help file is your friend. Look for Loop Statements.

[reida]

I know as in a cycle to pass under references from a file. How to pass similarly 

curl -L

 

[water]

Can you please post what you have tried so far?

[reida]

Addresses of pages are in a file ааа.txt

$oURL="http://alpari.com/"
$oIE = _IECreate($oURL, 0, 0)
$oFile = FileOpen("aaa.txt")
While 1
    $oURL = FileReadLine($oFile)
    If @error Then ExitLoop
_IENavigate($oIE, $oURL)


;EndIf
Sleep(100)
WEnd
_IEQuit($oIE)
FileClose($oFile)

 

Edited August 23, 2018 by reida

[water]

Doesn't look bad.
What doesn't work? Do you get an error message, script crash ...?

[reida]

It works. But how to make work without a file aaa.txt?

[reida]

That the script has found addresses of pages and has visited them. For example: http://alpari.com/, http://alpari.com/page1, http://alpari.com/page2...

Edited August 23, 2018 by reida

[reida]

water. You have understood about what I ask?

Forgive for my English)

Edited August 23, 2018 by reida

[JLogan3o13]

@reida how about you show some patience? Standard forum etiquette is to wait 24 hour to give people time to respond (you waited a whole 16 minutes).

[reida]

Excuse, I am not assured that correctly has formulated the thought on script transitions on all pages of a site. Therefore has asked)

[water]

Shouldn't be too hard to code.
You have a fixed and a variable part (1, 2 ...) of the page you want to open. Increment a counter and combine it with the fixed part.

[reida]

Addresses of pages are unknown to me, I know only the address of the main page. How to make transition on all pages except .jpg .rar. zip?

[water]

Then it becomes quite complex as you have to analyze every page. 

But why would you want to open all pages of a site?

Edited August 24, 2018 by water

[reida]

To open pages serially and to check presence of some words or word-combinations. I wish to keep the found information in a file for the further analysis.

[Gianni]

given that using inprudently recursive functions can cause ramifications out of control, here a simple example of a recursive approach

#include <array.au3>
#include <ie.au3>
Global $sStartingUrl = "https://alpari.com/"
Global $oIE = _IECreate()
Global $aVisited[] = ['']
_GoToURL_rec($sStartingUrl)
MsgBox(0, "Debug", "Done.")
_IEQuit($oIE)
Exit

; WARNING: recursive function
Func _GoToURL_rec($sUrl)
    If _ArraySearch($aVisited, $sUrl) = -1 Then ; url not already visited
        _ArrayAdd($aVisited, $sUrl) ; mark as already visited
        _IENavigate($oIE, $sUrl)
        Local $oUrls = _IETagNameGetCollection($oIE, "A")
        If @extended Then
            ; MsgBox(0, "debug @extended", @extended & "links found")
            For $ohyperlink In $oUrls
                $sHref = $ohyperlink.href
                ; do not remove the following check or you'll lose on the web
                If StringLeft($sHref, 18) = "https://alpari.com" Then ; stay in the same domain only
                    ; furter checking on url here
                    If Not StringInStr($sHref, "#") Then ; do not open link on same page
                        If Not StringInStr($sHref, ".pdf") Then ; do not open pdf
                            ; ..... add more checks if needed
                            _GoToURL_rec($sHref) ; ---> recursion
                        EndIf
                    EndIf
                EndIf
            Next
        EndIf
    EndIf
    Return
EndFunc   ;==>_GoToURL_rec

 

Edited August 24, 2018 by Chimp

[reida]

@Chimp Thank you so much.

Edited August 25, 2018 by reida