Jump to content

Recommended Posts

Posted

Hello all,

I've scripted AutoIT to scrape a web forum I'm part of using the IE.au3 #include.  The script works great if I'm logged in, but doesn't work as a service.  Anyone have any ideas on how I might achieve that or will I have to have the computer logged in at all times?

Thanks,

-John

  • Moderators
Posted
4 minutes ago, ChipConnJohn said:

Anyone have any ideas on how I might achieve that or will I have to have the computer logged in at all times?

@ChipConnJohn We can't offer much in the way of suggesting ways to improve your script without seeing the script ;)

"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Posted (edited)
15 minutes ago, JLogan3o13 said:

@ChipConnJohn We can't offer much in the way of suggesting ways to improve your script without seeing the script ;)

True that...

It appears to be a problem with creating the IE object, but I could be wrong.  I put in a little bit of logging and when running as a service, the log doesn't get written to, so the script isn't running correctly right out of the gate.

#include <IE.au3>
#include <MsgBoxConstants.au3>
#include <Array.au3>
#include <String.au3>
#include <file.au3>
#include <MailSender.au3>
;#include <ErrorHandler.au3>
const $GSusername = "test@gmail.com"
const $GSPassword = "+++"
const $GSGroupURL = "http://groupspaces.com/"
Global $sEmailBody
Global $sLogFile = "c:\zadmin\NewTopicEmailerLog.txt"

WriteLog($sLogFile, "NewTopicEmailer Service Started."&@CRLF)

; Open Groupspaces.com
Global $oIE = _IECreate ("http://www.groupspaces.com")
If @error <> 0 Then
    WriteLog($sLogFile,"IECreate failed with error "&@error)
    MsgBox($MB_SYSTEMMODAL,"IECreate Object failed.","Error number is: "&@error)
EndIf


$iTimer = TimerInit()
While 1
    ;MsgBox($MB_SYSTEMMODAL,"Timerdiff",TimerDiff($iTimer))
    If TimerDiff($iTimer) >= 60*60*1000 Then  ; This loops the script every HOUR
    WriteLog($sLogFile,"Running scrape.")
    ;Reset the timer
    $iTimer = TimerInit()
    ; Build the HTML of the email body
    $sEmailBody = "<html>" & @CRLF & "<head>" &@CRLF& "<title>New Topic on  Coaches Forum!</title>" &@CRLF& "</head>" &@CRLF& "<body>" &@CRLF
    $sEmailBody = $sEmailBody & "<table>" &@CRLF& "<tr>" &@CRLF& "<td>" &@CRLF& "<strong>New post in the Coaches Forum!</strong><br><br>" &@CRLF& "</td>" &@CRLF& "</tr>" &@CRLF
    ;MsgBox($MB_SYSTEMMODAL,"email body", $sEmailBody)

    Local $iFoundOne = 0    ; Marks that Just, minute, or minutes was found
    Local $sTopicName = ""
    Local $sCatName = ""
    Local $bHaveEmailtoSend = False
    ;Variables for EmailList
    Global $aEmailList[1]
    Global $intCount = 0
    Global $intRow = 0
    Global $intLineCount = 0
    local $rc

    _IELinkClickByText($oIE, "Sign in")

    Local $username = _IEGetObjByName($oIE, "username")
    Local $password = _IEGetObjByName($oIE, "password")
    Local $button = _IEGetObjByName($oIE, "Submit")

    _IEFormElementSetValue($username, $GSusername)
    _IEFormElementSetValue($password, $GSPassword)
    _IEAction($button, "click")
    _IELoadWait($oIE, 500)
    ;_IELinkClickByText($oIE, "View group")
    ;_IELinkClickByText($oIE, "Forum")
    _IENavigate($oIE,$GSGroupURL)

    Global $oLinksCategoryPage = _IELinkGetCollection($oIE)
    If @error <> 0 Then
        msgbox($MB_SYSTEMMODAL,"Error with Category Links object", "The error is: " & @error)
    EndIf

    Local $i = 1
    Global $aCatLinks[$i] = [0]                                       ;Array to hold the Category Links

    For $oLink In $oLinksCategoryPage
        If StringInStr($oLink.href,"forum_id") Then
            ;This is a category. Save this link in the array.
            ;MsgBox($MB_SYSTEMMODAL,"Array index before increment ",$i)
            $i = $i + 1
            redim $aCatLinks[$i]
            $aCatLinks[0] = $i-1                                         ; Set first element in array to total number of elements
            $aCatLinks[$i-1] = $oLink.href
        EndIf
    Next

    For $i = 1 to $aCatLinks[0]
        ;msgbox($MB_SYSTEMMODAL,"$i","$i = "& $i)
        ;    _ArrayDisplay($aCatLinks)
        _IENavigate($oIE,$aCatLinks[$i])
        ;Now at the thread level. Navigate into each thread and look at timestamp
        Global $oLinksOfThreads = _IELinkGetCollection($oIE)
        If @error <> 0 Then
            msgbox($MB_SYSTEMMODAL,"Error with Thread Links object", "The error is: " & @error)
        EndIf
        Local $j = 1
        Global $aThreadLinks[$j] = [0]
        For $oLinkThread In $oLinksOfThreads
            ;MsgBox($MB_SYSTEMMODAL,"Total thread links", "Times through "& $iTimesThrough & " out of total." & @extended)
            If StringInStr($oLinkThread.href,"thread_id") Then
                ;MsgBox($MB_SYSTEMMODAL,"The thread link is ",$oLinkThread.href)
                $j = $j + 1
                redim $aThreadLinks[$j]
                $aThreadLinks[0] = $j-1
                $aThreadLinks[$j-1] = $oLinkThread.href
            EndIf
        Next
        For $j = 1 to $aThreadLinks[0]
            ;msgbox($MB_SYSTEMMODAL,"$j","$j = "& $j)
            ;_ArrayDisplay($aThreadLinks)
            _IENavigate($oIE, $aThreadLinks[$j])
            ;Need to find instance of Posted and read text after the href.
            Local $sBodyText = _IEBodyReadText($oIE)
            ;Find first instance of the word "Posted" and check the two words to the right of it.
            Local $aArrayOfBodyText = _StringExplode($sBodyText," ",0)
            $sTopicName = ""
            $sCatName = ""
            ;_ArrayDisplay($aArrayOfBodyText, "The body text in Array form")
            ;Local $iIndexOfPosted = _ArraySearch($aArrayOfBodyText,"days")
            $iFirstPosted = 1
            For $iIndex = 0 to ubound($aArrayOfBodyText) - 1
                ;MsgBox($MB_SYSTEMMODAL,"Array pieces",StringLower($aArrayOfBodyText[$iIndex]))

                If StringInStr(StringLower($aArrayOfBodyText[$iIndex]), "posted") Then
                    ;If this is the first time Posted has been found, grab the TopicName in case it's needed.
                    If $iFirstPosted = 1 Then
                        $iFirstPosted = 0
                        ;Build topic name
                        ;_ArrayDisplay($aArrayOfBodyText)
                        $iStep = 0
                        While $aArrayOfBodyText[$iIndex - $iStep] <> ""
                            $iStep =  $iStep + 1
                            ;Add backwards.
                            $sTopicName = $aArrayOfBodyText[$iIndex - $iStep] & " " & $sTopicName
                        WEnd
                        ;Get me back to "Search"
                        While $aArrayOfBodyText[$iIndex - $iStep] <> "Search"
                            $iStep = $iStep + 1
                        WEnd
                        While $aArrayOfBodyText[$iIndex - $iStep] <> ">"
                            $iStep = $iStep + 1
                        WEnd
                        $iStep = $iStep + 1
                        ;_ArrayDisplay($aArrayOfBodyText)
                        While $aArrayOfBodyText[$iIndex - $iStep] <> ">"
                            $sCatName = StringStripWS($aArrayOfBodyText[$iIndex - $iStep], $STR_STRIPALL) & " " & $sCatName
                            ;msgbox($MB_SYSTEMMODAL,"CatName",":"&$sCatName&":")
                            If $sCatName = "ForumForum Index " Then
                                $sCatName = "Uncategorized"
                                ;msgbox($MB_SYSTEMMODAL,"CatName",":"&$sCatName&":")
                                ExitLoop
                            EndIf
                            $iStep = $iStep + 1
                        WEnd
                        ;MsgBox($MB_SYSTEMMODAL,"Topic name","Topic Name "&$sTopicName & " Category Name "&$sCatName)
                        ;The word Posted has been found. Look for "Just A moment ago", "minutes", and "minute"
                        ;msgbox($MB_SYSTEMMODAL,"Posted found","Found "&$aArrayOfBodyText[$iIndex]&" "&$aArrayOfBodyText[$iIndex+1]&" "&$aArrayOfBodyText[$iIndex+2]&" ")
                        ; I need to create an email that holds appropriate information.
                        ; I have the thread link stored in the $aThreadLink array.  That's easy.
                        ; It would be good to have the Thread Title and Category Title
                        ; I need to test this system with me and Scott for awhile before releasing this on everyone.
                        Select
                            Case StringLower($aArrayOfBodyText[$iIndex +1]) = "just"
                                ;"Just a moment ago.  Grab this one.
                                ;MsgBox($MB_SYSTEMMODAL, "Just found","yup and ThreadLinks[$j] = "& $aThreadLinks[$j])
                                ;Set a FoundIt variable
                                $iFoundOne = 1
                            Case Stringlower($aArrayOfBodyText[$iIndex +2]) = "minute"
                                ;"1 minute". Grab this one
                                ;msgbox($MB_SYSTEMMODAL,"Minute found","yup and ThreadLinks[$j] = "& $aThreadLinks[$j])
                                ;_ArrayDisplay($aArrayOfBodyText, "The body text in Array form")
                                $iFoundOne = 1
                            Case Stringlower($aArrayOfBodyText[$iIndex +2]) = "minutes"
                                ;"x minutes. Grab this one
                                ;msgbox($MB_SYSTEMMODAL,"minutes found","yup and ThreadLinks[$j] = "& $aThreadLinks[$j])
                                ;_ArrayDisplay($aArrayOfBodyText, "The body text in Array form")
                                $iFoundOne = 1
                        EndSelect
                        If $iFoundOne = 1 Then
                            $sEmailBody = $sEmailBody & "<tr>" &@CRLF& "<td>"  &@CRLF& "<br><br>" &@CRLF
                            $sEmailBody = $sEmailBody & " <strong>Category: " & $sCatName & "</strong>&nbsp;&nbsp;"
                            $sEmailBody = $sEmailBody & " <strong>Topic: " & $sTopicName & "</strong>&nbsp;&nbsp;<br>" & $aThreadLinks[$j]
                            $sEmailBody = $sEmailBody & "<br><br><br></td>" &@CRLF& "</tr>" & @CRLF
                            ;msgbox($MB_SYSTEMMODAL,"Email body after adding Topic name in Found One section",$sEmailBody)
                            $bHaveEmailtoSend = True
                            $iFoundOne = 0
                        EndIf  ; Found One
                    EndIf
                EndIf
                ;Need to back out to Threads in Category
            Next
            ;msgbox($MB_SYSTEMMODAL,"End of threads","going to next thread")
            ;_IEAction($oIE,"back")
        Next
        ;Back out to Category List
        ;msgbox($MB_SYSTEMMODAL,"End of Category","going to next Category")
        ;_IEAction($oIE,"back")
    Next
    $sTimestamp = "Sent at: " & @HOUR & ":" & @MIN & ":" & @SEC & " on " & @MON & "/" & @MDAY & "/" & @YEAR &@CRLF
    $sEmailBody = $sEmailBody & "<tr>" &@CRLF& "<td>" & "<br>" & $sTimestamp
    $sEmailBody = $sEmailBody &@CRLF& "</td>" &@CRLF& "</tr>" &@CRLF
    $sEmailBody = $sEmailBody & "</table>" & @CRLF & "</body>" & @CRLF & "</html>"
    ;msgbox($MB_SYSTEMMODAL,"Email body", $sEmailBody &" and the HaveEmailToSend " & $bHaveEmailtoSend)
    ;_IEQuit($oIE)
    If $bHaveEmailtoSend Then
        ;I have an email to send
        ; I should save the email addresses in a text file
        ; Read text file into array
        WriteLog($sLogFile,"Sending email.")
        $sEmailSubject = "New GroupSpaces Post"
        $rc = _FileReadToArray("C:\zadmin\EmailList.csv", $aEmailList, Default)
        if $rc <> 1 Then
            msgbox($MB_SYSTEMMODAL, "Error reading file", $rc)
        EndIf
        $intLineCount = _FileCountLines("C:\zadmin\EmailList.csv") - 1
        ;_ArrayDisplay($aEmailList)
        While $intCount <= $intLineCount
            $intCount = $intCount + 1
            $sTheEmailAddress = $aEmailList[$intCount]
            _SendMail($sEmailBody, $sTheEmailAddress, $sEmailSubject)
        WEnd
    Else
        ;Happens if there is no email to send.
    EndIf
; This line runs once per hour. If the @HOUR is 3pm it sends the heartbeat email.
If @HOUR <> 30 Then ;Limits sending heartbeat to once a day.
    WriteLog($sLogFile,"Sending Heartbeat email")
    _SendMail("Heartbeat. System is up. "&$sTimestamp, "test@gmail.com", "Forum Scrape Heartbeat")
EndIf
Else
    ;This happens when the hour hasn't been reached yet.
EndIf
sleep(10)
WEnd

Func WriteLog($sLFile, $sTextToWrite)
    ;Open the file for writing (append to the end of a file) and store the handle to a variable.
    Local $hFileOpen = FileOpen($sLFile, $FO_APPEND)
    If $hFileOpen = -1 Then
        MsgBox($MB_SYSTEMMODAL, "", "An error occurred whilst writing the temporary file.")
        Return False
    EndIf

    ; Write data to the file using the handle returned by FileOpen.
    FileWrite($hFileOpen, @HOUR&":"&@MIN&":"&@SEC&" on "&@MON&"/"&@MDAY&"/"&@YEAR&@TAB&$sTextToWrite&@CRLF)

    ; Close the handle returned by FileOpen.
    FileClose($hFileOpen)
EndFunc

 

Edited by JLogan3o13
Posted

I'm not sure if  a service can interact with window objects (stuff like _IEAction($button, "click" (I don't think so). you can try to run it from task scheduler. There you can tick the option "Run whether user is logged on or not".

 

 

 

Posted

Here's the solution I had to go with.

I had to set a workstation to autologin.  There was no way to get the script to work right if the machine wasn't at a desktop.  I have this running on a Win10 machine that's connected to a domain.  I used these registry mods to do the autologin.

HKLM\Software\Microsoft\Windows NT\CurrentVersion\winlogon

AutoAdminLogon = 1 (String Value Key) (0 means off, 1 means automatic)
DefaultUserName = Username (String Value Key)
DefaultPassword = Password (String Value Key)
DefaultDomainName = domain.com (String Value Key) (Only needed if this computer has joined a domain)

Reboot

Note: To bypass AutoAdminLogon, hold down the Shift key as Windows 8 initializes.

-John

 

Posted

How are you creating a service for this script?  It doesn't appear you are creating the service within the script, so how are you implementing the script as a service?  Normally, you would be right in that a service wouldn't be able to interact with the desktop.  However, if using the Non-sucking-service-manager, there is a configuration which allows for the created service to do just that.

8LE3hXJ.png

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...