Jump to content

Real-Time log file viewer


jguinch
 Share

Recommended Posts

Here is a small application made for my own need.
It allows you to follow the tail of a log file (or other txt file) in real time.
To avoid some memory issues, it loads about 10MB from the end of the file, so you can follow your huge log files in real-time with it.

image.png.b128f577832f0859101aeb501f824835.png

Here is the code :

#include <FileConstants.au3>
#include <GUIConstantsEx.au3>
#include <GuiEdit.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

Global $sAppTitle = "RT-LogFileViewer"
Global $iSizeToLoad = 10 * 1024 * 1024
Global $iBlockSize = 1024, $iPause = False
Global $sLastDirectory = RegRead("HKEY_CURRENT_USER\Software\" & $sAppTitle, "LastDirectory")
Global $iFileChange = 0, $sFileDate, $sLogFile, $iFileSize, $hGui
Global $idEdit, $idPause, $idSearch, $idFileName, $idProgress

If Not StringLen($sLastDirectory) Or Not FileExists($sLastDirectory) Then $sLastDirectory = @MyDocumentsDir
Opt("GUIOnEventMode", 1)
Opt("MustDeclareVars", 1)

_LoadGui()
If $CmdLine[0] Then _FileOpen($CmdLine[1])
ProcessWaitClose(@AutoItPID)


Func _LoadGui()
    Local $idOpen, $idAbout
    $hGui = GUICreate($sAppTitle, 800, 600, -1, -1, BitOR($WS_SYSMENU, $WS_SIZEBOX, $WS_MINIMIZEBOX, $WS_MAXIMIZEBOX) )
    $idEdit      = GUICtrlCreateEdit(""            , 10 , 50 , 780, 490, BitOr($ES_READONLY, $ES_AUTOVSCROLL, $WS_VSCROLL, $WS_HSCROLL, $ES_NOHIDESEL))
    $idOpen      = GUICtrlCreateButton(ChrW(0x2f)  , 10 , 10 , 30 , 30)
    $idPause     = GUICtrlCreateButton(ChrW(0x3b)  , 760, 10 , 30 , 30)
    $idSearch    = GUICtrlCreateButton(ChrW(0x2315), 720, 10 , 30 , 30)
    $idFileName  = GUICtrlCreateLabel (""          , 10 , 550, 680, 25)
    $idAbout     = GUICtrlCreateLabel ("About"     , 690, 550, 100, 25, $SS_RIGHT)
    $idProgress  = GUICtrlCreateProgress(10, 540, 780, 5)
    Local $aAccelKeys = [["^o", $idOpen], ["^p", $idPause], ["^f", $idSearch]]

    GUICtrlSetFont($idOpen    , 12, 400, 0, "Wingdings 2")
    GUICtrlSetFont($idPause   , 12, 400, 0, "Webdings")
    GUICtrlSetFont($idSearch  , 19, 800, 0, "Arial")
    GUICtrlSetFont($idEdit    , 10, 400, 0, "Courier")
    GUICtrlSetFont($idFileName, 9 , 400, 0, "Verdana")
    GUICtrlSetFont($idAbout   , 9 , 400, 4, "Verdana")
    GUICtrlSetTip($idOpen  , "Open file (CTRL + O)")
    GUICtrlSetTip($idSearch, "Search... (CTRL + F)")
    GUICtrlSetTip($idPause , "Pause monitoring (CTRL + P)")
    GUICtrlSendMsg($idEdit, $EM_LIMITTEXT, -1, 0)
    GUICtrlSetColor($idAbout, 0x0000ff)

    GUISetOnEvent    ($GUI_EVENT_CLOSE, "_Exit")
    GUICtrlSetOnEvent($idPause        , "_Pause")
    GUICtrlSetOnEvent($idSearch       , "_Search")
    GUICtrlSetOnEvent($idOpen         , "_FileOpenGUI")
    GUICtrlSetOnEvent($idAbout        , "_About")

    GUICtrlSetResizing($idOpen, $GUI_DOCKALL)
    GUICtrlSetResizing($idPause, $GUI_DOCKRIGHT + $GUI_DOCKTOP + $GUI_DOCKSIZE)
    GUICtrlSetResizing($idSearch, $GUI_DOCKRIGHT + $GUI_DOCKTOP + $GUI_DOCKSIZE)
    GUICtrlSetResizing($idEdit, $GUI_DOCKLEFT + $GUI_DOCKRIGHT + $GUI_DOCKTOP + $GUI_DOCKBOTTOM)
    GUICtrlSetResizing($idFileName, $GUI_DOCKLEFT + $GUI_DOCKRIGHT + $GUI_DOCKBOTTOM + $GUI_DOCKHEIGHT)
    GUICtrlSetResizing($idAbout, $GUI_DOCKRIGHT + $GUI_DOCKBOTTOM + $GUI_DOCKSIZE)
    GUICtrlSetResizing($idProgress, $GUI_DOCKLEFT + $GUI_DOCKRIGHT + $GUI_DOCKBOTTOM + $GUI_DOCKHEIGHT)
    GUICtrlSetState($idSearch, $GUI_DISABLE)
    GUICtrlSetState($idPause, $GUI_DISABLE)
    GUICtrlSetState($idProgress, $GUI_HIDE)
    GUISetAccelerators($aAccelKeys, $hGui)
    GUISetIcon(@SystemDir & "\SHELL32.dll", -25)
    GUISetState(@SW_SHOW)
EndFunc

Func _LoadTail($sLogFile)
    Local $sLines, $iPercent
    $iFileSize = FileGetSize($sLogFile)
    Local $hFile = FileOpen($sLogFile, $FO_READ)
    If @error Then Return 0
    $sFileDate = FileGetTime($sLogFile, 0, 1)

    Local $iStartPos =($iSizeToLoad > $iFileSize ? 0 : $iFileSize - $iSizeToLoad)
    GUICtrlSetData($idProgress, 0)
    GUICtrlSetState($idProgress, $GUI_SHOW)
    For $i = $iStartPos To $iFileSize - $iBlockSize Step $iBlockSize
        $iPercent = Int( $i * 100 / $iFileSize)
        GUICtrlSetData($idProgress, $iPercent)
        FileSetPos($hFile, $i, 0)
        $sLines &= FileRead($hFile, $iBlockSize)
        If $i = $iStartPos Then $sLines = StringRegExpReplace($sLines, "^\V*\R", "", 1)
    Next
    $sLines &= FileRead($hFile)
    GUICtrlSetData($idProgress, 100)
    Sleep(100)
    GUICtrlSetState($idProgress, $GUI_HIDE)

    _GUICtrlEdit_AppendText($idEdit, $sLines)
    FileClose($hFile)
    Return 1
EndFunc

Func _Search()
    _GUICtrlEdit_Find($idEdit)
EndFunc

Func _FileOpenGUI()
    _FileOpen()
EndFunc

Func _FileOpen($sFileToOpen = "")
    If Not FileExists($sFileToOpen) Then $sFileToOpen = FileOpenDialog("Select a file", $sLastDirectory, "Log File (*.log)|All (*.*)", 1, "", $hGui)
    If @error Then Return
    If $sFileToOpen = $sLogFile Then
        If not $iFileChange Then Return
        $iFileChange = 0
    EndIf
    GUICtrlSetData($idEdit, "")
    GUICtrlSetState($idPause, $GUI_ENABLE)
    If Not $iPause Then _Pause()
    If _LoadTail($sFileToOpen) Then
        $sLogFile = $sFileToOpen
        GUICtrlSetData($idFileName, $sLogFile)
        $sLastDirectory = StringRegExpReplace($sLogFile, "\\[^\\]+$", "")
        RegWrite("HKEY_CURRENT_USER\Software\" & $sAppTitle, "LastDirectory", "REG_SZ", $sLastDirectory)
    Else
        MsgBox(16, $sAppTitle & " - Error", "Unable to load the specified file.")
    EndIf
    _Pause()
EndFunc

Func _Refresh()
    Local $sLines, $hFile
    Local $iNewSize = FileGetSize($sLogFile)
    Local $sNewFileDate = FileGetTime($sLogFile, 0, 1)
    If $sNewFileDate = $sFileDate Then Return
    $sFileDate = $sNewFileDate

    If $iNewSize > $iFileSize Then
        $hFile = FileOpen($sLogFile, $FO_READ)
        FileSetPos($hFile, $iFileSize, 0)
        $sLines = FileRead($hFile, $iNewSize - $iFileSize)
        FileClose($hFile)
        $iFileSize = $iNewSize
        _GUICtrlEdit_AppendText($idEdit, $sLines)
    Else
        $iFileChange = 1
        _FileOpen($sLogFile)
        Return
    EndIf
EndFunc

Func _Pause()
    $iPause = Not $iPause
    If $iPause Then
        GUICtrlSetData($idPause, ChrW(0x34))
        GUICtrlSetState($idSearch, $GUI_ENABLE)
        AdlibUnRegister("_Refresh")
    Else
        GUICtrlSetData($idPause, ChrW(0x3b))
        GUICtrlSetState($idSearch, $GUI_DISABLE)
        AdlibRegister("_Refresh")
    EndIf
EndFunc

Func _About()
    Local $sAboutText = $sAppTitle & " is a real-time log file viewer. It allows you to monitor huge files by showing you the end of the file (not the whole file)." & @CRLF & _
                                     "Thanks to use it !" & @CRLF & @CRLF & _
                                     "Made with AutoIt version " & @AutoItVersion & " by JGUINCH."
    MsgBox(0, $sAppTitle, $sAboutText, 0, $hGui)
EndFunc

Func _Exit()
    Exit
EndFunc

 

 

Edited by jguinch
code update
Link to comment
Share on other sites

Nice idea!

5 hours ago, jguinch said:

It allows you to follow the tail of a log file (or other txt file) in real time.

...it must be said, however, that if the file is modified not by 'appending' new lines, but by modifying the contents of the lines inside the file or also adding new lines in the body of the file (instead of appending new lines as normally happens in a log file) the result displayed in the log viewer is a bit "garbled" and does not match the contents of the monitored file.

 

Edited by Chimp

 

image.jpeg.9f1a974c98e9f77d824b358729b089b0.jpeg Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Link to comment
Share on other sites

On 9/7/2020 at 5:15 PM, jguinch said:

Thanks guys.

@Chimp : you're right. I just edit the code. Now, it can detect that the content has changed (it reloads the end of the file)

 

Hi @jguinch ,thanks for the update,

... but there are still some imperfections ....
i tried to do the following to check:
saved this small text file and opened in notepad; also opened with your script so to track it.

begin
1
2
3
The end


I put the two windows next to each other and made a change to a line of the text in notepad, for example I changed the line with the number 3 to 333.
Saving the file in notepad, we see that the monitored text also changes but does not match. If you then simply save it a second time without any further changes then it falls into place (in the latter case the script only detects the file date change).
So wouldn't it be better to just check the file date and update the tracking if the date has changed without running the other file length checks?
sorry for the hassle ...

 

image.jpeg.9f1a974c98e9f77d824b358729b089b0.jpeg Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Link to comment
Share on other sites

@Chimp Thanks for your keedback, you're right.
The only way will be to reload the end of the file each time the file changes. I want to avoid it.
I tried with CMTrace (tool from Configuration Manager tools) to see its behaviour , and it's the same result.

So, for the moment, the tool stay as it, since a log file is usually not used as you describe. Maybe an option will be added for the refresh behaviour...

 

 

 

 

Link to comment
Share on other sites

indeed the fast and easy way is to use PS 

Quote

Get-content "C:\pfirewall.log" -Tail 10

You can use the WAIT parameter to only show the new lines added to the log file for montoring

Quote

Get-Content -Path "C:\pfirewall.log" -Wait

If you want the Gui version of FileOpenDialogue

CLS

[System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms") | Out-Null

$dlg = New-Object System.Windows.Forms.OpenFileDialog

$dlg.initialDirectory = """ + initialDir + """
$dlg.filter = "ZIP files|*.zip|Text Documents|*.txt|Shell Scripts|*.*sh|All Files|*.*"
$dlg.FilterIndex = 4
$dlg.Title = 'Select a file to upload'
$dlg.ShowHelp = $True
$dlg.ShowDialog() | Out-Null
$dlg.FileName

Get-content $dlg.FileName -Tail 10

Enjoy !

Link to comment
Share on other sites

oh well, have at it.

 

maybe you could use Powershell to do the tail and get the output to your gui

 

I would use 

Get-Content D:\log.txt -Tail 10 –Wait

 

to monitor the log and put that in your gui

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to comment
Share on other sites

@jguinch   

Oh yes I forget it is an autoit forum.

Here you go 😉

 

#AutoIt3Wrapper_UseX64=y

#include <MsgBoxConstants.au3>
#include "C:\CLR.au3"
#include <Clipboard.au3>

Local $PS_Script = " "

_Run_PSHost_Script('Get-content "C:\pfirewall.log" -Tail 10', 1)

Func _Run_PSHost_Script($PSScript, $iOutput = 0)
    Local $oAssembly = _CLR_LoadLibrary("System.Management.Automation")
    ConsoleWrite("!$oAssembly: " & IsObj($oAssembly) & @CRLF)

    ; Get Type
    Local $pAssemblyType = 0
    $oAssembly.GetType_2("System.Management.Automation.PowerShell", $pAssemblyType)
    ConsoleWrite("$pAssemblyType = " & Ptr($pAssemblyType) & @CRLF)

    Local $oActivatorType = ObjCreateInterface($pAssemblyType, $sIID_IType, $sTag_IType)
    ConsoleWrite("IsObj( $oAssemblyType ) = " & IsObj($oActivatorType) & @TAB & @CRLF)

    ; Create Object
    Local $pObjectPS = 0
    $oActivatorType.InvokeMember_3("Create",0x158, 0, 0, 0, $pObjectPS)
    ConsoleWrite("IsObject: " & IsObj($pObjectPS) & @TAB & "$pObject: " & ObjName($pObjectPS) & @CRLF)

; ------------- CONSOLE ---------------
    Local $oAssembly1 = _CLR_LoadLibrary("mscorlib")
    ConsoleWrite("!$oAssembly: " & IsObj($oAssembly1) & @CRLF)

    ; Get Type
    Local $pAssemblyType1 = 0
    $oAssembly1.GetType_2("System.Console", $pAssemblyType1)
    ConsoleWrite("$oAssembly1 = " & Ptr($oAssembly1) & @CRLF)

    Local $oActivatorType1 = ObjCreateInterface($pAssemblyType1, $sIID_IType, $sTag_IType)
    ConsoleWrite("IsObj( $oActivatorType1 ) = " & IsObj($oActivatorType1) & @TAB & @CRLF)

    ; Create Object
    Local $pObjectPS1 = 0
    Local $sText[] = [@CRLF & "AutoIT Rocks !!" & @CRLF & @CRLF]
    $oActivatorType1.InvokeMember_3("Write", 0x158, 0, 0, CreateSafeArray($sText), $pObjectPS1)
;~     ConsoleWrite("$pObjectPS1: " & IsObj($pObjectPS1) & @TAB & "$pObjectPS1: " & ObjName($pObjectPS1) & @CRLF)

; <<<<<<<<<<<<<<<<<<< PS COMMAND HERE >>>>>>>>>>>>>>>>>>>>

    $pObjectPS.AddScript($PSScript) ; Add Script here

; <<<<<<<<<<<<<<<<<<< Output >>>>>>>>>>>>>>>>>>>>
        Switch $iOutput
            Case 0
                $pObjectPS.AddCommand("Clip")
            Case 1
                $pObjectPS.AddCommand("Out-GridView")

            Case 2
                $pObjectPS.AddCommand("Out-Printer")
            Case 3
                $pObjectPS.AddCommand("Out-File")
                    $sFile = @DesktopDir & "\PShost.txt"
                    ConsoleWrite("> " & $sFile & @CRLF)
                $pObjectPS.AddArgument($sFile)
            Case 4
                $pObjectPS.AddCommand("Out-Null")
            Case Else
                MsgBox(0,"PSHost","Wrong Output Choice ?")
        EndSwitch

    $objAsync = $pObjectPS.BeginInvoke()
    ConsoleWrite("$objAsync " & IsObj($objAsync & @TAB & "$pObject: " & ObjName($objAsync) ) & @CRLF)


        While $objAsync.IsCompleted = False
            ContinueLoop
        WEnd
            ConsoleWrite("Completed : " & $objAsync.IsCompleted & @CRLF)

    $objPsCollection = $pObjectPS.EndInvoke($objAsync)

;============================================================
        Switch $iOutput
            Case 0
                MsgBox(0,"PSHost",_ClipBoard_GetData())
                ClipPut("")
                _ClipBoard_Close()
            Case 1
                WinWaitClose("")
            Case 3
                MsgBox(0,"PSHost","File Output Ready on Desktop.")
        EndSwitch
EndFunc

Enjoy !!

Link to comment
Share on other sites

that is brilliant @ptrex! nice way to get ps output into a nice gui
 

I was trying to write up a script just now to see if I could do it, but you knocked the ball out of the park with this. I need to test this on a huge log file being appended to but I have no HUGE files to test.

i just tested your code on a build that I do and you even have FILTERS!! woot, this is awesome.

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Link to comment
Share on other sites

You're welcome.

All credits go to the forum members that are far more gifted than me.

I just put the pieces together.

I am quit confident that performance will work great on huge files. Since the PS will run in a separate x64 process than Autoit.

Let the world know how the testing went.

Enjoy !

Link to comment
Share on other sites

  • 1 year later...

  

On 9/14/2020 at 2:43 PM, ptrex said:

@jguinch   

Oh yes I forget it is an autoit forum.

Here you go 😉

 

Enjoy !!

 

How to get the data as an array without using clipboard or saving into a file? (I'd like to convert the data into a database so it can be filtered/sorted/searched/color formatted, etc)

Edited by VAN0
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...