Find Text from ReadFile

[allSystemsGo]

zsutton92,

You want each log in a seperate tab? How many?

83MB is not really large. Regardless, based on the format you've posted it would be very easy to parse out data. One possibility is a dialog with 5 or 7 tabs, each tab representing a day's worth of data. You would then have a rolling week's worth of logs.

kylomas

I have been thinking of a way to possibly append the data to a database-like structure, then be able to query it based on keywords, or like a date. That way I could look at posts only from today, or yesterday, or a certain day. I am just using this log now because it was easy to access. I would like to be able to remotely monitor other PC's event logs, plus some other logs that various application grind out day by day.

Perhaps one tab per application, then for the remote events, be able to access the remote pc and then pull its event logs to me....Just in the brainstorming phase right now.

[jchd]

That's the standard way to parse largish logs. Simply browsing sequentially into ever-growing text log files is deemed to ever-increasing response times, regardless of the lack of flexibility of basic string searching versus the power and efficiency of SQL and its possible extensions. SQLite can be used for such purpose and proves very easy to use from AutoIt (part of the standard UDFs).

[allSystemsGo]

How does this get implemented?

[jchd]

It's fairly easy compared to some other DB engines, but it still needs some careful planning if you envision mid- or long-term robust use.

While the help file will introduce yourself to basic use, you will benefit from reading a good tutorial about generic SQL.

Af

ter you get the basics, I'd recommend you download SQLite Expert (freeware version will offer you most if not all of the features you'll need). Then make yourself familiar with simple test DB designs and working. Importing text files with more or less fixed format is easy.

All of that will give you a first feeling of how it can be made to work for you and the precise needs you may have.

Come back here or in PM (as SQLite DB design isn't the central topic in these forums) if you're stuck.

At any rate, don't dismiss the AutoIt + SQLite possibilities. It may require a learning curve but we're here to help.

[kylomas]

One suggestion, this project need more planning and definition. I know that you are just spitballing for now but it might be time to start formalizing some of your ideas. Start at a high level, what it looks like and what it can do, split that up into functions and start drilling down to how each function might work.

Using SQLite as the storage/query mechanism is the best idea in my opinion. Planning and definition are hugely important if you go the DB route.

kylomas

[jchd]

Yes, but focus on what entities you're going to manipulate, which are their relatioships, which constraints apply, which queries are you going to need over long term. Don't think "functions", think "entities" and their characteristics and relationships. This way of thinking is closer to OOP than procedural approach.

After that only start toying with a sample DB to better match your precise context. Using a good 3rd-party SQLite manager as recommended will save you countless of hours of painful coding.

Every log analysis tool around uses SQLish DBs. But SQL is not a spreadsheet, nor a multi-dimensionnal array.

Keep faith!

Again should you need guidance, post where you are and what you find hard to model.

Edited January 31, 2013 by jchd

[allSystemsGo]

Thanks for the ideas and support! I will definitely look into this when I get time. I'll keep you all updated.

[allSystemsGo]

While I am familiarizing myself with SQLite I would still like to use the script I have so far. What I have done is instead of just opening one log hard coded into the script, I added an open file dialog that lets me choose one. Also, I found in the server settings that creates our FTP log that it can be broken down into individual days, so that makes it easier to search. Now, I want to move on to tabs. Is there a way that I could create a "Open in new tab"-like action to the window?

edit---

I have looked over the help file, and while I see the functionality is there, I do not see how to execute what I am wanting...

Edited February 1, 2013 by zsutton92