ssheldon Posted June 5, 2018 Share Posted June 5, 2018 Hi all, I am brand new to AutoIT. I would like to try and automate testing a piece of software to verify that the UI matches a known good state. What I mean by this is I would take a screen shot of the software being tested and save it. Then I would use AutoIT to launch the software and take a screen shot of the software and compare it to the original saved screenshot. The software I am testing has different licensed configurations, and I figure by taking different screenshots of the UI under different licenses I can automate the testing of the UI under different license configurations to verify that the correct commands are enabled (or not) for different licensed scenarios. I have been reading about bitmap processing here: To make my first attempt at this, I want to make a simple program that does this: Launch the target software. Take a screenshot. Compare to a previous saved screenshot. I am currently working through the AutoIt_123 lessons. Can anyone tell me how to launch an executable from AutoIT? Thanks, Steve Link to comment Share on other sites More sharing options...
Xandy Posted June 5, 2018 Share Posted June 5, 2018 (edited) ShellExecute("path\filename.exe") Run("path\filename.exe") Either of these functions can launch an exe from AutoIt. After typing the function, hover over function name and press F1 to bring up the help file containing additional parameter information. Edited June 5, 2018 by Xandy ssheldon 1 Human Male Programmer (-_-) Xandy About (^o^) Discord - Xandy Programmer MapIt (Tile world editor, Image Tile Extractor, and Game Maker) Link to comment Share on other sites More sharing options...
ssheldon Posted June 5, 2018 Author Share Posted June 5, 2018 OK, found my answer in the help. To launch the program I will use: Run("myprogram.exe") Next up, how to take a screenshot. Steve Link to comment Share on other sites More sharing options...
Xandy Posted June 5, 2018 Share Posted June 5, 2018 (edited) I use these: Sample only specific window: https://www.autoitscript.com/autoit3/docs/libfunctions/_ScreenCapture_CaptureWnd.htm Sample Desktop: https://www.autoitscript.com/autoit3/docs/libfunctions/_ScreenCapture_Capture.htm Edited June 5, 2018 by Xandy Human Male Programmer (-_-) Xandy About (^o^) Discord - Xandy Programmer MapIt (Tile world editor, Image Tile Extractor, and Game Maker) Link to comment Share on other sites More sharing options...
water Posted June 5, 2018 Share Posted June 5, 2018 Define "known good state". Does this mean controls are in a defined position, state or have a defined content ...? Do you want to test different use cases as well? Means: enter data and check that the program acts correctly? Before starting to compare screenshots you should be sure that this is the best way to achieve your goal An example of both screens would greatly help. My UDFs and Tutorials: Spoiler UDFs: Active Directory (NEW 2024-07-28 - Version 1.6.3.0) - Download - General Help & Support - Example Scripts - Wiki ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts OutlookEX (2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki PowerPoint (2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki Task Scheduler (2022-07-28 - Version 1.6.0.1) - Download - General Help & Support - Wiki Standard UDFs: Excel - Example Scripts - Wiki Word - Wiki Tutorials: ADO - Wiki WebDriver - Wiki Link to comment Share on other sites More sharing options...
ssheldon Posted June 5, 2018 Author Share Posted June 5, 2018 17 minutes ago, water said: Define "known good state". Does this mean controls are in a defined position, state or have a defined content ...? Do you want to test different use cases as well? Means: enter data and check that the program acts correctly? Before starting to compare screenshots you should be sure that this is the best way to achieve your goal For example, if you launch Paint, the Crop, Outline, and Fill commands are all shown in grey, as they are disabled, while all of the commands in the Tools group are enabled. If I took a screen shot of the Paint ribbon bar, and then checked it through automation, and time ribbon bar changed somehow (commands added or missing, or enabled/disabled state changed), then the bitmap comparison should detect that. Sound good? Steve Link to comment Share on other sites More sharing options...
Moderators JLogan3o13 Posted June 5, 2018 Moderators Share Posted June 5, 2018 52 minutes ago, ssheldon said: For example, if you launch Paint, the Crop, Outline, and Fill commands are all shown in grey, as they are disabled, while all of the commands in the Tools group are enabled. If I took a screen shot of the Paint ribbon bar, and then checked it through automation, and time ribbon bar changed somehow (commands added or missing, or enabled/disabled state changed), then the bitmap comparison should detect that. Sound good? Steve Rube Goldberg would be proud... You say you're checking license configuration, are you truly saying there is no other way (license file, registry entry, dll presence, etc. etc. etc.) that you can check for this? If the correct license is applied, one would think the commands would not be grayed out. "Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball How to get your question answered on this forum! Link to comment Share on other sites More sharing options...
ssheldon Posted June 5, 2018 Author Share Posted June 5, 2018 1 minute ago, JLogan3o13 said: Rube Goldberg would be proud... You say you're checking license configuration, are you truly saying there is no other way (license file, registry entry, dll presence, etc. etc. etc.) that you can check for this? If the correct license is applied, one would think the commands would not be grayed out. I have the license file, but that does not mean that the software is acting on it correctly. The test we do now manually is fire up the software with a particular license file, and then go and verify that the parts of the software that should be active are, and that other parts are not. We currently do this by manually exercising the UI. I figured we could automate this by screen shot comparison of the various ribbon menus. Another way would be to interrogate the state of all commands, and I think there are ways to do that also, but it seems to me a lot faster to just examine the entire ribbon at one go. This will also catch any changes in the UI. Steve Link to comment Share on other sites More sharing options...
Moderators JLogan3o13 Posted June 5, 2018 Moderators Share Posted June 5, 2018 And you don't think that, instead, working with the vendor to resolve the issue of why applying the license file isn't making the right features available would be a better idea? AutoIt is great at doing almost anything you want to automate; just seems like you're trying to automate a way around not fixing the underlying issue. Earthshine 1 "Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball How to get your question answered on this forum! Link to comment Share on other sites More sharing options...
ssheldon Posted June 5, 2018 Author Share Posted June 5, 2018 To be clear here, what I am trying to do is automate testing that traditionally was done by hand through the UI while still testing automatically through the UI. We have APIs that I could exercise through Visual Studio to test functionality bypassing the UI, but this is not what I am trying to do. Steve Link to comment Share on other sites More sharing options...
ssheldon Posted June 5, 2018 Author Share Posted June 5, 2018 2 minutes ago, JLogan3o13 said: And you don't think that, instead, working with the vendor to resolve the issue of why applying the license file isn't making the right features available would be a better idea? AutoIt is great at doing almost anything you want to automate; just seems like you're trying to automate a way around not fixing the underlying issue. I am the vendor. I work in testing. One of the things that we test is that various license configurations work. This is done by installing the appropriate license file, then launching the software, and making sure that the correct set of commands is operational. This testing is currently done manually. This is only one tiny thing that I hope to ultimately automate. Ultimately I intend to test many aspects of our software via UI automation. Initially I was looking at macro recording software (is it possible to record macros in AutoIT?) but I did not find tools that had enough programming oomph behind them to be able to analyze what happens when the macro does things in the software. For example, I can easily record a macro that clicks File -> SaveAs. But can I tell if the dialog that pops up looks right? I haven't seen that kind functionality in a macro recording software. Steve Link to comment Share on other sites More sharing options...
careca Posted June 5, 2018 Share Posted June 5, 2018 Maybe different licenses could change the titlebar text, corresponding to specific things enabled/disabled, and it can be checked in autoit fast and easy. Or am i looking at this totally backwards? I know there are ways to compare screenshots, i've seen posts about it, one was about checking the surveilance camera for movement. It would check for changes in every other pixel or so. I think you want something like that.. Spoiler Renamer - Rename files and folders, remove portions of text from the filename etc. GPO Tool - Export/Import Group policy settings. MirrorDir - Synchronize/Backup/Mirror Folders BeatsPlayer - Music player. Params Tool - Right click an exe to see it's parameters or execute them. String Trigger - Triggers pasting text or applications or internet links on specific strings. Inconspicuous - Hide files in plain sight, not fully encrypted. Regedit Control - Registry browsing history, quickly jump into any saved key. Time4Shutdown - Write the time for shutdown in minutes. Power Profiles Tool - Set a profile as active, delete, duplicate, export and import. Finished Task Shutdown - Shuts down pc when specified window/Wndl/process closes. NetworkSpeedShutdown - Shuts down pc if download speed goes under "X" Kb/s. IUIAutomation - Topic with framework and examples Au3Record.exe Link to comment Share on other sites More sharing options...
Earthshine Posted June 5, 2018 Share Posted June 5, 2018 (edited) What licensing software are you using? Some of them set up a port on the local host and you can view all the license information that you want there in a browser and capture the real info Edited June 5, 2018 by Earthshine My resources are limited. You must ask the right questions Link to comment Share on other sites More sharing options...
ssheldon Posted June 5, 2018 Author Share Posted June 5, 2018 I cannot control what the license files do - that is already fixed. So having them change the behavior of the software other than what the license files already do is not an option. Imagine we launch paint. The ribbon looks like this: This is the "known good state". In other words, this is what it should look like when we open Paint on the Home tab. Now imagine I launch Paint and go to the home tab and it looks like this: Notice that some of the commands have changed state: There is probably a way to programmatically interrogate each command and discover its state. But this would require a check for every command. In addition, it won't detect other changes in the UI, like commands moving around, or other anomolies. I figured an easy test here would be to create a "baseline" screen capture, and then when the automated test runs it could check a section of the screen against that baseline. Not only will this detect changes in command states, but any other deviations in the UI presentation also. Steve Link to comment Share on other sites More sharing options...
ssheldon Posted June 5, 2018 Author Share Posted June 5, 2018 7 minutes ago, Earthshine said: What licensing software are you using? Some of them set up a port on the local host and you can view all the license information that you want there in a browser and capture the real info We use FlexLM. It is possible that I could go "under the hood" to find the states, but I am trying to test it through the UI, as a user would. For example, our software has numerous APIs that can be called programmatically to do things that the software does. Like draw a box, for example. But this only tests the API, not the UI. I want to test the software by simulating users actually poking buttons in the UI. As an analogy, in the old days you could reach under the hood of your car and actuate the throttle linkage and rev the engine. You can also, of course, sit in the driver's seat and push the gas pedal to rev the engine. One is not an adequate test of the other. Just because I can rev the engine under the hood does not mean that the gas pedal works like it is supposed to. I'm trying to get through the actual user interface, not bypass it. This is why I am looking to UI automation solutions rather than just writing programs that access our internal APIs. Not to mention I think it will be easier. Steve Xandy 1 Link to comment Share on other sites More sharing options...
Earthshine Posted June 5, 2018 Share Posted June 5, 2018 we use flexlm too. ok. As both systems and software engineer and avid tester I understand where you're coming from completely and I think a couple of post above somebody gave a link to some comparison stuff or we can help look it up My resources are limited. You must ask the right questions Link to comment Share on other sites More sharing options...
Earthshine Posted June 5, 2018 Share Posted June 5, 2018 You have to understand that image comparison alone will only get you so far. You don't need to use the APIs and visual studio to mimic things but you could use their libraries to manipulate the actual controls that you care about in the UI to verify its behavior. Depending on the application you may be able to use auto IT alone to do the GUI manipulations and in doing so you could prove out the state of your license installation My resources are limited. You must ask the right questions Link to comment Share on other sites More sharing options...
ssheldon Posted June 5, 2018 Author Share Posted June 5, 2018 It's possible. Image comparison just seemed easier to me. I don't know how to manipulate the UI programmatically. I just discovered that the buttons on the ribbon are not "controls", for example, so I don't know how to automate clicking on the Text button in Paint, for example, via AutoIT (or any other means). Presumably one can click via x,y coordinates, but this might be a problem if the program is run minimized, or on a different resolution monitor, or whatnot (but I expect that will blow image comparison out of the water, also). So my expectation is that this kind of program will need to be run on a specific machine. Steve Link to comment Share on other sites More sharing options...
Xandy Posted June 5, 2018 Share Posted June 5, 2018 (edited) The tools in this example will achieve BMP image comparison. Unless I am wrong, and I am never wrong. It's just an example of the tools however. This is kind of what you want. #include <GDIPlus.au3> _GDIPlus_Startup() $hBMP_somefile = _GDIPlus_BitmapCreateFromFile($filepath) For $y To $height For $x To $width If _GDIPlus_BitmapGetPixel($hBMP_Screen, $x, $y) = _GDIPlus_BitmapGetPixel($hBMP_somefile, $x, $y) Then ; Maybe you want to say if Not Equal <> ; I do it with a counter checksum, but then you must set a counter EndIf Next; x Next; y _GDIPlus_Shutdown() Forgive I didn't put much time into this example. Also it may not be the 'best'. It's not as fast as CPP (writing this in CPP is not that difficult and it's probably 100..1000 times faster). There are lots of other examples that I have been hopeful of, but this is what works for me. Perhaps a hash could be generated from screenshot area, and image loaded from file as hash. Never done it that way, but this concept code can do what you want in a reasonable amount of time. Remember to keep the compared area as small as you can. Less pixels; done faster. You can also use a mask to disregard pixels from a rectangular area or use a list. Use the tools in my posts above to collect the image screen data. ScreenCapture or whatever. Edited June 6, 2018 by Xandy SkysLastChance 1 Human Male Programmer (-_-) Xandy About (^o^) Discord - Xandy Programmer MapIt (Tile world editor, Image Tile Extractor, and Game Maker) Link to comment Share on other sites More sharing options...
careca Posted June 5, 2018 Share Posted June 5, 2018 3 hours ago, ssheldon said: ...one can click via x,y coordinates, but this might be a problem if the program is run minimized, or on a different resolution monitor, or whatnot I guess you could force the application window to be enabled, active and have a specific resolution that would fit all possible screens, and then test it by pixels or that screenshot method. Spoiler Renamer - Rename files and folders, remove portions of text from the filename etc. GPO Tool - Export/Import Group policy settings. MirrorDir - Synchronize/Backup/Mirror Folders BeatsPlayer - Music player. Params Tool - Right click an exe to see it's parameters or execute them. String Trigger - Triggers pasting text or applications or internet links on specific strings. Inconspicuous - Hide files in plain sight, not fully encrypted. Regedit Control - Registry browsing history, quickly jump into any saved key. Time4Shutdown - Write the time for shutdown in minutes. Power Profiles Tool - Set a profile as active, delete, duplicate, export and import. Finished Task Shutdown - Shuts down pc when specified window/Wndl/process closes. NetworkSpeedShutdown - Shuts down pc if download speed goes under "X" Kb/s. IUIAutomation - Topic with framework and examples Au3Record.exe Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now