Is UDF source viewable, and where do they reside?

[mrubin]

I'm new to AutoIt, having found out about it this past week, and am captivated by the framework and the rich ecosystem around it.  I do, however, have a question about UDFs.

Yesterday I went searching for the ability to locate a bitmap on the screen and found a wealth of references to ImageSearch.  There are also many references to its not working as expected on Windows 10, etc.

I therefore went searching for the source code for ImageSearch.  Although I have found links to ZIP files containing the .au3 and the .dll files, I cannot ascertain if one can view the ImageSearch source code itself (to build it locally or contribute).

• Is the ImageSearch source code hosted somewhere (e.g., GitHub)?
• Are all UDFs hosted publicly?
• How can one determine the contact details of UDF authors?

I found the AutoIt "User Defined Functions" wiki page, but did not find ImageSearch listed.  Is this list incomplete?

Thanks in advance,

Moshe Rubin

[water]

Welcome to AutoIt and the forum!
Using ImageSearch can get quite complex and doesn't always return a reliable result.
If you can tell us which program you try to automate we might suggest a better solution.

[mrubin]

@water Thank you for your prompt reply -- nice to be aboard!

I found AutoIt because of a technical issue at work.  As the maintainer of an important internal proprietary software library, I wrote a large regression test suite for the library.  The regression test runs as an MS Windows console application, with the app making hundreds of calls to the library, checking the results carefully to verify nothing breaks between versions.  The regression test suite takes about 30 minutes to run, and at different points in the run, the tester is required to interact with a web browser, logging in and out of online web sites.  It has reached a stage where I need to run the test suite four different times (taking 4 x 30 = 2 hours), run each time against four different web sites (each web site is interacted with differently).

I want to fully automate the regression tests, enabling it to run itself multiple times against different web sites.  The technical problem is identifying buttons within a browser.  Because the buttons are drawn in HTML5 or CSS, MS Windows buttons in the web browser are not window objects -- they are just pixels drawn on the browser canvas (this can be verified easily with Au3Info.exe).  So how can I have my script press a particular button if it is not a real MS Windows window object?

The solution is to use ImageSearch to find the button by providing ImageSearch with a bitmap of the button to search for.  That is why I would like to use ImageSearch.

Part of my question was, where do UDF sources reside?  The reason is that, as a seasoned and experienced software engineer, I would be glad to debug ImageSearch and submit fixes, corrections, and improvements.  For this, the source code needs to be hosted in some public location like GitHub as an Open Source project.  This is the best way to improve any project.  Why is ImageSearch unreliable?  Can experienced software developers contribute to make it a reliable component?

So my questions are:

1. Is ImageSearch hosted publicly?  Can one contribute to it?
2. Are UDFs in general hosted publicly?

Thank you very much for your initial response, and looking forward to reading your reply,

Moshe

[TheDcoder]

1. Is ImageSearch hosted publicly?  Can one contribute to it?
2. Are UDFs in general hosted publicly?

1. Yes, Its hosted here on AutoIt Forums, But the author of ImageSearch seems inactive , You might want to start a new topic and contribute to ImageSearch that way

2. Yes, Its upto the Author of UDF to host it... There is a convenient Downloads feature provided by the forums

[Melba23]

mrubin,

"Standard" UDFs (those installed with AutoIt and maintained by the AutoIt team) are found in the "...\AutoIt3\Include" folder - they are all written in AutoIt code and you can suggest changes/improvements to them via our Trac system (see the BugTracker link in the menu bar).

"Personal" UDFs (written by forum members) usually reside in the Examples section of the forum (for example the links in my sig below), although some are also available for direct download (again via the menu bar). These are often in AutoIt code, but sometimes will involve DLLs as I believe is the case with ImageSearch, which you can find here. Suggestions for changes/improvements to these UDFs should be in the specific thread so that the author can deal with them directly (look at any of my UDF threads to see this).

As a community, we welcome any offers of help to improve AutoIt, so as a "seasoned and experienced software engineer" do feel free to join in.

M23

[jchd]

mrubin,

The solution is to use ImageSearch to find the button by providing ImageSearch with a bitmap of the button to search for.  That is why I would like to use ImageSearch.

Have you tried using standard IE* functions instead?

[mrubin]

@TheDcoder @Melba23 Thank you for the links, but in none of those locations did I find source code for building the DLL.  TheDcoder has pointed out that the author is inactive.  If the source code is not available, there's no way to contribute to the UDF.

@jchd I assume you're referring to the AutoIt "_IE" functions.  Besides the fact that I want the ability to run in any browser (e.g., Chrome), which specific _IE functions did you have in mind?  Can I query the DOM for the absolute/relative location of an HTML/CSS object (e.g., a button)?

Thanks to you all for the responses.

[TheDcoder]

@mrubin I wasn't aware that the author did not provide the source for the DLL. As for the question regarding "_IE" functions, they are a wrapper for Internet Explorer's COM interface

[jchd]

mrubin,

It isn't obvious to me how the browser or API matters for automating your task. You don't have to devise where a button or field is actually displayed just to interact with it. You could even factorize out the browser/API altogether and do it the (very) hard way by using WinHttp functions, but of course I wouldn't recommend that.

[mrubin]

@jchd My testing involves interacting with websites' OAuth 2.0 authentication screens through a browser, inserting usernames and password and scraping things like window titles, URL fields, and inner HTML from the browser windows.  I want to mimic a human's interaction with the screen, certainly not using WinHttp to grab the browser contents.  So I think that ImageSearch() is exactly what I'm looking for.

You write "You don't have to devise where a button or field is actually displayed just to interact with it".  How else can I click the button (which is not a Windows object) if I don't know where it is on the screen?

Edited December 3, 2015 by mrubin

[jchd]

Again, I strongly recommend you have a close look at how IE* functions work and how one can use them to automate interaction with the web, before discarding this easy possibility.

Then WinHttp is not meant to grab contents.

[mrubin]

@jchd I'm always open for good suggestions, will check out the IE* functions and will see if they can help me out.  Can you give me a sneak preview of which IE* functions would help me with my project described above?

Thanks!

[jchd]

For example, look at the second sample code in help for _IELinkClickByText. You see, no need to even have the faintest idea how the thing is rendered by the html/css code: you can click on an IE COM object (button, link). Depending on how the web page is structured (frames or not) you'll have to use different functions to access/manipulate various items. Look at examples in IE help as well as more involved code provided in many posts in "Example scripts" section of the forum.

Many people here are much, much more knowledgeable than me about web automation.

Edited December 4, 2015 by jchd

[JohnOne]

There have been many forks of imagesearchdll and they are linked (some dead some not) in the pages of that topic.

A search on github also yields multiple results for source code.

[jpm]

My way to solve unknown widows reaction is wait the Windows title [,text]  not always possible so I understand the ImageSearch.

But when it is possible I resize the windows to me personnal size and after it is easy th click to a specific area.

For regression I also Capture the specific image and I compare it to a reference Image. It is the way I do regression on AutoIt functions a UDF Tickets fixing.

Cheers

[mrubin]

@jchd Thank you very much for a most interesting lead.

@JohnOne: Wouldn't it be great to have a single, centralized, and public repository? :-)

[Jos]

Wouldn't it be great to have a single, centralized, and public repository? :-)

Oh please let's not, It already difficult as it is to maintain our supplied version.
Do you seriously think there will ever be consensus and a single version that will work for different versions of AutoIt3? 

If so: Keep dreaming

Jos

[bustasnipe]

Imagesearch is ported over from autohotkey, it is really easy to use and works great on windows 7, why don't  you just load up a win7 vm and work on it in there?

[JohnOne]

That's a point, Autohotkey is open source, so it's probably best to look at that source as it will be maintained.

[mrubin]

@Jos Sorry, didn't realize I was wandering into a minefield ;-)  It's just that I come the world of the Boost peer-reviewed portable C++ source libraries, where fixes made to the Open Source GitHub-based repository are maintained run on platforms such as Windows, Mac OS, Linux, Android, iOS, Windows Phone, and more, and are tested daily for the following compilers:

• Linux:
  • Clang: 3.4
  • Clang, C++14: 3.6
  • GCC: 4.4.7, 4.8.2, 4.9.3
  • GCC, C++11: 4.4.7
  • GCC, C++14: 4.9.3
  • Intel: 15.0
• Windows:
  • Visual C++: 8.0, 9.0, 10.0, 11.0, 12.0, 14.0
• Android:
  • Clang: 3.6
  • GCC: 4.8, 4.9
• SunOS:
  • Sun: 6.0
• QNX:
  • QCC: 4.4.2

Boost's additional test compilers include:

• Linux:
  • Clang: 3.0, 3.1, 3.2, 3.3, 3.4
  • Clang, C++14: 3.6, 3.7
  • GCC: 4.4.7, 4.5.3, 4.6.4, 4.7.3, 4.8.1, 5.1.0, 6.0.0
  • GCC, C++11: 4.4.7
  • GCC, C++14: 4.9.3, 5.1.1
  • Intel: 15.0
• Android:
  • Clang: 3.6
  • GCC: 4.8, 4.9
  • FreeBSD:
  • Clang: 3.4.1
  • GCC: 4.8.5, 5.1.0, 6.0.0

(Sorry for the long list but I was trying to make a point <g>.)  It's not inconceivable that the AutoIt3 project would test important and critical UDFs nightly/weekly/monthly against current and previous versions of AutoIt3, requesting that contributors help keep the UDFs (open sourced, of course) running smoothly.  I've contributed to projects like Boost and others, and I'd do it again for AutoIt3.  It just needs some prime movers in the AutoIt3 world to get it rolling.

@bustasnipe and @JohnOne How difficult/easy is it to continually port the AutoHotKey source to AutoIt3 every time a new ImageSearch version comes out?  Can it be automatically converted?  If so, someone can maintain AI3's ImageSearch on a continual basis based on AHK's.  It doesn't sound like rocket science, and would benefit everyone.

Just my 2 cents :-)

Moshe