Can I use AutoIt for Google Traslate on double-click?

[Indubhushandas]

Hello! ;-) My name is Ivan, and I never heard of AutoIt.

Well, I am studying c++ now, just had some task with classes and fractions. Looked into wiki for history of fractions, found Hindi version of page. As I was living for one year in India (and now I am a Krishna devotee, or say Hindu), this naturally attracts me (you know, Indians invented zero, all 10 digits, which came from Indians through Arabs to Europe etc). I wanted to read a page in Hindi, but my knowledge is very weak.  

Actually, I have one tool in Chrome Browser, it is called "Lingua-Leo extension" (see : https://chrome.google.com/webstore/detail/lingualeo-english-transla/nglbhlefjhcjockellmeclkcijildjhi?hl=ru ). Well, idea is that I need a tool, so that I can double-click any word in Hindi language and get a translation into my Russian, Ukrainian or English etc.
More generally, I'ld like a tool so that I can double-click any word in any language of google-translate, and get translation (like in LinguaLeo: double click English word - immediately get around 5 translations in Russian, it is enough in many cases) to my language ...
So, I was thinking, how to do that... In Chrome, I haven't found good extension to do that. There are extensions for Firefox, but they are also not that perfect - 1) they are not as good as Google Translate. 2) it is also so slow, as Google-translate, because requires to do so many clicks.

Usually, if I want to use google translate, it is very slow and boring in the way they did it. One thing - I can just open separate window-tab, and place there https://translate.google.com.ua/?hl=ru address, and so I can have on one side - page in Hindi etc, and on left hand - translation in Russian\English\Ukrainian etc. It is good, but SLOW! this is why I almost never use it... However, when I have lingua-leo - it is just fun! I double-click  and get a few meanings for words (some I know, but forgot - they are just in passive dictionary, so I simply need to remember them quick, without opening another tab, which takes time, and makes me kind of nervous, I'ld say it is even a stress to do that nonsense of clicking like a "monkey", though knowing that good programs must be better than what google translate offers now)...

so, recently i had found one nice program - called "Window hacker". actually, i needed it only to use some "stylus" for programs like Adobe illustrator, Corel, autocad etc - so that I can draw on screen some images in vector instead of pen\pencil on paper). and that program had so old driver, that it didn't show fonts well. I was kind of irritated. driver program showed only half of window, hiding all other buttons to set sensitivity of stylus etc etc... So I found Window-Hacker (see http://www.aezay.dk/aezay/wndhack/ ) and this solved the problem. In this program, I found window hwnd etc... I used a program called psr.exe (we learned that on our "Win10"\Ubuntu admin\cisco course), just run in any new MS Windows - it records all events to zip, also with some screenshots, and tracking which controls I click - that's what I need! and that I used for Window-Hacker - so I could change sizes of window, and set necessary setting, and great! I felt myself a debugger, who was working for the company, who developed the stylus ;-)

anyway,now I again faced this problem: technologies are sometimes so slow, that it demotivates from doing anything... thus, talents get worse, they are like in story from Bible: "talents are digged underground". That's bad, but big companies (like "google translate") do not care much, so have to kind of do their work ourselves.

some 15-20 years ago I often used a program KeyText (http://www.mjmsoft.com/). It was fun. usually I used it for scanning. So the scanner was very slow. I mean, software wanted so many user input, and it was bugging me like nothing. It always prompted me to enter the file name, but I just wanted to enter automatic number... I simply used keytext to enter the date as filenames, and it was cool. My HP scanner got a new life, and I saved so much time automating the tasks... and yes, HP should have done that themselves... Again, feeling myself like part of their company... doing their job, but noone payed me for this. anyway...

so, in Window-hacker I turned-on a mode "scan window at mouse", thus I found out the control which browser used. So, I found its name: chrome_renderwidgethosthwnd widgetwin
I googled for it, and found your site, where people discuss these things... though it is quite complicated... I've read a few books on assembler for windows, so know a bit about hwnd etc, but it always was kind of "scary" words and things to use them... and to have any motivation to get to know how all this works...

so i want to ask: is it worth this? do I really need to use AutoIt for what I want?
I mean: I want to make some kind of extension which can (on my double click, or mouse-over some word in foreign language) - to show in hint\tooltip window - the translation....

well, in general, i'ld like to know how to do some tweaking around some programs available (like I did with that old driver). I want to become a programmer, and we will study WinAPI in future - maybe after 1-2 semesters, I am not sure. 

So, question is: "Can I use AutoIt for Google Translate on double-click?" It should work like Lingua-Leo... Double-click and get translations from any language to my language(s)...
can I do it with AutoIT? Or it is too complicated for such programs?

I was thinking that I can make "double browser". Say, like I have experience with VIsual Basic\C#, so I can make two browser Windows, in one - open wiki in foreign language, and in another window - on side, or below - open google translate site. but still it will not be as good I imagine it should be... I'ld require to do extra stuff - not just double click, but:
1 double click to select word(s)
2 open context menu and select copy to buffer (or ctrl c) which takes much time for thousands of words... I want to learn hundreds of thousands of words, whole Hindi (and go live in India, marry some Indian, and get Indian residence), so I don't have much time for "monkey" copy-paste (it feels like with HP scanner, which bugged me do extra stuff like entering manual numbering of files etc)...
3 then I need to paste that in google translate window. of course, it also has a feature to double click on the word - and sometimes in translates it. but sometimes - not! and I hell don't know why they don't do it normally (such a big company - shame on them... I'm silent for many g@ys working in Google and Microsoft... and the last forced us to buy licensed Windows... but MS came out to spend 1 100 000 dollars for inventing condoms, including 100-000$ were spent for making them from cow's\bull's organs etc...! hat a shame and sin. ok. let's forget it)... (yes, im telling all this, because i think such people work in such big companies, that's why programs are so buggy, so we have to change that)...

so can AutoIt help with this?
I can get Keytext again to try do that. I usually assigned some "hot-key", and it could have done  "copy to clipboard", switch to another browser window, paste there... but I want more control of that, so probably AutoIt is the way...Can anyone guide on how can I quickly jump into AutoIt and get this working what I want?

 

Spoiler

And I am not even sure it is really quite possible ... google translate has some API available, and it seems to be not free, so they may limit this... thus, what happens, I often tell this story about Hitler: he was not allowed to continue studying arts\drawing... what happened? he didn't forget and didn't forgive - he killed many people... actually, you know, I live in Ukraine. there is war now here... around 13 000 people are killed already since 2014... "Donetsk\Lugansk national republics" claimed that they are "successors" to Soviet Union ("Krivoy Rog Donetsk republic") etc... And it very sad... Putin was called on TV as "Hitler" (what Putin said when Russia annexed\stolen our Crimea Penninsula - sounded very similar to what Hitler did when he stolen so many lands)... So, when knowledge is now allowed, stolen by companies - Hitler, Putins arise and start killing each other... (but that' another story, and let's stop it on this point)... (well, yes, by the way, some "atheist number one in Ukraine", Duluman,who was working in our kiev-politechnical institute, said "anyone who believes in God much be mass-killed with machine gun"... it was in 2012. he died on 2013, though... on my birthday. anyway... he was also member of communist party... and often communists declare "war" against Europe\America\West as "fascists" (claiming they, communists\bolsheviks, are better, because they "won the 2nd world war")... Even our ex-president, Yanukovich, said "fascists now came to rule Ukraine". no we are not fascists. We simply wanted to go to European Union, and not go with Yanukovich to Russia\USSR (Crimea\Donetsk-Lugansk republics now are almost USSR, sadly)... ) Yanukovich now must go to prison for 13 years (he was there already 2 times before becoming president)...

I remember, how Bill Gates said: "open-source is software communism". But it is not so! if knowledge is shared, software source is shared - it can be good. Google doesn't open (I think) its google-translate source code, it's dictionaries. so it is bad, people can't learn languages well (by the way, Satya Nadella from India - is now ruling Microsoft, he is Indian)...

I am thinking about all this now, because yesterday our teacher said "Indian code is bad"... I can partially agree, as Microsoft has Indians working, and some are like Satya Nadella (meat-eaters etc - sinners, in short). yes, I know, there is always growing demand on programmers. by the way, Mercury is the planet, which rules both languages and artificial languages (programming). around 2000 I wanted to make a program, which would convert texts into movies (see my profile for details - https://connect.unity.com/u/ivan-donets ), so I am interested in programming... 

Sorry for so much text.

 

 

[JLogan3o13]

Wow, you really put our resident word-wallers to shame. How about a short, concise description of what you're trying to accomplish? I got lost somewhere around Chapter 3

 

[therks]

AutoIt doesn't interface with Chrome (or most browsers) very well since it's not a standard window with standard controls. To my understanding, it's basically one big "control" that Chrome renders on, so AutoIt can't see individual controls or elements. I think a better tool for what you want would be an extension for Chrome. I've never written one so I'm not sure what's involved, but if you haven't found one you might have to make your own. You could also look into extensions that allow you to write scripts for pages, Tampermonkey is a popular one, and there is a large amount of userscripts available if you look around.

Good luck.

[JLogan3o13]

2 hours ago, therks said:

AutoIt doesn't interface with Chrome (or most browsers) very well since it's not a standard window with standard controls.

I would disagree with this assessment wholeheartedly. Both the WebDriver thread

and the UI Automation thread

can assist you in automation Chrome, along with other browsers.

[seadoggie01]

Undoubtedly AutoIt works well with browsers, but the kind of interaction he's talking about would be difficult in AutoIt, if not impossible. He's trying to translate any highlighted word and I don't think there's an event thrown when things are highlighted. That's probably best suited to a Chrome extension in JavaScript. *shudders*

Edited December 10, 2019 by seadoggie01

[JLogan3o13]

Understood, and agree in this specific case. But "AutoIt can't see individual controls or elements" isn't a statement we would want coming up in a Google search for someone trying to automate a browser, without at least some explanation.

[Nine]

Yes you can with IE events.  Just intercept the dblClick event, grab the select value, and run a check on whatever translator you prefer...

[LarsJ]

I think that's exactly what the AnnotationType_Highlighted constant is for.

I'm pretty sure all the requirements in first post can be implemented with UI Automation code.

I've made a note about this thread, but I've already booked the coming weeks for another project.

[junkew]

Sure this can be done with uiautomation just check what narrator is speaking when you hover over text. If it speaks you can reach it with uiautomation.

And simplespy will give you the base but make sure for chrome uiautomation accessibility is turned on.

Edited December 12, 2019 by junkew