Which Language to Learn?

[BigDaddyO]

I have been dubbed the File Validation guy at work recently which is fine with me, I kinda like it and AutoIT has been great to do this with except for 1 big thing.  it's super slow.

I'm validating files with millions of rows and some almost 4 thousand characters long in each line and it's taking 4 to 6 hours to finish validations.

I'm using regular expressions, String length, and if X then Y to validate the files.

I'm wondering if I should try and learn C++ or C# or something else in an attempt to speed up the process as the project teams are rather cranky on the time it takes even though it would never get fully validated without it.

 

right now, my focus is on speeding up my file validations but I'd also like to ensure the time spent learning the language will be useful in the future on other apps.

 

Thanks,

Mike

 

[JLogan3o13]

Perhaps if you post your script forum members could offer suggestions on optimization. Beyond that, if you search the forum there are several threads going regarding using FreeBasic to overcome AutoIt's lack of speed with graphics; perhaps the same can be done with file parsing. Would be easier than jumping into a whole new language.

[BigDaddyO]

Just wanted to post back here as I have had some more time to research and work on this.  I decided on C++ and re-wrote my script as a console application.

The AutoIT script was running almost exactly 6 hours.

The C++ console app which performs all the same actions runs in 36 minutes.  

Only downside is that I don't have a nice GUI or File>open to pick the file so I have to type the file into the console at run-time.

 

it was a lot more work to get CodeBlocks, mingw32, and boost setup and functioning properly than it was to figure out and write the C++ code as I was able to copy the majority of my Regular Expressions from my script and paste into the C++ code.

AutoIT will certainly stay my go-to language but it's nice to have C++ for the heavy lifting when needed.

[JLogan3o13]

Have you tried adding the OpenFileDialog component to your C++ code?

https://msdn.microsoft.com/en-us/library/61097ykx(v=vs.110).aspx?cs-save-lang=1&cs-lang=cpp#code-snippet-1

[LarsJ]

It must have been exceedingly well written AutoIt code. Only about 12 times slower than compiled C++ code. Usually this kind of AutoIt code is 100 - 1000 times slower than compiled C++ code.

[BigDaddyO]

JLogan3o13:  I tried adding the OpenFileDialog many different ways but it's not recognized by the gcc mingw32 compiler.  I read that there is a switch I can add to the compiler to recognize it properly but I barely got the compiler working as-is.

 

LarsJ:  As for the AutoIT script, I did a lot of work testing each piece of the validation code with timers and switching code out to save microseconds where ever I could.  I really don't think it was possible to make it any faster.  as for my C++ code only being 12 times faster, it may just be that my C++ code sucks as it's my first attempt with that language.  Though, if I could figure out OpenMP so I could use more than one CPU core I could speed it up even more as it maxes out one core the entire time it's running.

 

I have lot's to learn, but I think I need to push my company to get me VisualStudio as this CodeBlocks and gcc mingw32 is just tough!

 

[UEZ]

My current favorite is FreeBasic, easy to learn and very fast!

My latest code in FB -> moving 69.696 pixels in 30fps on my notebook.  Just move your mouse over the image to see the effect.

 

PS: I know that you are not looking for such kind of things...

Edited October 12, 2016 by UEZ

[Gianni]

waw.. wonderful effect
very nice @UEZ!

here a similar effect in Javascript:

http://fabricjs.com/particles

p.s.
sorry for the little hijack...

[Danyfirex]

Hello. Here is a CodeBlock+GCC OpenFileDialog  Example.

 

OpenDialogSample.zip

Saludos

[LarsJ]

BigDaddyO, Do not take my comment too seriously. It was meant as a joke. I know that it's much harder to learn a language like C++ than AutoIt. If on the other hand, you have a good understanding of a language like AutoIt you can learn any language. It just takes more time and requires more reading. And a better language than C++ does not exist.

You can get a free Visual Studio here as you can use for learning.

Edited October 12, 2016 by LarsJ

[BrewManNH]

Don't want to start a huge discussion of which language is better, but I'm pretty sure that there are a number of better languages out there than C++.

Of course "better" is a relative term whose definition would change depending on what it is you're trying to do with the language. Such as, HTML is a far better language than C++ for writing web pages. But I wouldn't want to write high level programs in HTA. I'd bet that there are quite a few people that would argue that C# is a better language than C++ depending on what you're doing in it. Delphi would probably be as good as, if not better than, C++.

[UEZ]

16 hours ago, Chimp said:

waw.. wonderful effect
very nice @UEZ!

here a similar effect in Javascript:

http://fabricjs.com/particles

p.s.
sorry for the little hijack...

Thx for the link. Interesting stuff.

Edited October 13, 2016 by UEZ

[AndyG]

21 hours ago, BigDaddyO said:

 As for the AutoIT script, I did a lot of work testing each piece of the validation code with timers and switching code out to save microseconds where ever I could.  I really don't think it was possible to make it any faster.

For 99.9% of all applications, AutoIt is fast enough, just when it comes to GUI-related stuff! Not to mention that the creation of the GUI´s is very fast in AutoIt!

If you need SPEED, in the case of many calculations probably mostly in (several) nested loops, those calculations could be done in a "faster" language like C(++), FreeBasic or ANY other compiling language. Profile your script to find the "inner loop", extract these handful lines and put them into a DLL. Call this DLL from AutoIt and you have the advantage of a fast (GUI)-development and a fast calculation by compiled code.

It is NOT necessary to be a C(++)-Crack to do some calculations in C(++), and it is NOT necessary to use OOP! Replace "C(++)" with any other compiling language you want...

 

I prefer Assembler to speed up code, because mostly the runtime of the code can be done faster compared to a compiled code. Yes, the time to write (draw up/create) ASM-code is MUCH longer than typing some lines in C(++) or any other compiling language, but if i want my program REALLY fast, i analyze the code made by a compiler and "see" if the expected acceleration by ASM is worth the time. Most of the time i can speed up compiled code with handcrafted ASM by factor of 2 to 4 (and up). With the use of SSE/SIMD the improvement can be much higher! I have never seen compiled SSE/SIMD code which uses 4x32Bit in a XMM-Register to calculate for example 4 Pixels of a image-filter at the same time....

Most of the code in HLL´s is poorly programmed, and so it is not a surprise that the compiled result is not turned into gold by flipping some compiler-switches...

 

[BigDaddyO]

21 hours ago, Danyfirex said:

Hello. Here is a CodeBlock+GCC OpenFileDialog  Example.

 

OpenDialogSample.zip

Saludos

Thanks for the sample.  It works great within that sample project, but...

That code doesn't work with mine.  I even copied the contents of the main.cpp into a new blank projects main.cpp and it didn't work there either.

undefined reference to 'GetOpenFileNameA@4'

There must be some configuration setting somewhere that I can't find that allows it to work within your sample project.

Mike

 

EDIT:  Found the setting.

Projects > Build options...

Debug > Linker settings tab > add Comdlg32

 

I'm assuming I'll also need to do that for the Release item as well.

 

Edited October 13, 2016 by BigDaddyO
Figured out the setting

[Danyfirex]

Yes you need to add link libraries to Release version too.

 

Saludos

[BigDaddyO]

so...  just to make sure everyone knows i'm a total newbe with C++, I realized when adding that library link "thanks Danyfirex" that I was only compiling the debug version of my validation program.

After compiling the Release version and running it from my local drive instead of a network drive it runs in just over 10 minutes.  so, I am now running 36 times faster with the C++ code vs my original AutoIT code.

 

Mike

[kcvinu]

Hi,

I am a guy who started learning programming with vb6. I have stopped it after 3 months. Then a few years later i have started again with VB.Net. Things went cool. I learned the basics of variables , control flow and databases. But not learned about functions and classes and loops. At that time i heard about python. I have read that it's easy. I started learning python. It helped me to learn loops and functions.  There is web tool called Python Visualizer. It teach me how to use loops. But OOP in python was horrible for me. So i came back to VB.Net. Then i started learning OOP  in VB.Net and so many books helped me. After that, i need to learn the low level programming of windows. I met Win32 API but failed to learn. Then i learned AutoIt. It helped me to learn win32 API easily. it points me to inner secrets of windows. And the book of Charles Petzold helped me a lot. Then i learned the basics of FreeBasic, ThinBasic, FBSL.