StringRegExp, the script crashes, why?

[Jos]

The issue is that I haven't been able to replicate it in the latest Beta. The REGEX stuff was upgrade in one of the Betas.

- Changed: PCRE regular expression engine updated to 7.7.

We really need a script that replicates a crash in the Beta version before we can test/debug.

Jos

Edited August 25, 2008 by Jos

[HowToBen]

I did a several more tests.

The result is: It crashes as often with the beta as with the stable version.

Of 100 runs it crashes about 10 times.

I'm sorry, but I have no influence on your system.

I'm sure the new Beta didn't fix the problem.

Maybe you have to test it with the bigger Log-File to replicate a crash.

I don't know.

Ben

[Xenobiologist]

Hi,

can you strip the script down to the necessary parts. I will run it 100 times to see whether it crashes for me, too.

Mega

[Jos]

Ben,

Are you now talking about the script I stripped down and posted in this thread?

Jos

[Jos]

Just tested again but this time with the whole file you send me and that does crash with Beta.

I will created a bugreport and upload the whole packages so it can be looked at.

Jos

[HowToBen]

If anybody found out something new about the StirngRegExp problem.

please contact one of my college with the username "metris".

He took over all my tasks regarding AutoIt.

Thanks Ben

[Community On Patrol]

Hi HowToBen,

Just a note on Regular Expressions and AutoIt in case you didn't know.

AutoIt uses the PCRE (Perl Compatible Regular Expressions) engine.

If you are using google for tutorials or just searching for specifics, keep this in mind when looking for expressions or help on expressions.

Some things that may help you along with future questions or concerns:

QuickStart

Tutorials

RegExCoach

[Valik]

Well, I found the bug and it turns out I was right in my assessment. The bug has been fixed in PCRE, but there's no release that contains the fix.

I'm not sure how old the bug was in PCRE, but I can say this, I'm a little surprised it wasn't found quicker but at the same time not really. As this code demonstrates:

char *s = new char[6];
 s[8] = 'a';
 std::cout<<s[8]<<std::endl;

It's quite possible to access what appears to be out-of-bounds memory simply due to how dynamic memory is allocated. The only reason this crash didn't appear sooner is because the heap generally allocates more memory than requested and that memory is accessible (though you shouldn't, of course, because it's not guaranteed). I'm not entirely sure what's causing this crash - rather, why the buffer is suddenly not getting that normal extra space it usually does. My guess is that the heap is fragmented and it just happens to hit a perfectly sized block which leads to the crash.

Anyway, I'll patch our version of PCRE since it's a stupidly simple fix that I now feel safe doing since I see it's the official fix.

Edit: Jos, to answer your question asked on the tracker, to debug this, you do the following:

• Comment out AutoIt's anti-debugger check so you can attach a debugger on a Release build.
• Set the C++\General\Debug Information Format option to "Program Database" for Release builds for both AutoIt and PCRE (If you don't set it for PCRE, you can't see inside PCRE functions).
• Set the Linker\Debugging\Generate Debugging Info to "Yes" for Release builds.
• Build.
• Run the script using the newly built AutoIt and wait for the crash.
• Use the "Debug" button in the crash dialog and debug the crash in an instance of Visual Studio that has the AutoIt source loaded.

Voila, you can do postmortem crash analysis of a Release build crash and see the source where it crashed as well as most of the variables, the stack frame, et cetera. For me I was able to see all of the variables around the crash location in PCRE as well as the memory which allowed me to deduce what the bug was. From there I knew what to search for on Google which led me to the fix on the PCRE site. Edited September 25, 2008 by Valik

[SmOke_N]

Well, I found the bug and it turns out I was right in my assessment. The bug has been fixed in PCRE, but there's no release that contains the fix.

I'm not sure how old the bug was in PCRE, but I can say this, I'm a little surprised it wasn't found quicker but at the same time not really. As this code demonstrates:

char *s = new char[6];
 s[8] = 'a';
 std::cout<<s[8]<<std::endl;

It's quite possible to access what appears to be out-of-bounds memory simply due to how dynamic memory is allocated. The only reason this crash didn't appear sooner is because the heap generally allocates more memory than requested and that memory is accessible (though you shouldn't, of course, because it's not guaranteed). I'm not entirely sure what's causing this crash - rather, why the buffer is suddenly not getting that normal extra space it usually does. My guess is that the heap is fragmented and it just happens to hit a perfectly sized block which leads to the crash.

Anyway, I'll patch our version of PCRE since it's a stupidly simple fix that I now feel safe doing since I see it's the official fix.

Edit: Jos, to answer your question asked on the tracker, to debug this, you do the following:

• Comment out AutoIt's anti-debugger check so you can attach a debugger on a Release build.
• Set the C++\General\Debug Information Format option to "Program Database" for Release builds for both AutoIt and PCRE (If you don't set it for PCRE, you can't see inside PCRE functions).
• Set the Linker\Debugging\Generate Debugging Info to "Yes" for Release builds.
• Build.
• Run the script using the newly built AutoIt and wait for the crash.
• Use the "Debug" button in the crash dialog and debug the crash in an instance of Visual Studio that has the AutoIt source loaded.

Voila, you can do postmortem crash analysis of a Release build crash and see the source where it crashed as well as most of the variables, the stack frame, et cetera. For me I was able to see all of the variables around the crash location in PCRE as well as the memory which allowed me to deduce what the bug was. From there I knew what to search for on Google which led me to the fix on the PCRE site.

Nice work... Revision log was in July of 2008, I know that my last build for the PCRE library I use with other languages was 7.7 and that was in like April or May. The last revision was on August 25 as far as I can tell with no release, so I'm wondering now when that release/fix will come out for the rest of the world ... but nice to see Superman up there found us a fix for good ole' autoit >_< ... (Guess I'll be digging through the source code and recompiling my own now lol)

[Valik]

Ron, it's line 4698 of pcre_exec.c if you're using PCRE 7.7. It should be:

while (start_match < end_subject && !WAS_NEWLINE(start_match))

Which is just changing <= to <.

[SmOke_N]

Ron, it's line 4698 of pcre_exec.c if you're using PCRE 7.7. It should be:

while (start_match < end_subject && !WAS_NEWLINE(start_match))

Which is just changing <= to <.

I saw that thanks (and fixed >_< )

[Jos]

Good find!.

Edit: Jos, to answer your question asked on the tracker, to debug this, you do the following:

• Comment out AutoIt's anti-debugger check so you can attach a debugger on a Release build.
• Set the C++\General\Debug Information Format option to "Program Database" for Release builds for both AutoIt and PCRE (If you don't set it for PCRE, you can't see inside PCRE functions).
• Set the Linker\Debugging\Generate Debugging Info to "Yes" for Release builds.
• Build.
• Run the script using the newly built AutoIt and wait for the crash.
• Use the "Debug" button in the crash dialog and debug the crash in an instance of Visual Studio that has the AutoIt source loaded.

Voila, you can do postmortem crash analysis of a Release build crash and see the source where it crashed as well as most of the variables, the stack frame, et cetera. For me I was able to see all of the variables around the crash location in PCRE as well as the memory which allowed me to deduce what the bug was. From there I knew what to search for on Google which led me to the fix on the PCRE site.

Thanks... Learning something new every day.

Jos

[freakazoid]

Hello,

I realise that this "issue" has been fixed in one of the recent beta versions, but due to some other changes I'd like to use the latest release version for now. I noticed and got reports about random crashes from my application which extensively used regexp expressions to find matching windows/elements WinGetHandle("[REGEXPTITLE... with Opt("WinTitleMatchMode", 4). Am I correct to assume that this bug is a likely cause for this? If I switch down to TitleMatchMode 2 (I'm only using .*bla.* basically) is this based on a RegExp mechanism as well or using a different way which should be uneffected?

Edit: The reason why I strongly believe that the problem comes from the window titles is, that using the address that caused the exception and looking on it with ollydbg, some sort of parsing of the window titles was done.

Edited November 18, 2008 by freakazoid

[freakazoid]

Any chance for insights on this, at least on whether or not TitleMatchMode 2 uses regexp internally (and would be victim to the crash bug) or not?

[metris]

sorry for my late response.

Yes, with autoit v3.2.13.9-beta the crash don't appears.

Thanks for the excellent support.