Is a crash more powerful than an Exit?

[mLipok]

you mean this:

Local $b
For $a = 1 To 5556
    $b &= 0
Next
ClipPut($b)
StringRegExp($b, '(.)*')

btw. 5555 works OK.

EDIT: You can track this error.

Edited October 9, 2014 by mLipok

[Morthawt]

you mean this:

Local $b
For $a = 1 To 5556
    $b &= 0
Next
ClipPut($b)
StringRegExp($b, '(.)*')

btw. 5555 works OK.

EDIT: You can track this error.

It is weird that this would happen.

[guinness]

Not really weird, when you consider there are limitations  >> http://www.pcre.org/pcre.txt (search SIZE AND OTHER LIMITATIONS).

Edited October 9, 2014 by guinness

[Morthawt]

If it is a limitations thing, why does it work if you remove the brackets from either side of the dot? It should make zero difference as far as regex goes.

[JohnOne]

That is AutoIt crashing, not your script.

EDIT:

oops never noticed second page.

Edited October 9, 2014 by JohnOne

[guinness]

If it is a limitations thing, why does it work if you remove the brackets from either side of the dot? It should make zero difference as far as regex goes.

Because (.)* and .* are two different expressions. The second regexp is greedily selecting any character from zero to manycharacters, whereas the other is recursive.

[BrewManNH]

The error message you're getting -1073741571 in hex is 0xC00000FD which indicates a stack overflow error.

[mLipok]

Because (.)* and .* are two different expressions. The second regexp is greedily selecting any character from zero to manycharacters, whereas the other is recursive.

 

here is small example:

#include <array.au3>
Local $b
For $a = 1 To 100
    $b &= 0
Next
$b &= 2
ClipPut($b)
Local $aTest1 = StringRegExp($b, '(.)*2', 3)
_ArrayDisplay($aTest1, '$aTest1')
Local $aTest2 = StringRegExp($b, '.*2', 3)
_ArrayDisplay($aTest2, '$aTest2')

btw.

using this snippet i have another interesting observation.... but about this i post in few minutes (i think so)

[mLipok]

Question: 

if this error should occur every time, Or may be subject to the other "circumstances" ?

[jchd]

The stack overflow is the consequence of enormous backtracking which is built-in (volontarily here, carelessly at other times) in the pattern.

There is nothing surprising that the PCRE library outburst its stack space.

BTW you shouldn't blame PCRE in this case: PCRE is aimed towards portability over a large number of hardware/software platforms and can't massage stack handling so differently. It is always possible to double check available stack space before actually using it but that would penalize 99.99999% of users with no real benefit. Windows doesn't offer any possibility to specify stack space at program launch (contrary to Linux, for instance). So every Windows program is limited to the stack space parametrized at build time.

Given that we can efficiently process huge real-world subjects with very complex patterns with our PCRE implementation, I don't see a point allocating multiple Gb of stack space, which no practical situation will ever use.

EDIT: PCRE uses a good number of pattern optimizations at pattern compile time which you can disable by prefixing the pattern with "(*NO_START_OPT)". These clever optimizations don't waste cycles trying to handle maliciously backtracking patterns but instead focus on making practical patterns work faster.

Edited October 9, 2014 by jchd

[Morthawt]

The stack overflow is the consequence of enormous backtracking which is built-in (volontarily here, carelessly at other times) in the pattern.

There is nothing surprising that the PCRE library outburst its stack space.

BTW you shouldn't blame PCRE in this case: Windows doesn't offer any possibility to change stack space at program launch (contrary to Linux, for instance). So every Windows program is limited to the stack space parametrized at build time.

Given that we can process huge real-world subjects with very complex patterns with our PCRE implementation, I don't see a point allocating Gb of stack space which no practical situation will ever use.

If it has no negative downside, why wouldn't you? I do not know exactly what you are talking about here, but your last point seemed to be more of a "whats the point" rather than it shouldn't be done because of XYZ.

I was flummoxed for hours due to this problem. I was trying to write a script that detected function names and compiled them into a list and I was coding this at the top of another script that I wanted to experiment with some home-made obfuscation on and just had an exit before it to prevent that code actually running. So I told it to read it's own script file and boom.. I had the problem but it came and went, I would remove parts and it would crash, add a line back and it would not crash etc. Highly unstable.< 6k of data is not a lot of data to be causing this kind of problem. I could understand it if I was processing 20MB of data, but I wasn't. My tiny code to replicate the issue creates less than 6K of data and causes this crash.

 

EDIT

Also when I do testing of the two mentioned patterns using the site I always test my work on: http://regexpal.com/ it shows the same end result regardless of if I use the brackets or not and every reference I have read say that the brackets by themselves are only to group things up.

Edited October 9, 2014 by Morthawt

[jchd]

Whatever crazy (and useless in practice) stack size you're going to promote, it'd be dead easy to write a one-liner to make it overflow. So there must be a limit and it has to be sensible enough to allow sensible patterns to run. It doesn't need complex patterns to cause factorial-growth in backtracking, hence in required stack size. So even basic innocent-looking {subject, pattern} couple may overflow any fixed stack size.

For instance, it would only take a small increment in subject size to overflow a stack twice as large than default. Just a little larger and burst a stack ten times as large. Where is your limit? I'd rather keep Gb of OS storage cache live than allocate that much to a stack size that will never serve any purpose. Also users with limited available memory would be pissed of by such compile-time decision.

The correct answer is not using uselessly backtracking patterns. Laziness, atomic grouping and possessive modifier are the tools of choice to keep away from backtracking black holes.

PCRE is not JS flavor, even if it has an option to mimick it. Capturing parenthesis, well, ... capture substrings. If an engine doesn't reflect that, then it's badly flawed.

It's also very easy to cause an OS-level exception, e.g. passing an invalid pointer to some DLL or invoking stdcall instead of cdecl. All these situations are best avoided.

[guinness]

Perhaps give us a better idea of the regular expression you used, as (.)* has zero benefit in my humble opinion. In the 4 years I have been using AutoIt and regular expressions, I have never had a problem. Okay maybe once in PreExpand, but then I asked in the Forum and jchd came up with a proposed solution. So perhaps use their advice and knowledge to your advantage, instead of saying it's a problem when it really isn't.

Just for your benefit, every language has a limitation on how much can be pushed to a stack e.g. the call has a limit to stop infinite recursion taking place.

Possible reading: https://en.wikipedia.org/wiki/Stack_overflow

Edited October 10, 2014 by guinness

[mLipok]

Ok.
So maybe this one "stack overflow", can be catch by AutoIt and report this in @error like this "3 PCRE Stack overflow"

I think that such a solution would be very helpful, and would prevent unnecessary struggle "what and where am I doing wrong".

[jchd]

 

So maybe this one "stack overflow", can be catch by AutoIt

I hardly see how this could be done.

[Morthawt]

Perhaps give us a better idea of the regular expression you used, as (.)* has zero benefit in my humble opinion. In the 4 years I have been using AutoIt and regular expressions, I have never had a problem. Okay maybe once in PreExpand, but then I asked in the Forum and jchd came up with a proposed solution. So perhaps use their advice and knowledge to your advantage, instead of saying it's a problem when it really isn't.

Just for your benefit, every language has a limitation on how much can be pushed to a stack e.g. the call has a limit to stop infinite recursion taking place.

Possible reading: https://en.wikipedia.org/wiki/Stack_overflow

I do not have it any longer. It was all on the same source and I was more interested in finding the cause of the crash than continuing my failed experiment. So I just kept modifying it until I was left with that small nugget of code.

[trancexx]

I hardly see how this could be done.

That's actually easy. Interpreter can, at any point, be compiled to deal with the issue(s) using structured exception handling.

But really, AutoIt should never hard-crash for correct usage of built-in functions. I can't understand your point of view about regexp, it simply doesn't make sense to me. AutoIt runs on Windows, Windows limits available stack space, hence AutoIt's regexp shouldn't use stack for recursion particularly because regexp offers heap allocation for recursion.

You could say "but heap is slower", I would say "no it's not". How can you compare speed of crashing code to speed of the opposite?

[jchd]

PCRE crashing patterns are always tailored on purpose just to exhaust stack space. I don't call that "correct usage".

I will not be the one writing in the changelog: "StringRexExp[Replace] have been slown down to 1/3 of the speed you enjoyed in previous releases because some morons can and will write patterns guaranteed to exhaust stack space, irrespective of how large we make it. And we insist on giving them a nice message about their exploit."

AutoIt should it also use structured exception handling to catch every attempt to DllCall things with invalid pointers / structures / parameters as well? That or orphanned hooks and such?

AutoIt isn't #.

[Morthawt]

PCRE crashing patterns are always tailored on purpose just to exhaust stack space. I don't call that "correct usage".

I will not be the one writing in the changelog: "StringRexExp[Replace] have been slown down to 1/3 of the speed you enjoyed in previous releases because some morons can and will write patterns guaranteed to exhaust stack space, irrespective of how large we make it. And we insist on giving them a nice message about their exploit."

AutoIt should it also use structured exception handling to catch every attempt to DllCall things with invalid pointers / structures / parameters as well? That or orphanned hooks and such?

AutoIt isn't #.

 

Could you possibly be any more offensive? It has taken me ages to learn regex for the use of creating ban rules in TeamSpeak. It took me a while and I always check my work on web based testers so that I know what I am doing works. I was not purposefully trying to exhaust stack space (I have no idea what that even is) and I am not a moron. I was trying to process code to pull out a list of specific things when this happened. Just because I do not know every flavour of regular expression and can format things perfectly for every program I use that requires it does not make me a moron. I wouldn't expect this BS attitude from an MVP.

Edited October 10, 2014 by Morthawt

[Jos]

ok, guess you are done.

Maybe it would be wise not to take things too personal.

*click*