StringRegExp General Help

[Nessie]

Hi,

I'm stuck with a simple regexp, however it does not work.

I have an url, and I want to extract the file name from it, so it will strip the caracters until the last slash and will stop from that until a query "?" or a fragment "#"

This is what I've done so far :

Local $s = "", $a = 0

$s = "http://www.example.com/path/?toto#fragment"

;~ $a = StringRegExp($s, "/([^/]*$)", 1) ;1st part, works
;~ $a = StringRegExp($s, "(.*?)[#|\?]", 1) ;2nd part, works

$a = StringRegExp($s, "/([^/]*$)[#|\?]", 1) ;both: does not work.

ConsoleWrite($a[0] & @crlf)

Thanks for anyhelp.

Br, FireFox.

So in that case do you want to grab "toto" ?

Hi!

[FireFox]

So in that case do you want to grab "toto" ?

No, from my example the result would be blank.

You let me notice that it's not the best example to be understandable.

Edit :

$s = "http://www.example.com/path/index.php?toto#fragment"

;should return index.php

Br, FireFox.

Edited March 14, 2013 by FireFox

[Nessie]

If extension is always .php you could do something like that:

Local $s = "", $a = 0

$s = "http://www.example.com/path/index.php?toto#fragment"
$a = StringRegExp($s, "[^/]*(?=\.php.*)", 1) ;both: does not work.
ConsoleWrite($a[0] & @crlf)

Hi!

[FireFox]

If extension is always .php you could do something like that:

Thanks but no it's not the case, it can be anything.

[Nessie]

Thanks but no it's not the case, it can be anything.

What about that?

Local $s = "", $a = 0

$s = "http://www.example.com/path/index.php?toto#fragment"
$a = StringRegExp($s, "[^/]*(?=\.?\?)", 1) ;both: does not work.
ConsoleWrite($a[0] & @crlf)

Hi!

[FireFox]

Yup thank you, it should do the trick.

I would like someone to explain me why my regexp does not work in spite of the two first do?

[PhoenixXL]

I have explained your problem with a possible easy solution. The explanation is full and in detail. Ask if you still don't understand what is happening.

QuickTip

• You cannot combine RegEx(es) to make a combined search (you will understand what I mean by reading the code thoroughly)
• There is no need to escape special chars except # inside a character set (this is even explained in the following code)

expandcollapse popup

#include <Array.au3>
Local $s = "", $a = 0

#cs
[\?] is the same as [?]
no need to escape special characters inside a set.
With the exception of any \# special chars like to match either "\" or "r"
use [\\r] or [r\\]
#ce


$s = "http://www.example.com/path/index.php?toto#fragment"

$a = StringRegExp($s, "/([^/]*$)", 1) ;1st part, works
;/ (Non-Capturing) Slash | Non-Capturing Match : / (this is the last "/" since you used "$")
;([^/]*$) (capturing group) consisting of "non-slash" characters (greedy) after which a line-break is present | Capturing Match : index.php?toto#fragment

ConsoleWrite($a[0] & @CRLF)

$a = StringRegExp($s, "(.*?)[#?]", 1) ;2nd part, works
;(.*?) (Capturing Match) Anything from the start except line-feed(Lazy) | Capturing Match : http://www.example.com/path/index.php
;[#?] (non-capturing) after which "#" or "?" is present | Non-Capturing Match : ?

ConsoleWrite($a[0] & @CRLF)

$a = StringRegExp($s, "/([^/]*$)[#?]", 1) ;both: does not work.
If Not IsArray($a) Then ConsoleWrite("It is not an Array" & @CRLF)
;/ (Non-Capturing) Slash | Non-Capturing Match : / (this is the last "/" since you used "$")
;([^/]*$) (capturing group) consisting of "non-slash" characters (greedy) after which a EOL is present | Capturing Match : index.php?toto#fragment
;[#?] either match # or ?. Now here is the problem or error. Everything is already matched. No Hash or Question-mark is left to match. Hence an array is not returned.



;.....Solution...====================================================================================================================================================================================================================================


$a = StringRegExp($s, "([^#?]+)(.*)", 1) ;This would work
;([^#?]+) (capturing group) consisting of "non-(question-mark and hash)" characters (greedy) | Capturing Match : http://www.example.com/path/index.php
;(.*) (capturing group)Logic: We know that this is either a Hash or a Question-mark, so lets match everything till the end. We know that "." will not match any line-feed character | Capturing Match : ?toto#fragment

ConsoleWrite("-: " & $a[0] & @CRLF & "-: " & $a[1] & @CRLF)

;Note that if you just want to receive "index.php" in the zero-index
;Add a this group in the beginning (?:.*/)
;What this would do ?
;(?:.*/) : (non-capturing) ".*" will consume the full string. Then when "/" is encountered the regex-engine will backtrace to the last "/" and hence the non-captuing match would be "http://www.example.com/path/".
;Thereafter the remaining regex will match only "index.php" & "?toto#fragment"

Hope you figure out your problems. Thumbs up if helped.

Regards

Edited March 14, 2013 by PhoenixXL

[FireFox]

Thank you very much for your answer.

This are the comments I don't understand :

Everything is already matched. No Hash or Question-mark is left to match.

Why?

Anything from the start except line-feed(Lazy)

What do you mean by Lazy ?

[ ] means that it will be captured and ( ) means that it won't ? But there is capturing and non capturing groups... What about the first slash non captured and outside [ ] and ( ) ?

Br, FireFox.

[PhoenixXL]

Everything inside a () is captured. If the set [] is outside of any capturing group () or inside any non-capturing group (?: ) it won't be captured, it would only be matched.

If you don't know the difference b/w a Lazy(*?)and Greedy(*) quantifier look here orelse further explanation would be difficult ..!

In short

• Greedy : match everything which satisfies the pattern. After match if the next pattern matches, the engine goes on but otherwise the characters matched by the greedy operator is back-traced(given back one by one) till the next pattern is satisfied
• Lazy : match once and check if rest of the pattern satisfies. If yes don't match any more, if no, match one more and check again.

What about the first slash non captured and outside [ ] and ( ) ?

It is just matched not captured.Anything not in capturing group is just matched

http//www.example.com/path/index.php?toto#fragment. The slash in bold and underline is matched in your regex(which failed).

I would provide a step-wise decode. Hope then you could understand deeper. Hold a while

Edited March 14, 2013 by PhoenixXL

[FireFox]

Everything inside a () is matched. If the set [] is outside of any capturing group () or inside any non-capturing group (? it won't be captured.

If you don't know the difference b/w a Lazy and Greedy quantifier look here orelse further explanation would be difficult ..!

I understood everything, thanks !

The helpfile is more unhelpful that it can be for this function, I don't know if it's planned to update it; but someone like you should explain all these things.

I would provide a step-wise decode. Hope then you could understand deeper. Hold a while

okay

[jchd]

@FireFox,

Read the PCRE documentation pointed to in my .sig

[PhoenixXL]

Now finally ...

The full Decode with step-wise explanation.

Check the code.

expandcollapse popup

#region -Pattern Decode
;([^#?]+) (capturing group) consisting of "non-(question-mark and hash)" characters (greedy) | Capturing Match : http://www.example.com/path/index.php
;(.*) (capturing group)Logic: We know that this is either a Hash or a Question-mark, so lets match everything till the end. We know that "." will not match any line-feed character | Capturing Match : ?toto#fragment
#endregion

#include <Array.au3>

$a = StringRegExp("http://www.example.com/path/index.php?toto#fragment", "/([^/]*$)[#?]", 1) ;This won't work...
_ArrayDisplay( $a )


#region -Full Decode of Engine-
;======= First Attempt ===========
; Search for "/"
;Fail at 1 - "h", at 2 - "t", at 3 - "t", at 4 - "p", goes till 6

;======= Sixth Attempt ===========
;Match found "/" at 6
; Search for "[^/]*" greedily i.e. match everything which satisfies the pattern
;Fail 7 - "/". Match Fails here.
;........Again Start

;======= Seventh Attempt ===========
; Search for "/"
;Match Found at 7 - "/"
; Search for "[^/]*" greedily i.e. match everything which satisfies the pattern
;Match found at 8,9,10,11.....22.Char 22 is "m" hence this pattern has consumed "www.example.com". After "m", the 23rd character is a slash "/" which doesn't satisfy the pattern. Hence the engine moves to the next pattern sequence.
; Search for $ (ie End Of File). This is an assertion(a different case). This doesn't match/consume characters nor do it capture them.
;Fail at 23 which is "/" and not EOF. Match Fails here.
;........Again Start

;======= Eighth Attempt ===========
;same goes as seventh attempt. This is just from 23rd char "/" to 28th char "/" which is again another slash but not a EOF character.
;........Again Start

;======= Nineth Attempt ===========
; Search for "/"
;Match Found at 28 - "/"
; Search for "[^/]*" greedily i.e. match everything which satisfies the pattern
;Match found at 29,30,31.....51.Char 51 is "t" hence this pattern has consumed "index.php?toto#fragment". After "t", the EOF is reached. The Engine stops here.
; Search for $ (ie End Of File). This is an assertion(a different case). This doesn't match/consume characters nor do it capture them.
;After 51st char the EOF is reached hence the assertion is fulfilled
; Search for "[#?]" ie either "#" or "?"
;Now every character is consumed, nothing left to start again or back-trace.

;========== All the Attempts have been made. A match satisfing the complete pattern sequence was not found. The Engine quits here with an error =========================

#endregion -Full Decode of Engine-


;Now try with this pattern and you would get everything if you understood.
;......... "/([^/]*$)" ............

Hope now you understand.

Thumbs up if helped.

Regards

Edited March 14, 2013 by PhoenixXL

[FireFox]

I'm going to read jchd's PCRE doc, and I will come back if I have any question, it's yet hard to clearly understand "the subtleties".

[jchd]

I just added a link to good regexp tutorial. PCRE official docs are a reference not an actual tutorial.

[PhoenixXL]

Yup, I found it to be the best tutorials over the internet

[guinness]

Here's one for the gurus which I'm working on for SciTE Jump. BUT it doesn't work on nested loops.

Local $sData =  FileRead(@ScriptFullPath)
 _StripForVariables($sData)
 ConsoleWrite($sData & @CRLF)
For $i = 1 To 2 ; Strip any reference of $i in the For loop.
    For $j = 1 to 2 ; Strip any reference of $j in the For loop.
        $j = $j
    Next
    $i = $i
Next

; Strip For loop variables.
Func _StripForVariables(ByRef $sData)
    Local $aArray = StringRegExp('For $ = 0' & @CRLF & 'Next' & @CRLF & $sData, '(?ims)(^\h*For\h*\$(\w+)?.+?\h*Next)', 3), _
            $sReplace = ''
    $aArray[0] = UBound($aArray) - 1
    For $i = 1 To $aArray[0] Step 2
        $sReplace = StringRegExpReplace($aArray[$i], '\$\b' & $aArray[$i + 1] & '\b', '0')
        If @extended > 0 Then
            $sData = StringReplace($sData, $aArray[$i], $sReplace)
        EndIf
    Next
EndFunc   ;==>_StripForVariables

Edit: One way is to loop through the script file.

expandcollapse popup

#include <Array.au3>

Local $sData = FileRead(@ScriptFullPath)
_StripForVariables($sData)

For $i = 1 To 2 ; Strip any reference of $i in the For loop.
    For $j = 1 To 2 ; Strip any reference of $j in the For loop.
        $j = $j
    Next
    $i = $i
    ConsoleWrite('')
Next

; Strip For loop variables.
Func _StripForVariables(ByRef $sData)
    Local $aFileRead = StringSplit($sData, @CRLF, 1)
    Local $aReturn[$aFileRead[0] + 1], $iIndex = 0
    For $i = 1 To $aFileRead[0]
        Select
            Case StringRegExp($aFileRead[$i], '^\h*For\h*\$\w+') = 1
                If $iIndex = 0 Then
                    $aReturn[0] += 1
                    $iIndex += 1
                Else
                    $iIndex += 1
                EndIf
                $aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
            Case StringRegExp($aFileRead[$i], '^\h*Next\h*') = 1
                $iIndex -= 1
                $aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
            Case $iIndex > 0
                $aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
        EndSelect
    Next
    ReDim $aReturn[$aReturn[0] + 1]
    _ArrayDisplay($aReturn)
EndFunc   ;==>_StripForVariables

Edited March 15, 2013 by guinness

[guinness]

Well, here is a possible workaround.

expandcollapse popup

Local $sData = FileRead(@ScriptFullPath)
_StripForVariables($sData)
ConsoleWrite($sData)

For $i = 1 To 2 ; Strip any reference of $i in the For loop.
    For $j = 1 To 2 ; Strip any reference of $j in the For loop.
        $j = $j
    Next
    $i = $i
    ConsoleWrite('')
Next

; Strip For loop variables.
Func _StripForVariables(ByRef $sData)
    Local $aFileRead = StringSplit($sData, @CRLF, 1)
    Local $aReturn[$aFileRead[0] + 1], $iIndex = 0
    For $i = 1 To $aFileRead[0]
        Select
            Case StringRegExp($aFileRead[$i], '^\h*For\h*\$\w+') = 1
                If $iIndex = 0 Then
                    $aReturn[0] += 1
                    $iIndex += 1
                Else
                    $iIndex += 1
                EndIf
                $aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
            Case StringRegExp($aFileRead[$i], '^\h*Next\h*') = 1
                $iIndex -= 1
                $aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
            Case $iIndex > 0
                $aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
        EndSelect
    Next
    ReDim $aReturn[$aReturn[0] + 1]
    Local $aArray = 0, $sReplace = ''
    For $i = 1 To $aReturn[0]
        $aArray = StringRegExp('For $i = 0' & @CRLF & $aReturn[$i], '(?im)^\h*For\h*\$(\w+)', 3)
        $aArray[0] = UBound($aArray) - 1
        $sReplace = $aReturn[$i]
        For $j = 1 To $aArray[0]
            $sReplace = StringRegExpReplace($sReplace, '\$\b' & $aArray[$j] & '\b', '0')
        Next
        $sData = StringReplace($sData, $aReturn[$i], $sReplace)
    Next
EndFunc   ;==>_StripForVariables

[PhoenixXL]

Do you want to delete all the vars inside the For Loop ?

[guinness]

Only those that are declared in For loop scope e.g.

For $DeleteRefOfThis = 0 To UBound($aArray) - 1
$aArray[$DeleteRefOfThis] = 0
Next

Run the example above to see what I mean.

[PhoenixXL]

Here is an error I found in your workaround

expandcollapse popup

Local $sData = FileRead(@ScriptFullPath)
_StripForVariables($sData)
ConsoleWrite($sData)

For $i = 1 To 2 ; Strip any reference of $i in the For loop.
For $j = 1 To 2 ; Strip any reference of $j in the For loop.
$j = $j
Next
$i = $i
$x = 0
$j = 2
ConsoleWrite('')
Next

; Strip For loop variables.
Func _StripForVariables(ByRef $sData)
Local $aFileRead = StringSplit($sData, @CRLF, 1)
Local $aReturn[$aFileRead[0] + 1], $iIndex = 0
For $i = 1 To $aFileRead[0]
Select
Case StringRegExp($aFileRead[$i], '^\h*For\h*\$\w+') = 1
If $iIndex = 0 Then
$aReturn[0] += 1
$iIndex += 1
Else
$iIndex += 1
EndIf
$aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
Case StringRegExp($aFileRead[$i], '^\h*Next\h*') = 1
$iIndex -= 1
$aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
Case $iIndex > 0
$aReturn[$aReturn[0]] &= $aFileRead[$i] & @CRLF
EndSelect
Next
ReDim $aReturn[$aReturn[0] + 1]
Local $aArray = 0, $sReplace = ''
For $i = 1 To $aReturn[0]
$aArray = StringRegExp('For $i = 0' & @CRLF & $aReturn[$i], '(?im)^\h*For\h*\$(\w+)', 3)
$aArray[0] = UBound($aArray) - 1
$sReplace = $aReturn[$i]
For $j = 1 To $aArray[0]
$sReplace = StringRegExpReplace($sReplace, '\$\b' & $aArray[$j] & '\b', '0')
Next
$sData = StringReplace($sData, $aReturn[$i], $sReplace)
Next
EndFunc   ;==>_StripForVariables

The $j present in the second For/Next loop is replaced also in the first level For/Next loop.

Sometime ago, I made a regex for nested structures. It works well with your requirement

Have a look

expandcollapse popup

#include <Array.au3>

Local $sData = FileRead(@ScriptFullPath)
_StripForVariables($sData)
ConsoleWrite($sData & @CR)


For $i = 1 To 2 ; Strip any reference of $i in the For loop.
For $j = 1 To 2 ; Strip any reference of $j in the For loop.
$j = $j
Next
$i = $i
$x = 1
$j = 2
ConsoleWrite('')
Next

; Strip For loop variables.
Func _StripForVariables(ByRef $sData, $sReplace = "0")
Local $iLevel = 1
Local $aData = SplitToArray($sData, $iLevel)
#cs Now
$aData[0] = The outer For-Next Loops
$aData[1] = The innermost For-Next Loop.
$aData[2] = The remaining part of the outer For-Next loops.
#ce
If $aData = -1 Then Return ;No For Next Loop

Local $sVar ;Var Name
While IsArray($aData)
_ArrayDisplay($aData)
;Get the Variable name of the innermost element
$sVar = StringRegExp($aData[1], '(?mi)^\h*for\h+(\$\w+)', 3)
$sVar = $sVar[0]
;Lets Replace the Var everywhere in the innermost For-Next Loop.
$aData[1] = StringRegExpReplace($aData[1], "\" & $sVar, $sReplace)
;Wrap up the Array, make the new string.
$sData = _ArrayToString($aData, '')

;Next Level
$iLevel += 1
;Again for the next nested element.
$aData = SplitToArray($sData, $iLevel)
WEnd

;Done
Return 1
EndFunc ;==>_StripForVariables


Func SplitToArray($tStructure, $iLevel = 1, $Start = '^\h*For', $S_End = '^\h*Next')
StringRegExpReplace($tStructure, "(?mi)^\h*" & $Start, '')
Local $iCount = @extended ;Get the number of starting "TAG"
$iCount -= $iLevel ;subtract the count
If $iCount < 0 Then Return -1

Local $sPattern
If $iCount > 0 Then
$sPattern = _
'(?ims)((?:' & $Start & '.*?){' & $iCount & '})' & _;The string before the last Tag [1 group]
'(' & $Start & '.*?' & $S_End & ')' & _ ;The innermost tag [2 group]
'((?:.*?' & $S_End & '){' & $iCount & '})'
Else
$sPattern = '(?s)(^)(.*)($)' ;Capture everything in the second group if $iCount = 0
EndIf

;~ ConsoleWrite($sPattern & @CR)
;Splits the string such that the 1-Index of the Array is the innermost element, the array is 2-based.
Local $aRet = StringRegExp($tStructure, $sPattern, 3) ;The rest of the Stirng [3 group]
Return $aRet
EndFunc ;==>SplitToArray

Please give feedback of the regex.

Regards

Edited March 16, 2013 by PhoenixXL