StringRegExp General Help

[AZJIO]

If at the end of the line " _" you must unite with the following line

[FireFox]

StringRegExpReplace($s, "_\r\n(\t{2})?", "")

Remains the problem with matching the following variables.

[AZJIO]

expandcollapse popup

#include <Array.au3>
$sText = _
        'Func Example1()' & @CRLF & _
        '    Local $toto = ''toto'', $tata = ''ta,$xaxa'', _ ; ,$comments' & @CRLF & _
        '    $msg,  $aState[2] = [$GUI_CHECKED, $GUI_UNCHECKED], $tutu' & @CRLF & _
        @CRLF & _
        '    Local $zzz = ''toto'', $xxx = ''tata'', _' & @CRLF & _
        '    $jjj, $titi = ''titi'',  $aScv[2] = ($GUI_CH, $GUI), $uuu = ''tutu''' & @CRLF & _
        @CRLF & _
        'EndFunc   ;==>Example1'
; $sText = StringRegExpReplace($sText, '(?m)^((?:[^''";]*([''"]).*?\2)*[^;]*);.*$', '\1') ; remove one-line comments
_stripComments($sText)
$sText = StringRegExpReplace($sText, '\h+(?=[\r\n\z])', '') ; remove space
$sText = StringRegExpReplace($sText, ' _(\r\n|\n|\r)', ' ')
$aText = StringRegExp($sText, '(?mi)^\h*(?:Local|Global)\h+(.+?)\s*$', 3)
If Not @error Then
    $sRes = ''
    For $i = 0 To UBound($aText) - 1
        $aText[$i] = StringStripWS($aText[$i], 8)
        $aText[$i] = StringRegExpReplace($aText[$i], '([''"]).*?\1', '')
        Do
            $aText[$i] = StringRegExpReplace($aText[$i], '\([^()]*?\)', '')
        Until Not @extended
        Do
            $aText[$i] = StringRegExpReplace($aText[$i], '\[[^\[\]]*?\]', '')
        Until Not @extended
        $aVar = StringRegExp(',' & $aText[$i], '(?<=,)\$\w+', 3)
        If Not @error Then
            For $j = 0 To UBound($aVar) - 1
                $sRes &= $aVar[$j] & @CRLF
            Next
        EndIf
        $sRes &= @CRLF
    Next
EndIf
MsgBox(0, "Yes?", $sRes)

; "Organize Includes"
Func _stripComments(ByRef $string)
    $string = StringRegExpReplace($string, "(?s)\r(?!\n)", '\r\n') ; Adds @LF in string, AZJIO
    ;Author: Prog@ndy
    $string = StringReplace(StringReplace($string, "#comments-start", "#cs", 0, 2), "#comments-end", "#ce", 0, 2)
    $string = StringRegExpReplace($string, "(?si)(\v|\A)\h*#cs\b.*?\v\h*#ce\b", '') ; remove simple block comments (Modified AZJIO)
    #Region remove nested block-comments
    Local $match, $depth, $offset, $start, $CommentsAfterce
    While 1
        $depth = 0
        $match = StringRegExp($string, "(?im)^\h*#cs\b", 1)
        $start = @extended
        If @error Then ExitLoop
        Do
            $match = StringRegExp($string, "(?im)^\h*#c([se])\b", 1, $offset)
            $offset = @extended
            Select
                Case @error
                    Return False
                Case $match[0] = "e"
                    $depth -= 1
                Case Else
                    $depth += 1
            EndSelect
        Until $depth < 1

        $string = StringLeft($string, $start - 4) & StringRegExpReplace(StringMid($string, $offset), ".*", '', 1)
;~         $string = StringLeft($string, $start-4) & StringMid($string, $offset)
    WEnd
    #EndRegion remove nested block-comments
    $string = StringRegExpReplace($string, '(?m)^((?:[^''";]*([''"]).*?\2)*[^;]*);.*$', '\1') ; remove one-line comments (недостаток - не работает если @CR без @LF)
    Return True
EndFunc   ;==>_stripComments

Edited March 21, 2013 by AZJIO

[FireFox]

Yes?

Yes ! Thank you very much

Br, FireFox.

[PhoenixXL]

Well the underscore part is above this simple implementation. There could be a followed comment, there might be a semi-colon inside the comment. The literals may have the Underscore or the semi-colon.

There are regexes available on this Forum (one is mine and the other is that of Guinness - what I have discovered).

Mine regex is in this Topic in the first post under advanced examples.

Simultaneously to get the variables, with the scope or Const and Static keyword, Guinness had a great idea and has a script

Furthermore, for the sake of simplicity use a negative assertion with Local|Global scope to eliminate the lines without them, and there you have it

[AZJIO]

PhoenixXL,

Check the comments. Updated

Edited March 20, 2013 by AZJIO

[guinness]

You can find my regular expressions in SciTE Jump in the _Functions.au3 UDF. All functions start with the prefix _Strip***.

[guinness]

Looking at AZJIO's post, it's pretty much how I do it, but I take into consideration Const, Static and Enum.

It's great to now be apart of the SRE elite.

[Melba23]

guinness,

It's great to now be apart of the SRE elite

But do the little beggars still make your brain bleed?

I reckon that unless you can produce a complicated RegEx "off the cuff" without turning the locality "blue" with frustration, you are still a neophyte - in which category I firmly place myself for those very reasons!

M23

[guinness]

But do the little beggars still make your brain bleed?

I was about to throw my laptop across the room yesterday, but figured it wasn't a good idea as people were on the other side. I was trying to break down a Robjong SRE into parts as to further understand regular exp<b></b>ressi&#111;ns. That guy creates some complex stuff!

Edit: I've also been working on something today and yesterday, so I will post here once finished.

Edited March 20, 2013 by guinness

[guinness]

What I've been working on ...

expandcollapse popup

#include <Array.au3>

; Stripped Au3 file with all continuation lines merged and comments stripped.
Local Const $sData = 'Local $sData = "", $iExample = 0' & @CRLF & _
        'Local Const $iLConst = 1000' & @CRLF & _
        'Const $iConst = 1000' & @CRLF & _
        'Local Enum Step *2 $eLEnum_1, $eLEnum_2' & @CRLF & _
        'Enum Step *2 $eEnum_1, $eEnum_2' & @CRLF & _
        'Local Static $sStatic = "Local Static"' & @CRLF & _
        'Static $sStatic = "Static"' & @CRLF

; Display Local, Global, Const, Enum and Static but don't include the keyword.
Local Const $aScoped = _GetVariablesScoped($sData, 2 + 4 + 8 + 16) ; 1 + 2 + 4 + 8 + 16
_ArrayDisplay($aScoped)

Func _GetVariablesScoped(ByRef Const $sData, $iFlags = Default)
    Local Enum Step *2 $eVARSCOPE_INCLUDESCOPE, $eVARSCOPE_LGONLY, $eVARSCOPE_CONST, $eVARSCOPE_ENUM, $eVARSCOPE_STATIC ; 1, 2, 4, 8, 16
    #cs
        Possible combinations:
        Local
        Local Const
        Local Enum
        Local Static
        Global
        Global Const
        Global Enum
        Global Static
        
        Const
        Enum [Step */-/+Int]
        Static
        
        [Const] ByRef [Const]
    #ce
    If $iFlags = Default Or $iFlags = 0 Then
        $iFlags = BitOR($eVARSCOPE_INCLUDESCOPE, $eVARSCOPE_LGONLY, $eVARSCOPE_CONST, $eVARSCOPE_ENUM, $eVARSCOPE_STATIC)
    EndIf

    Local $sPatternAdditional = '', $sPatternEnumStep = '', $sPatternIncludeScope = '?:', $sSREPattern = ''
    If BitAND($iFlags, $eVARSCOPE_INCLUDESCOPE) Then
        $sPatternIncludeScope = ''
    EndIf
    If $iFlags = $eVARSCOPE_LGONLY Or $iFlags = $eVARSCOPE_INCLUDESCOPE + $eVARSCOPE_LGONLY Then
        $sSREPattern = '(?im:^\h*(?=Global|Local)(' & $sPatternIncludeScope & 'Global|Local)\h+(?!Const|Enum|Static))([^\r\n]+)'
    Else
        If BitAND($iFlags, $eVARSCOPE_CONST) Then
            $sPatternAdditional &= '|Const'
        EndIf
        If BitAND($iFlags, $eVARSCOPE_ENUM) Then
            $sPatternAdditional &= '|Enum'
            $sPatternEnumStep = '(?:(?<=Enum)\h+Step\h+[+*-]\d+)?'
        EndIf
        If BitAND($iFlags, $eVARSCOPE_STATIC) Then
            $sPatternAdditional &= '|Static'
        EndIf

        Local Const $sPatternStripped = StringTrimLeft($sPatternAdditional, StringLen('|'))
        If BitAND($iFlags, $eVARSCOPE_LGONLY) Then
            $sSREPattern = '(?im:^\h*(?=Global|Local' & $sPatternAdditional & ')(' & $sPatternIncludeScope & 'Global|Local)?\h*(' & $sPatternIncludeScope & $sPatternStripped & ')?' & $sPatternEnumStep & '\h*)([^\r\n]+)'
        Else
            $sSREPattern = '(?im:^\h*(?=Global|Local' & $sPatternAdditional & ')(' & $sPatternIncludeScope & 'Global|Local)?\h*(?=' & $sPatternStripped & ')(' & $sPatternIncludeScope & $sPatternStripped & ')' & $sPatternEnumStep & '\h*)([^\r\n]+)'
        EndIf
    EndIf

    ConsoleWrite('SRE Pattern: ' & $sSREPattern & @CRLF)
    Return StringRegExp($sData, $sSREPattern, 3)
EndFunc   ;==>_GetVariablesScoped

Edited March 20, 2013 by guinness

[guinness]

I've updated the last post by including a flag to include the keyword used.

[guinness]

Thanks AZJIO for "(?<=,)$w+", I was parsing the variable string using the following ...

expandcollapse popup

Func _GetVariablesScopedParse(ByRef $sData, ByRef $aArray)
    Local Enum $ePARSE_CONTINUE, $ePARSE_VARIABLE
    Local $fIgnoreVars = False, _
            $iDimension = $aArray[0], $iIndex = 0, $iParenthesis = 0, $iState = $ePARSE_CONTINUE, _
            $sChr = '', $sStack = ''

    $sData &= ' '
    While Not ($iIndex = StringLen($sData))
        $iIndex += 1
        $sChr = StringMid($sData, $iIndex, 1)

        Switch $iState
            Case $ePARSE_CONTINUE
                Switch $sChr
                    Case '(', '['
                        $iParenthesis += 1
                    Case ')', ']'
                        $iParenthesis -= 1
                        If Not $iParenthesis And $fIgnoreVars Then $fIgnoreVars = False
                    Case '='
                        $fIgnoreVars = True
                    Case ','
                        If Not $iParenthesis And $fIgnoreVars Then $fIgnoreVars = False
                    Case '$'
                        $iState = $ePARSE_VARIABLE
                        $sStack &= $sChr
                EndSwitch
            Case $ePARSE_VARIABLE
                Switch $sChr
                    Case 'A' To 'Z', 'a' To 'z', '0' To '9', '_', '$'
                        $sStack &= $sChr
                    Case Else
                        $sStack = StringRegExpReplace($sStack, '^(\$\w+).*', '\1')
                        If Not $iParenthesis And Not ($sStack == '$') And Not ($sStack == '') And Not $fIgnoreVars Then
                            If ($aArray[0] + 1) >= $iDimension Then
                                $iDimension = Ceiling(($aArray[0] + 1) * 1.2)
                                ReDim $aArray[$iDimension]
                            EndIf
                            $aArray[0] += 1
                            $aArray[$aArray[0]] = $sStack
                        EndIf
                        $sStack = ''
                        $iState = $ePARSE_CONTINUE
                EndSwitch
                If $iState = $ePARSE_CONTINUE Then
                    $iIndex -= 1
                EndIf
        EndSwitch
    WEnd
EndFunc   ;==>_GetVariablesScopedParse

But it was very slow. I will reference you in the next version of SciTE Jump.

[guinness]

Actually AZIO, beware that Local Const $aState[2] = [$GUI_CHECKED, $GUI_UNCHECKED] would trigger $GUI_UNCHECKED to be recognised as a declared variable too. Best to strip anything in between [] and ().

[BrewManNH]

Would you want to strip variables inside () or is there a separate check for Function parameters?

[guinness]

Would you want to strip variables inside () or is there a separate check for Function parameters?

Aren't we just discussing variables that are exclusively declared?

[BrewManNH]

I haven't looked at the code, just was commenting on what you were saying about $gui_unchecked. Does his regex show it as being declared? Because what would happen in a function if you used $someparameter = $gui_unchecked? I'm just curious.

[guinness]

We're talking about variables declared with the Local/Global keyword not function parameters. Try the function.

I also deal with the function parameters before stripping.

Note: Everything I'm talking about is available in SciTE Jump, for FREE.

Edited March 21, 2013 by guinness

[AZJIO]

guinness,

I added the removal of brackets. But do not think that it will be faster.

[guinness]

AZJIO, try this on your version.

Local $sText = 'Local $GUI_CHECKED, $GUI_NOTDECLARED, $GUI_UNCHECKED, $sSomeVar, $sSomeVars' ^ @CRLF & _
        'Local $sData = "", $iExample = 0' & @CRLF & _
        'Local Const $iLConst = 1000' & @CRLF & _
        'Const $iConst = 1000' & @CRLF & _
        'Local Enum Step *2 $eLEnum_1, $eLEnum_2' & @CRLF & _
        'Enum Step *2 $eEnum_1, $eEnum_2' & @CRLF & _
        'Local Static $sStatic = "Local Static"' & @CRLF & _
        'Static $sStatic = "Static"' & @CRLF & _
        'Local $aArray[10][2] = [[$GUI_CHECKED, 100], [$sSomeVar, $GUI_NOTDECLARED]], $aSRE[2] = [$GUI_UNCHECKED, $sSomeVars]' & @CRLF & _
        'Local $aSRE_2[2] = [$GUI_UNCHECKED, $sSomeVars]' & @CRLF & _
        'Local Const $iStringLeft = StringLeft($sData, StringInStr("", "")), $iStringRight = StringRight("", 100)' & @CRLF