Capturing Array into XML tagging

[KickStarter15]

Hi Experts,

I've been practicing RegExp() starting when @mikell help👍 and provided me the solution on my first related concern found in this link  where it solved my concern. However, during my practice, I tried doing the below RegExp() to get the array I need from the string. The problem is, I'm not able to distribute the array to it's desired element.

This is the code so far😅:

expandcollapse popup

#include <FileConstants.au3>
#include <WinAPIFiles.au3>
#Include <Array.au3>
#include <File.au3>
#include <MsgBoxConstants.au3>

Local $astr[2] = ["Wagner W., Raudsepp M., Holtz P., Sen R., (2017) Essence politics Identity work and stereotyping in inter-group relations. Haward C., Matt C., The social psychology of everyday politics. London. Routledge. 121-150.", _
"Wagner W., Raudsepp M., Holtz P., Sen R., (2017) Essence politics Identity work and stereotyping in inter-group relations. Haward C., Matt C., The social psychology of everyday politics. London. Routledge."] ; no pages with error

   $n = 0
   For $str In $astr

   $res = StringRegExp($str, '(*UCP)(?x) (?|  ' & _
      '([[:alpha:]]+)\h([A-Z][^,]+),\h? ' & _          ; names
      ' |  ([[a-z\h.]+),\h? ' & _               ; et al
      ' |  \((\d+)\)\.?\h? ' & _                ; year
      ' |  ([A-Za-z][^.]+.\h) ' & _             ; ChapterTitle
      ' |  ([A-Z][^.,]+[^.,]\h?) ' & _          ; Eds.
      ' |  ([A-Z][^.]+\.)\h? ' & _              ; PubLoc, Name
      ' |  (\d+[\(\);.-]) ' & _                 ; vol, issue, pages
      ' |  (?:https?://)?(\w+\S+)$) ' , 3)                 ; url

   $n += 1
     _ArrayDisplay($res, $n)

   Local $i = 0, $s = '<File xml:id="1110">' & @crlf & _
                        '<citation type="journal" xml:id="00001">'& @crlf

   While StringRegExp($res[$i], '^[A-Z]')
      $s &= '<Person>' & '<familyName>' & $res[$i] & '</familyName>, ' & _
                     '<givenNames>' & $res[$i+1] & '</givenNames>' & '</Person>, '
      $i += 2
   Wend

   If StringRegExp($res[$i], '^[a-z\h.]+$') Then
       $s &= $res[$i] & @crlf
       $i += 1
   EndIf

   If $i < UBound($res) AND StringIsDigit($res[$i]) Then
      $s &= '(<pubYear year="' & $res[$i] & '">' & $res[$i] & '</pubYear>);' & @crlf
      $i += 1
   EndIf

   If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<chapterTitle>' & $res[$i] & '</chapterTitle>. ' & @crlf
      $i += 1
   EndIf

   While StringRegExp($res[$i], '^[A-Z]')
      $s &= '<editor>' & '<givenNames>' & $res[$i] & '</givenNames>, ' & _
                     '<familyName>' & $res[$i+1] & '</familyName>' & '</editor>, '
      $i += 2
   Wend

   If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<OtherTitle>' & $res[$i] & '</OtherTitle>. '
      $i += 1
   EndIf
   If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<Location>' & $res[$i] & '</Location>. '
      $i += 1
   EndIf

   If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<LocationName>' & $res[$i] & '</LocationName>. '
      $i += 1
   EndIf
   If $i < UBound($res) AND StringRegExp($res[$i], '^\d+-$') Then
       $s &= '<pageFirst>' & StringTrimRight($res[$i], 1) & '</pageFirst>'
       $i += 1
   EndIf
   If $i < UBound($res) AND StringRegExp($res[$i], '^\d+\.$') Then
       $s &= '‐<pageLast>' & StringTrimRight($res[$i], 1) & '</pageLast>.'
       $i += 1
   EndIf
   If $i < UBound($res) AND StringInStr($res[$i], "doi") > 0 Then  ;<<<<<<
       $s &= '<url href="' & $res[$i] & '">' & $res[$i] & '</url>'
       $i += 1
   ElseIf $i < UBound($res) AND StringInStr($res[$i], "www") > 0 Then
       $s &= '<url href="' & $res[$i] & '">' & $res[$i] & '</url>'
       $i += 1
   ElseIf $i < UBound($res) AND StringInStr($res[$i], "http") > 0 Then
       $s &= '<url href="' & $res[$i] & '">' & $res[$i] & '</url>'
       $i += 1
   EndIf

   $s &= '</citation>' & @crlf
   $s &= '</File>' & @crlf
   Msgbox(0, $n, $s)
Next

 

And it should be like the below sample output but it's not the way I expected.😥 mikell will get angry on me for this...😅

<File xml:id="12312-bib-0000">
<citation type="journal" xml:id="12312-bib-0000">
<Person><familyName>Wagner</familyName>, <givenNames>W.</givenNames></Person>, <Person><familyName>Raudsepp</familyName>, <givenNames>M.</givenNames></Person>, <Person><familyName>Holtz</familyName>, <givenNames>P.</givenNames></Person>, <Person><familyName>Sen</familyName>, <givenNames>R.</givenNames></Person>
(<pubYear year="2018">2018</pubYear>).
<chapterTitle>Essence politics Identity work and stereotyping in inter-group relations</chapterTitle>.
<editor><givenNames>Haward</givenNames> <familyName>C.</familyName></editor>, <editor><givenNames>Matt</givenNames> <familyName>C.</familyName></editor>
<bookTitle>The social psychology of everyday politics</bookTitle>,
<pageFirst>121</pageFirst>&#x2013;<pageLast>150</pageLast>,
<publisherLoc>London City</publisherLoc>: <publisherName>Routledge Press</publisherName>.
</citation>
</File>

 

Need you help on this Experts, please and thanks in advance.

 

KS15

Edited May 23, 2019 by KickStarter15

[mikell]

Redundant alternations in the expression are useless. Try this one :

$res = StringRegExp($str, '(*UCP)(?x) (?|  ' & _
      '  ([[:alpha:]]+)\h([A-Z][^,]+),\h? ' & _          ; names, Eds
      ' |  ([[a-z\h.]+),\h? ' & _               ; et al
      ' |  \((\d+)\)\.?\h? ' & _                ; year
      ' |  ([A-Z][^.]+.\h) ' & _             ; ChapterTitle, PubLoc, Name
      ' |  (\d+[\(\);.-]) ' & _                 ; vol, issue, pages
      ' |  (?:https?://)?(\w+\S+)$) ' , 3)                 ; url

along with a little more restrictive condition here :

While StringRegExp($res[$i], '^[A-Z]\S*$')  ;<<<<<<<<<
      $s &= '<editor>' & '<givenNames>' & $res[$i] & '</givenNames>, ' & _
                     '<familyName>' & $res[$i+1] & '</familyName>' & '</editor>, ' & @crlf
      $i += 2
   Wend

[KickStarter15]

Thank you so much @mikell, it is working as expected. However, if I change the presentation with

Local $astr[1] = ["Wagner, W., Raudsepp, M., Holtz, P., Sen, R., (2017) Essence politics Identity work and stereotyping in inter-group relations. Haward, C., Matt, C., The social psychology of everyday politics. London. Routledge. 121-150."]

And change the expression.

From this.

' ([[:alpha:]]+)\h([A-Z][^,]+),\h? ' & _          ; names, Eds

To this.

' ([A-Z][^,]+),\h? ' & _          ; names, Eds

The first person names were captured correctly as expected but for the Eds. name it will wrongly presented.

[KickStarter15]

@mikell, I tried having the expression this way and the capturing is correct. Please check if I made it correctly or is there any incorrect expression declared.😅

'([A-Za-z][^.]+),\h([A-Z][^,]+),\h? ' & _          ; names, Eds

 

[mikell]

It's correct. But if you want the whole thing to work in both cases, this must be mentioned in the expression by putting 2 alternations

expandcollapse popup

#Include <Array.au3>

 Local $astr[1] = ["Wagner W., Raudsepp M., Holtz P., Sen R., (2017) Essence politics Identity work and stereotyping in inter-group relations. Haward C., Matt C., The social psychology of everyday politics. London. Routledge. 121-150."] 

; Local $astr[1] = ["Wagner, W., Raudsepp, M., Holtz, P., Sen, R., (2017) Essence politics Identity work and stereotyping in inter-group relations. Haward, C., Matt, C., The social psychology of everyday politics. London. Routledge. 121-150."]

   $n = 0
   For $str In $astr

   $res = StringRegExp($str, '(*UCP)(?x) (?|  ' & _
      '   ([A-Za-z]+)\h([A-Z][^,]+),\h? ' & _          ; names, Eds , case #1
      ' | ([A-Za-z][^.]+),\h([A-Z][^,]+),\h?  ' & _          ; names, Eds , case #2
      ' |  ([[a-z\h.]+),\h? ' & _               ; et al
      ' |  \((\d+)\)\.?\h? ' & _                ; year
      ' |  ([A-Z][^.]+\.\h) ' & _             ; ChapterTitle, PubLoc, Name
      ' |  (\d+[\(\);.-]) ' & _                 ; vol, issue, pages
      ' |  (?:https?://)?(\w+\S+)$) ' , 3)                 ; url

   $n += 1
     _ArrayDisplay($res, $n)

   Local $i = 0, $s = '<File xml:id="1110">' & @crlf & _
                        '<citation type="journal" xml:id="00001">' & @crlf

   While $i < UBound($res) AND StringRegExp($res[$i], '^[A-Z]')
      $s &= '<Person>' & '<familyName>' & $res[$i] & '</familyName>, ' & _
                     '<givenNames>' & $res[$i+1] & '</givenNames>' & '</Person>, ' & @crlf
      $i += 2
   Wend

   If $i < UBound($res) AND StringRegExp($res[$i], '^[a-z\h.]+$') Then
       $s &= $res[$i] & @crlf
       $i += 1
   EndIf

   If $i < UBound($res) AND StringIsDigit($res[$i]) Then
      $s &= '(<pubYear year="' & $res[$i] & '">' & $res[$i] & '</pubYear>);' & @crlf
      $i += 1
   EndIf

   If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<chapterTitle>' & $res[$i] & '</chapterTitle>. ' & @crlf
      $i += 1
   EndIf

   While $i < UBound($res) AND StringRegExp($res[$i], '^[A-Z]\S*$')
      $s &= '<editor>' & '<givenNames>' & $res[$i] & '</givenNames>, ' & _
                     '<familyName>' & $res[$i+1] & '</familyName>' & '</editor>, ' & @crlf
      $i += 2
   Wend

   If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<OtherTitle>' & $res[$i] & '</OtherTitle>. ' & @crlf
      $i += 1
   EndIf
   If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<Location>' & $res[$i] & '</Location>. ' & @crlf
      $i += 1
   EndIf

   If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<LocationName>' & $res[$i] & '</LocationName>. ' & @crlf
      $i += 1
   EndIf
   If $i < UBound($res) AND StringRegExp($res[$i], '^\d+-$') Then
       $s &= '<pageFirst>' & StringTrimRight($res[$i], 1) & '</pageFirst>' & @crlf
       $i += 1
   EndIf
   If $i < UBound($res) AND StringRegExp($res[$i], '^\d+\.$') Then
       $s &= '?<pageLast>' & StringTrimRight($res[$i], 1) & '</pageLast>.' & @crlf
       $i += 1
   EndIf
   If $i < UBound($res) AND StringInStr($res[$i], "doi") > 0 Then  
       $s &= '<url href="' & $res[$i] & '">' & $res[$i] & '</url>' & @crlf
       $i += 1
   ElseIf $i < UBound($res) AND StringInStr($res[$i], "www") > 0 Then
       $s &= '<url href="' & $res[$i] & '">' & $res[$i] & '</url>' & @crlf
       $i += 1
   ElseIf $i < UBound($res) AND StringInStr($res[$i], "http") > 0 Then
       $s &= '<url href="' & $res[$i] & '">' & $res[$i] & '</url>' & @crlf
       $i += 1
   EndIf

   $s &= '</citation>' & @crlf
   $s &= '</File>' & @crlf
   Msgbox(0, $n, $s)
Next

 

Edited May 24, 2019 by mikell

[KickStarter15]

Thank you so much @mikell it's much better now. however, I have two more questions so far,😅...

First: what if the person name contains the word "and" before the last person mentioned. Like the below:

Local $astr[2] = ["Wagner, W., Raudsepp, M., Holtz, P., Sen, R., (2017) Essence politics Identity work and stereotyping in inter-group relations. Haward C., Matt C., The social psychology of everyday politics. London. Routledge. 121-150." , _
 "Wagner, W., Raudsepp, M., Holtz, P., and Sen, R., (2017) Essence politics Identity work and stereotyping in inter-group relations. Haward, C., and Matt, C., The social psychology of everyday politics. London. Routledge."] ; with word "and" before the last person...

Second: There are extra words within the other person name which is the (Eds.) abbreviation for "Editors".

Local $astr[2] = ["Wagner, W., Raudsepp, M., Holtz, P., Sen, R., (2017) Essence politics Identity work and stereotyping in inter-group relations. Haward C., Matt C., (Eds.), The social psychology of everyday politics. London. Routledge. 121-150." , _ ; this contains word "(Eds.)"
 "Wagner, W., Raudsepp, M., Holtz, P., and Sen, R., (2017) Essence politics Identity work and stereotyping in inter-group relations. In: Haward, C., and Matt, C., (Eds.), The social psychology of everyday politics. London. Routledge."] ; this is containing both words "In:" and "(Eds.)"

And I made the expression to read the word Eds. to be excluded but it will produced incorrect capturing. Is this correct😥..

If StringRegExp($res[$i], '((Eds.))') Then
       $s &= $res[$i] & @crlf
       $i += 1
   EndIf

 

[FrancescoDiMuro]

@KickStarter15

Both round parentheses and the dot are metacharacters, so, to let them be threated literally by the RegEx engine, you need to escape each of them with a backslash

[KickStarter15]

@FrancescoDiMuro, yes, I tried that doing "(\(Eds\.)\)" to match the expression but this word should not be tag in XML, it's the same with "et al.,".😅 same as below.

[mikell]

As I see the things becoming gradually more and more complicated with extra words emerging, The best way I can think of is to first carefully remove all these useless extra words from the initial string, so the code will not be overcomplicated and the reliability will be greatly improved

; delete "and" in names, (Eds.), In: ... to be probably continued
$str = StringRegExpReplace($str, '((?<=,\h)and(?=\h[A-Z])|\(Eds.\)|(?<=\.\h)In:)\h?', "")

 

[KickStarter15]

@mikell, Thanks and that would be a great idea and it's fine, but how can I add these words back in the XML tags at the same place where they were replaced/removed from the string  and they don't need to be in the element same as below.😅

The words "and", "In:" and "Eds." should be putted back to where they should be and no need to be tagged in element.

Edited May 27, 2019 by KickStarter15

[mikell]

2 hours ago, KickStarter15 said:

how can I add these words back in the XML

Maybe my comment was not clear enough. When I said "remove them" I also meant "forget them" because they are really useless. Example :

• "(Eds.)" is useless because the previous names have the <editor> tag. No possible confusion.
• "and" is useless as long as the following author's name is still mentioned and present
• etc

Please understand that due to the various formats the strings may have, making a working reliable regex is already a difficult and hazardous task. Furthermore, the more the regex will be complicated, the less the result will be reliable.
If you absolutely need these words to be present, I strongly suggest to do a later step (after the final xml was built) to add them inside

[KickStarter15]

@mikell agreed, and yes it's more complicated that way so I tried using the "If $i == $i  then..." read back the last array and insert the word "and" before the last array.😅 still working on it.

For the word "In:" and "(Eds.)", yes indeed I can add this in the later XML creation. Just like the below on where they should be inserted.

If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '<chapterTitle>' & $res[$i] & '</chapterTitle>. In: '
      $i += 1
   EndIf

If $i < UBound($res) AND StringIsUpper(StringLeft($res[$i], 1)) Then
      $s &= '(Eds.), <otherTitle>' & $res[$i] & '</otherTitle>. '
      $i += 1
   EndIf

The rest of the code is working as expected and thank you so much for your big help👍😁....

[KickStarter15]

@mikell, Sorry for this but I just want to ask if possible to read the string backwards and capture it in XML? I tried using _ArrayReverse() function but don't know how to use it or even reading the string RegExp.😥

[mikell]

Backwards ?     Do you mean this "string a is this" ?  or something else ?

[KickStarter15]

Apologies @mikell😅, I mean something like this.

#include <Array.au3>
Local $astr = "Zhu, X., Zhuang, Q., Gao, X., Sokolov, A., & Schlosser, C. A., et al., (2013). Pan-Arctic land–atmospheric, fluxes: of methane and carbon; dioxide in response to climate change over the 21st century. Environmental Research Letters. 8(4); 45-53. https://doi.org/10.1088/1748-9326/8/4/045003"

$newlines = StringReplace($astr, CHR(10), ". ")
$rev = StringSplit($newlines, " ")
_ArrayDisplay($rev,"")
_ArrayReverse($rev)
_ArrayDisplay($rev,"")

This is to ignore some presentation that is not consistent like the below scenario.

Local $astr[3] = ["Zhu, X., Zhuang, Q., Gao, X., Sokolov, A., & Schlosser, C. A., et al., (2013). Pan-Arctic land–atmospheric, fluxes: of methane and carbon; dioxide in response to climate change over the 21st century. Environmental Research Letters. 8(4); 45-53. https://doi.org/10.1088/1748-9326/8/4/045003";, _
"Tsitsigiannis I., Bok, J., Andes D., Nielson K., Frisvad J., Keller N., (2005) Aspergillus cyclooxygenase-like enzymes. Associated with µ prostaglandin Production and virulence. Infection and Immunity. 73(8);4548–4559. https://doi.org/10.1128/IAI.73.8.4548-4559.2005", _ ; period was placed bewteen the title
"Saliba A. M., Nascimento D. O., Silva M. C. A., Assis M. C., Gayer C. R. M., Raymond B., Plotkowski M. C., (2005) Eicosanoid-mediated 1.0 vs 10.1 proinflammatory activity of Pseudomonas aeruginosa ExoU. Cellular, Microbiology. 7(12);1811. https://doi.org/10.1111/j.1462-5822.2005.00635.x."] ; title contains period identifying decimal numbers

   $n = 0

   For $str In $astr
   
   $res = StringRegExp($str, '(*UCP)(?x) (?|  ' & _
      '([A-Za-z][^.]+),\h([A-Z][^,]+),\h? '  & _          ; names
      ' |  ([[a-z\h.]+),\h? ' & _               ; et al
      ' |  \((\d+)\)\.?\h? ' & _                ; year
      ' |  ([A-Z][^.]+\.)\h?  ' & _             ; title, subtitle
      ' |  (\d+[\(\);.-]) ' & _                 ; vol, issue, pages
      ' |  (?:https?://)?(\w+:\S+)$) ' , 3)                 ; url
   $n += 1
     _ArrayDisplay($res, $n)

By reading them backwards all I can declare in RegExp() will be triggered accordingly and can capture the array correctly. What do you think? is this a best way to read them properly?

Edited June 6, 2019 by KickStarter15

[mikell]

The more you go and the more you run into something inextricable 

I wonder how a regex could be able to differenciate when 3 sentences are found, if it means
- a 3-sentences title with no subtitle
- a 2-sentences title with a 1-sentence subtitle
- a 1-sentence title with a 2-sentences subtitle

For the rest all I can suggest is the following
But it would really be a better way to parse or pre-parse the strings beforehand. Where do you get these strings from ?

#Include <Array.au3>

Local $astr[3] = ["Zhu, X., Zhuang, Q., Gao, X., Sokolov, A., & Schlosser, C. A., et al., (2013). Pan-Arctic land–atmospheric, fluxes: of methane and carbon; dioxide in response to climate change over the 21st century. Environmental Research Letters. 8(4); 45-53. https://doi.org/10.1088/1748-9326/8/4/045003", _
"Tsitsigiannis I., Bok J., Andes D., Nielson K., Frisvad J., Keller N., (2005) Aspergillus cyclooxygenase-like enzymes. Associated with µ prostaglandin Production and virulence. Infection and Immunity. 73(8);4548–4559. https://doi.org/10.1128/IAI.73.8.4548-4559.2005", _ ; period was placed bewteen the title
"Saliba A. M., Nascimento D. O., Silva M. C. A., Assis M. C., Gayer C. R. M., Raymond B., Plotkowski M. C., (2005) Eicosanoid-mediated 1.0 vs 10.1 proinflammatory activity of Pseudomonas aeruginosa ExoU. Cellular, Microbiology. 7(12);1811. https://doi.org/10.1111/j.1462-5822.2005.00635.x."] ; title contains period identifying decimal numbers

   $n = 0
For $str In $astr
   $res = StringRegExp($str, '(*UCP)(?x) (?|  ' & _
      '([A-Za-z][^.]+?),?\h([A-Z][^,]+),\h? '  & _          ; names
      ' |  ([[a-z\h.]+),\h? ' & _               ; et al
      ' |  \((\d+)\)\.?\h? ' & _                ; year
      ' |  ([A-Z].+?\.(?!\d))\h?  ' & _             ; title, subtitle
      ' |  (\d+[\(\);.–-]) ' & _                 ; vol, issue, pages
      ' |  (?:https?://)?(\w+:\S+)$) ' , 3)                 ; url
   $n += 1
     _ArrayDisplay($res, $n)
Next

 

[KickStarter15]

@mikell thanks for the usual help in my thread, 😅 and yes these cases are unique that means rare case only. I was not able to anticipate this type of sentence within the title and subtitle. However, for the second string (see below).

FROM:

Tsitsigiannis I., Bok J., Andes D., Nielson K., Frisvad J., Keller N., (2005) Aspergillus cyclooxygenase-like enzymes. Associated with µ prostaglandin Production and virulence. Infection and Immunity. 73(8);4548–4559. https://doi.org/10.1128/IAI.73.8.4548-4559.2005

SHOULD BE:

Tsitsigiannis
I
Bok
J
Andes
D
Nielson
K
Frisvad
J
Keller
N
(2005)
Aspergillus cyclooxygenase-like enzymes. Associated with µ prostaglandin Production and virulence ; period should be ignored if found within the title.
Infection and Immunity
73
8
4548
4559
https://doi.org/10.1128/IAI.73.8.4548-4559.2005

Other strings are fine and thank you so much mikell.😄