convert tabs to spaces

[Professor_Bernd]

These darn tabs often cause problems. Who invented them!?

Ok, I've dealt with tab problems before, but then I usually created a workaround for the situation. Now I would like to have a general solution to convert tabs to spaces and vice versa.

The first thought is often: "What should be so hard about that! You take the tab length and replace each tab with this number of spaces". But it is not that simple.

Suppose a user has set a tab length to 4. If he now writes: "This is a test" and uses a tab between each word. Then after "This" there is an indention of 4 places, after "is" there is an indention of 2 places and after "a" there is an indention of 3 places. Not to mention what happens if the user inserts something afterwards, e.g. "This is a nice little test"

If you want to process text, e.g. to align it in the script (see the posting Spaces vs tabs by Richard Robertson)

This    is  a   nice    little  test ; text with tabs to convert
This    is  a   nice    little  test ; expected result - text with spaces and
                                     ; all words in the same position as before

or in my case to position a CallTip in the right place, tabs can put a damper on your work.

 

As I said, I'd like to have a general solution, like an UDF, which allows you to convert tabs to spaces and vice versa. In editors like SciTE PSPad there is a built-in tab to space conversion, so it should be possible to create a reliable conversion UDF, no matter if tab width 2, 3, 4 or whatever.

Is there already a UDF for this? Or does anyone have an idea for such an UDF?

 

Edit: After all this I realized that the reverse conversion of spaces to tabs cannot work.

Edit: After all this I realized that the reverse conversion direction (from spaces to tabs) cannot work. (see posting) Therefore I changed the thread title from "Converter UDF for tabs to spaces and vice versa" to "convert tabs to spaces". Prof Bernd.

Edited October 27, 2020 by Professor_Bernd
text example and screenshot added - double section removed - mention of Tidy and SciTE removed, because they only removes the tabs at the beginning of a line

[Jos]

I am pretty sure that a single regex can do that leading tab/spaces conversion for you, but am not behind my laptop at the moment.

Jos

[mikell]

Hmmm

$res = StringRegExpReplace($txt, '\t+', " ")

[Jos]

Think that needs a little change as I assume it is only leading tabs/space, so at the beginning of the line and skip all other occurrences. .... and each tab needs to be converted to x spaces.

Edited October 25, 2020 by Jos

[Professor_Bernd]

@mikell

Sorry, I see that such problems are not easy to describe. I try to use commented code, because the code shows the formatting, like an editor.

This    is  a   nice    little  test ; text with tabs to convert
This    is  a   nice    little  test ; expected result - text with spaces and
                                     ; all words in the same position as before

This is a nice little test           ; result with RegEx from mikell

Edit: The expected result with spaces should show the same format as the source text with tabs. This means that all tabs should be replaced with so many spaces that the words are in the same positions as before.

 

@Jos You are right, not only the leading tabs should be converted, but all tabs.

Application example is the placement of a CallTip at the first letter of a function in the text editor. (Precondition is a monospace font). Here is a screenshot, where I intentionally mixed tabs and spaces to demonstrate what is meant.

 

I don't know exactly how to name the space that 1 sign (except tabs!) occupies. Should I call it "space" or "digit"? At the moment I call it "digit".

To position the CallTip exactly at the first letter of "MsgBox" you have to know how many digits (NOT signs) there are from the beginning of the line to "MsgBox". It is important to know that counting the signs does not work, because a tab is counted as 1 sign. Instead, the digits must be counted. The easiest way to do this is to convert the tabs to spaces and keep the words in the same position in the editor.

Edited October 25, 2020 by Professor_Bernd

[Gianni]

here a quick and dirty function which returns a string in which tabs have been replaced with spaces

$sInputString = "This" & Chr(9) & "is" & Chr(9) & "a" & Chr(9) & "nice" & Chr(9) & "little" & Chr(9) & "test"

ConsoleWrite($sInputString & @CRLF) ; string with tabs
ConsoleWrite(_VirtualTabs($sInputString) & @CRLF) ; string with only spaces (no tabs)

Func _VirtualTabs($sString, $iTabStop = 8)
    StringReplace($sString, Chr(9), Chr(9))
    Local $iNrOfTabs = @extended ; number of tabs in the string (if any)
    For $i = 1 To $iNrOfTabs
        $sString = StringReplace($sString, Chr(9), StringFormat('%' & (Mod(StringInStr($sString, Chr(9)), $iTabStop) * -1) + $iTabStop + 1 & 's', ""), 1)
    Next
    Return $sString
EndFunc   ;==>_VirtualTabs

 

[Jos]

3 hours ago, Professor_Bernd said:

@Jos You are right, not only the leading tabs should be converted, but all tabs.

mmm that isn't what I said as SciTE & Tidy only do something with the Leading (indents) of the code when it comes to Tab/Spaces conversion.
You just opened to total different "can-of-worms" when you also want to convert inside of a line as there could be literal strings there too containing  spaces or tabs which you really don't want to convert i assume. 

Jos

[Professor_Bernd]

@Chimp

I don't know how the SciTE console formats, but if you copy the result from the console back into the editor area, you will see the following result. Or you can use ClipPut() as in the following spoiler.

Spoiler

;~ $sInputString = "This" & Chr(9) & "is" & Chr(9) & "a" & Chr(9) & "nice" & Chr(9) & "little" & Chr(9) & "test"
$sInputString = "This" & " " & "is" & Chr(9) & "a" & Chr(9) & "nice" & Chr(9) & "little" & Chr(9) & "test"

ConsoleWrite($sInputString & @CRLF) ; string with tabs
ConsoleWrite(_VirtualTabs($sInputString) & @CRLF) ; string with only spaces (no tabs)

Global $sRes
$sRes = $sInputString & @CRLF ; string with tabs
$sRes &= _VirtualTabs($sInputString) & @CRLF ; string with only spaces (no tabs)
ClipPut($sRes)

Func _VirtualTabs($sString, $iTabStop = 8)
    StringReplace($sString, Chr(9), Chr(9))
    Local $iNrOfTabs = @extended ; number of tabs in the string (if any)
    For $i = 1 To $iNrOfTabs
        $sString = StringReplace($sString, Chr(9), StringFormat('%' & (Mod(StringInStr($sString, Chr(9)), $iTabStop) * -1) + $iTabStop + 1 & 's', ""), 1)
    Next
    Return $sString
EndFunc   ;==>_VirtualTabs

;~ Result:
#cs
This    is  a   nice    little  test
This    is      a       nice    little  test

This is a   nice    little  test
This is         a       nice    little  test

#ce

 

I replaced the first and second tab in the source text with 1 space each (the user can write as he wants) and the result looks like this. (Notice where the words are positioned below each other to see the difference).

Unfortunately this is not the desired result.

[Nine]

nice going @Chimp but it will not work if words are longer :

$sInputString = "This" & Chr(9) & "is" & Chr(9) & "a" & Chr(9) & "nice" & Chr(9) & "longbutnolittle" & Chr(9) & "test"

I am not be picky here, just trying to test your algorithm

[Professor_Bernd]

1 hour ago, Jos said:

mmm that isn't what I said as SciTE & Tidy only do something with the Leading (indents) of the code when it comes to Tab/Spaces conversion.
You just opened to total different "can-of-worms" when you also want to convert inside of a line as there could be literal strings there too containing  spaces or tabs which you really don't want to convert i assume. 

Oh, I didn't check this carefully, SciTe actually only replaces the tab at the beginning of a line. - I am used to PSPad converting the desired tabs to spaces in the whole text, even in comments. (This is the desired result.)

[Professor_Bernd]

30 minutes ago, Nine said:

I am not be picky here, just trying to test your algorithm

Tests are good, they help us to find a solution.

[Gianni]

1 hour ago, Professor_Bernd said:

@Chimp

I don't know how the SciTE console formats, but if you copy the result from the console back into the editor area, you will see the following result. Or you can use ClipPut() as in the following spoiler.

  Reveal hidden contents

;~ $sInputString = "This" & Chr(9) & "is" & Chr(9) & "a" & Chr(9) & "nice" & Chr(9) & "little" & Chr(9) & "test"
$sInputString = "This" & " " & "is" & Chr(9) & "a" & Chr(9) & "nice" & Chr(9) & "little" & Chr(9) & "test"

ConsoleWrite($sInputString & @CRLF) ; string with tabs
ConsoleWrite(_VirtualTabs($sInputString) & @CRLF) ; string with only spaces (no tabs)

Global $sRes
$sRes = $sInputString & @CRLF ; string with tabs
$sRes &= _VirtualTabs($sInputString) & @CRLF ; string with only spaces (no tabs)
ClipPut($sRes)

Func _VirtualTabs($sString, $iTabStop = 8)
    StringReplace($sString, Chr(9), Chr(9))
    Local $iNrOfTabs = @extended ; number of tabs in the string (if any)
    For $i = 1 To $iNrOfTabs
        $sString = StringReplace($sString, Chr(9), StringFormat('%' & (Mod(StringInStr($sString, Chr(9)), $iTabStop) * -1) + $iTabStop + 1 & 's', ""), 1)
    Next
    Return $sString
EndFunc   ;==>_VirtualTabs

;~ Result:
#cs
This    is  a   nice    little  test
This    is      a       nice    little  test

This is a   nice    little  test
This is         a       nice    little  test

#ce

 

I replaced the first and second tab in the source text with 1 space each (the user can write as he wants) and the result looks like this. (Notice where the words are positioned below each other to see the difference).

Unfortunately this is not the desired result.

@Professor_Bernd, it appears that tab stops in the console are 8 characters long (they are fixed every 8 characters), while in the editor they have a length of 4, so copying a string from the console and pasting it into the editor, the tabs (chr (9 )) react differently by adapting to the new tabulators, while the spaces remain the same. You need to decide what length you need to set the tab stop to and then pass that value as the second parameter of the function.

 

1 hour ago, Nine said:

nice going @Chimp but it will not work if words are longer :

$sInputString = "This" & Chr(9) & "is" & Chr(9) & "a" & Chr(9) & "nice" & Chr(9) & "longbutnolittle" & Chr(9) & "test"

I am not be picky here, just trying to test your algorithm

@Nine, You are right, thanks for the testing, I made a small correction in the function

; $sInputString = "This is a" & Chr(9) & "nice" & Chr(9) & "little" & Chr(9) & "test"
$sInputString = "This" & Chr(9) & "is" & Chr(9) & "a" & Chr(9) & "nice" & Chr(9) & "longbutnolittle" & Chr(9) & "test"

ConsoleWrite($sInputString & @CRLF) ; string with tabs
ConsoleWrite(_VirtualTabs($sInputString) & @CRLF) ; string with only spaces (no tabs)

Func _VirtualTabs($sString, $iTabStop = 8)
    StringReplace($sString, Chr(9), Chr(9))
    Local $iNrOfTabs = @extended ; number of tabs in the string (if any)
    For $i = 1 To $iNrOfTabs
        Local $iMod = Mod(StringInStr($sString, Chr(9)), $iTabStop)
        $sString = StringReplace($sString, Chr(9), StringFormat('%' & ((($iMod * -1) + $iTabStop * ($iMod <> 0)) + 1) & 's', ""), 1)
    Next
    Return $sString
EndFunc   ;==>_VirtualTabs

 

 

[Professor_Bernd]

I am impressed! About half a year ago I was looking for a solution and I don't know how long I searched without success. When I see your solution now, "StringFormat" is actually obvious for this task. But I really didn't come up with it.

OK, first small tests your new code has passed with flying colors! @Nine and "@ all", I would be glad about further tests. You should also test whole files, with comments, line breaks and whatever evil things you can think of.

If your code works without problems after the tests, would you post it to the AutoIt Example Scripts? There I first searched and found nothing. Would be nice if the next one finds your code then.

Can I post your code on the German Autoit Forum and name you as the author? If it's ok with you, I'm going to rename the function there a bit.

From
Func _VirtualTabs($sString, $iTabStop = 4)
to
Func TabsToSpaces($sString, $iTabSize = 4) ; or $iTabWidth ?  (see note 2)
Func TabsToSpaces($sString, $iTabStop = 4)

... or whatever you like best, the main thing is that a user can find the function when searching for it.

Apart from that I like very much that you can pass the tab width in the function!

Thanks for your good work! 👍

Note: After all this I realized that the reverse conversion of spaces to tabs can't work.

Note: After all this I realized that the reverse conversion direction (from spaces to tabs) cannot work.  I will change the thread title accordingly.

Note 2: Now that I've understood the difference again between "replace with Tab-Width" and "replace until Tab-Stop", it must be $iTabStop, of course.

Edited October 27, 2020 by Professor_Bernd

[Jos]

10 hours ago, Professor_Bernd said:

Oh, I didn't check this carefully, SciTe actually only replaces the tab at the beginning of a line. - I am used to PSPad converting the desired tabs to spaces in the whole text, even in comments. (This is the desired result.)

This is the find/replace function which is totally different from the Indent function, but is also available in SciTE! 

[Professor_Bernd]

3 hours ago, Jos said:

This is the find/replace function which is totally different from the Indent function, but is also available in SciTE! 

I did not mean the find/replace function, in PSPad there is a special function "Convert Tabs to Spaces".

Spoiler

 

[Jos]

8 minutes ago, Professor_Bernd said:

I did not mean the find/replace function, in PSPad there is a special function "Convert Tabs to Spaces".

  Reveal hidden contents

 

I know, but that is really simply an Replace All "x09" with "x spaces" function, nothing more nothing less.  

[Professor_Bernd]

3 minutes ago, Jos said:

I know, but that is really simply an Replace All "x09" with "x spaces" function, nothing more nothing less.  

This is not correct. "Convert Tabs to Spaces" in PSPad replaces all tabs in such a way that in the editor the words are displayed at the same position as before the replacement.

[Jos]

19 minutes ago, Professor_Bernd said:

This is not correct. "Convert Tabs to Spaces" in PSPad replaces all tabs in such a way that in the editor the words are displayed at the same position as before the replacement.

Yes ...and how is that different from what I stated?

From the PSPAd helpfile:

Quote

Convert Tabs to Spaces

Converts tab characters (09h) to spaces. The number of spaces is set by the "Tab Width".

Jos

 

[Professor_Bernd]

In the screenshot you can see the difference between TabsToSpaces and a Replace All "x09" with "x spaces" function.

[Jos]

Ok...  but that mean you need to define "tab-stops" to be able to determine the number of spaces required to replace the Tab.... So that is again a whole other case that your initial request.  

Understand what I mean by tab-stops?  ( Fixed positions on the line where the tab would jump to)