Jump to content

Issue with accented characters (CLI → clibpoard)


Go to solution Solved by ioa747,

Recommended Posts

Hello,

I use the following script to 1) run a command-line application (ie. "DOS" app), 2) copy its output to the clipboard, and 3) paste it to Notepad as a simple way to search the data.

I notice that accented chars are wrong, eg. "é" becomes "é"

Is it because of the DOS to Windows character pages?

[]
;run CLI app
Local $iPID = Run($WholeLine, $szDir, @SW_HIDE,$STDOUT_CHILD)
If @error Then Exit MsgBox($MB_ICONERROR, "Error", "Running failed.")
Local $sOutput = StdoutRead($iPID)
If @error Then Exit ConsoleWrite("Error returned by StdOutRead! @error=" & @error & ", @extended=" & @extended & @CRLF)
While True
    $sOutput &= StdoutRead($iPID)
    If @error Then ExitLoop
WEnd

;Poor man's search: copy STDOUT into clipboard, launch Notepad, paste
ClipPut($sOutput)
$iPID = Run("notepad.exe")
Opt("WinTitleMatchMode", 2) ;1=start, 2=subStr, 3=exact, 4=advanced, -1 to -4=Nocase
Local $hHdl = WinWait("[TITLE:Untitled - Notepad2; CLASS:Notepad2]","",10)
if  not $hHdl then Exit MsgBox($MB_ICONERROR,"Error","Windows not found")
Sleep(500)
Send("^v") ;paste
Sleep(1000)
Send("^{HOME}") ;jump to first line in notepad

Thank you.

Link to comment
Share on other sites

Thanks. Looks like I'm using it wrong: This time, "é" turns into "+¬".

;run CLI app
Local $iPID = Run($WholeLine, $szDir, @SW_HIDE,$STDOUT_CHILD)
If @error Then Exit MsgBox($MB_ICONERROR, "Error", "Running failed.")
Local $sOutput = StdoutRead($iPID)
If @error Then Exit ConsoleWrite("Error returned by StdOutRead! @error=" & @error & ", @extended=" & @extended & @CRLF)
While True
    $sOutput &= StdoutRead($iPID)
    If @error Then ExitLoop
    ;ConsoleWrite($sOutput & @CRLF)
WEnd

;convert from DOS to ANSI
$sOutput = _WinAPI_OemToChar ($sOutput)

;Poor man's search: copy STDOUT into clipboard, launch Notepad, paste
ClipPut($sOutput)
Link to comment
Share on other sites

I use the following function (taken from the forum):

; #INTERNAL_USE_ONLY# ===========================================================================================================
; Name ..........: _Ansi2Oem
; Description ...: Translates a string from ANSI to OEM.
; Syntax ........: _Ansi2Oem($sText)
; Parameters ....: $sText - Text to translate
; Return values .: Success - Returns the translated text
;                  Failure - None
; Author ........: Zedna
; Modified ......:
; Remarks .......:
; Related .......: Oem2Ansi (see link below)
; Link ..........: https://www.autoitscript.com/forum/topic/201398-german-umlauts-and-console-programs/?do=findComment&comment=1445253
; Example .......:
; ===============================================================================================================================
Func _Ansi2Oem($sText)
    Local $aText
    $aText = DllCall('user32.dll', 'Int', 'CharToOem', 'str', $sText, 'str', '')
    Return ConsoleWrite($aText[2])
EndFunc   ;==>_Ansi2Oem

 

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2024-07-28 - Version 1.6.3.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
OutlookEX (2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
PowerPoint (2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki
Task Scheduler (2022-07-28 - Version 1.6.0.1) - Download - General Help & Support - Wiki

Standard UDFs:
Excel - Example Scripts - Wiki
Word - Wiki

Tutorials:
ADO - Wiki
WebDriver - Wiki

 

Link to comment
Share on other sites

Lets try something simple that you and I can actually run :

#include <WinAPIConv.au3>
#include <Constants.au3>

Local $iPID = Run(@ComSpec & " /c dir c:\", "", @SW_SHOW, $STDERR_MERGED)
ProcessWaitClose($iPID)
Local $sStream = StdoutRead($iPID)
MsgBox($MB_OK, "", _WinAPI_OemToChar($sStream))

Does that work ?

Link to comment
Share on other sites

  • Solution

Out of curiosity, have you checked this? or unchecked?

original?v=v2&px=-1

Settings > Time & Language >  Language >  Administrative language settings >  Change system locale

I know that I know nothing

Link to comment
Share on other sites

2 hours ago, water said:

I use the following function (taken from the forum):

; #INTERNAL_USE_ONLY# ===========================================================================================================
; Name ..........: _Ansi2Oem
; Description ...: Translates a string from ANSI to OEM.
; Syntax ........: _Ansi2Oem($sText)
; Parameters ....: $sText - Text to translate
; Return values .: Success - Returns the translated text
;                  Failure - None
; Author ........: Zedna
; Modified ......:
; Remarks .......:
; Related .......: Oem2Ansi (see link below)
; Link ..........: https://www.autoitscript.com/forum/topic/201398-german-umlauts-and-console-programs/?do=findComment&comment=1445253
; Example .......:
; ===============================================================================================================================
Func _Ansi2Oem($sText)
    Local $aText
    $aText = DllCall('user32.dll', 'Int', 'CharToOem', 'str', $sText, 'str', '')
    Return ConsoleWrite($aText[2])
EndFunc   ;==>_Ansi2Oem

 

The input file is UTF8, so that didn't work.

2 hours ago, Nine said:

Does that work ?

Nope, still wrong characters.

;Local $iPID = Run(@ComSpec & " /c dir c:\", "", @SW_SHOW, $STDERR_MERGED)
Local $iPID = Run(@ComSpec & " /c type test.utf8.txt", "", @SW_SHOW, $STDERR_MERGED)

 

1 hour ago, ioa747 said:

Out of curiosity, have you checked this? or unchecked?

original?v=v2&px=-1

Settings > Time & Language >  Language >  Administrative language settings >  Change system locale

That solved it (reboot required.)

Thanks all.

Link to comment
Share on other sites

FYI, on my system (Win10 x64 20h2) checking this setting has side effects: when I run a very simple piece of code from Scite, random strings are displayed in Scite console.

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to comment
Share on other sites

1 minute ago, ioa747 said:

checked ? or unchecked?

From unchecked to checked.

7 minutes ago, jchd said:

FYI, on my system (Win10 x64 20h2) checking this setting has side effects: when I run a very simple piece of code from Scite, random strings are displayed in Scite console.

Thanks for the tip. If it occurs on mine, I'll revert back, and use an alternative — the application I need to use also as the option of saving the output to a file instead.

Checking the option in Windows added a second keyboard layout while I only had one before. No biggie, though.

Link to comment
Share on other sites

As @jchd, checking this box does not work for me either.  I am unable to write french characters correctly in Scite console.  Did not test it extensively as I have no benefit of using it, or I may say, I have no issue without it...;)

Link to comment
Share on other sites

@Nine, I fixed that by 1) setting this In SciTeUserProperties:

code.page=65001
output.code.page=65001

and using a good Unicode font:

font.monospace=font:DejaVu Sans Mono,size:11

Then I use CW() instead of ConsoleWrite():

Func CW($s = "")
    If @Compiled Then
        _CUI_ConsoleWrite($s)
    Else
        _ConsoleWrite($s)
    EndIf
EndFunc   ;==>CW


Func _CUI_ConsoleWrite(ByRef $s)
    Local Static $hDll = DllOpen("kernel32.dll")
    Local Static $hCon = __CUI_ConsoleInit($hDll)
    DllCall($hDll, "bool", "WriteConsoleW", "handle", $hCon, "wstr", $s & @LF, "dword", StringLen($s) + 1, "dword*", 0, "ptr", 0)
    Return
EndFunc   ;==>_CUI_ConsoleWrite


; internal use only
Func __CUI_ConsoleInit(ByRef $hDll)
    DllCall($hDll, "bool", "AllocConsole")
;~  The following 2 lines don't work for compiled scripts due to a MS bug
;~      see last post in thread https://developercommunity.visualstudio.com/t/setconsoleoutputcpcp-utf8-results-in-code-page-850/413190
;~      where MS support acknowledges it's a (still unfixed) bug in Windows itself
;~      So you can only display current locale codepage using current console font in a console when your script is compiled
;~      When running your script from SciTE, the UTF8 setting of SciTE overrides this bug and Unicode can be displayed
;~  DllCall("Kernel32.dll", "bool", "SetConsoleCP", "uint", 65001)
;~  DllCall("Kernel32.dll", "bool", "SetConsoleOutputCP", "uint", 65001)
    Return DllCall($hDll, "handle", "GetStdHandle", "int", -11)[0]
EndFunc   ;==>__CUI_ConsoleInit


; Unicode-aware ConsoleWrite
Func _ConsoleWrite(ByRef $s)
    ConsoleWrite(BinaryToString(StringToBinary($s & @LF, 4), 1))
EndFunc   ;==>_ConsoleWrite

 

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to comment
Share on other sites

Using the above changes, run this:

$s = "Большая проблема  大问题  बड़ी समस्या  مشكلة كبيرة Test Title 😭" & @LF
$s &= "Our familly " & ChrW(0xD83D) & ChrW(0xDC68) & ChrW(0xD83C) & ChrW(0xDFFB) & ChrW(0x200D) & ChrW(0xD83D) & ChrW(0xDC69) & ChrW(0xD83C) & ChrW(0xDFFF) & ChrW(0x200D) & ChrW(0xD83D) & ChrW(0xDC66) & ChrW(0xD83C) & ChrW(0xDFFD & @LF)

CW($s)

Result:

Большая проблема  大问题  बड़ी समस्या  مشكلة كبيرة Test Title 😭
Our familly 👨🏻‍👩🏿‍👦🏽

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to comment
Share on other sites

On 4/21/2024 at 6:12 PM, littlebigman said:

Checking the option in Windows added a second keyboard layout while I only had one before. No biggie, though.

This seems to work to remove that second keyboard and go back to normal. I had a bad feeling about making changes to the Region section…and wasn't disappointed.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...