LFN UDF - overcome MAX_PATH limit of 256 chars

[corz]

It's great that this topic is getting some traction now. The new list of functions Jon posted yesterday bodes well for the future!

And hey! Busy we understand! No apologies required.

;o) Cor

[orbs]

updated at first post:

modified _LFN_FileCopy() to include a reversion method to use native FileCopy() if possible - i.e. if all following conditions are met:

 

- flags do not require special consideration

- dst did not end with "" => dst is not a folder => dst is a file

- dst is short (<260 characters)

- src is a single file (not having * or ? characters)

- src is short (<260 characters)

 

other functions to follow...

 

modified to add empty line before section header:

_LFN_IniDelete()

_LFN_IniRenameSection()

_LFN_IniWrite()

_LFN_IniWriteSection()

[mLipok]

Did you see:

'?do=embed' frameborder='0' data-embedContent>>

[corz]

Nice job orbs! I look forward to the others!

Did you see:

'?do=embed' frameborder='0' data-embedContent>>

 

That's the thread we were referring to (above)!

;o) Cor

[orbs]

updated at first post:

 

added function to toggle blank line before section header:

__LFN_IniSetSectionsApart()

this function, although classified as internal, is actually meant to be used by the calling script. usage:

__LFN_IniSetSectionsApart(1) -> (default) create a blank line

__LFN_IniSetSectionsApart(0) -> do not create a blank line

 

modified to accommodate for the above, and to maintain encoding at rewrite:

_LFN_IniDelete()

_LFN_IniRenameSection()

_LFN_IniWrite()

_LFN_IniWriteSection()

 

modified for reversion (as described in post #22 above):

_LFN_FileMove()

_LFN_FileDelete()

_LFN_FileSetAttrib()

_LFN_FileSetTime()

 

and, as it so happens, it seems this UDF will be decommissioned with the next release of AutoIt, as extended paths lengths will be natively supported. however, as things stand today, while that feature is still in beta - and while this UDF offers some features that probably will not find their way to the native functions - this UDF will still be maintained in some form or another, if only for reference.

[corz]

Thanks again for keeping up the good work!

My own copy is modified, so I will be copying over all the good stuff ASAP, particularly the new clever reversions!

I tried your new __LFN_IniSetSectionsApart(), but it still doesn't do what I expect/need, which is to keep the existing blank lines in the ini file. Updating this..

; Version number
;
; This is updated automatically and enables you to get automatic ini updating
; as well as check for new versions
;
version=0.0.0.0


;
; Text Editor                   [string/path]   [default: editor=notepad.exe]
;
; We start with this setting - in case you are in Notepad right now, and can
; barely tell the comments and settings apart!
;
; Enter the path to your preferred text editor..
; This is invoked if you select the "edit preferences" menu option.
;
editor=C:\Program Files\Text Ed\editplus\editplus.exe


;
; Show ToolTip Help                         [bool]  [default: show_tips=true]
;
; Some helpful information when you hover your mouse over controls..
;
show_tips=true

converts it to..

; Version number
;
; This is updated automatically and enables you to get automatic ini updating
; as well as check for new versions
;
version=1.0.0.0
;
; Text Editor                   [string/path]   [default: editor=notepad.exe]
;
; We start with this setting - in case you are in Notepad right now, and can
; barely tell the comments and settings apart!
;
; Enter the path to your preferred text editor..
; This is invoked if you select the "edit preferences" menu option.
;
editor=C:\Program Files\Text Ed\editplus\editplus.exe
;
; Show ToolTip Help                         [bool]  [default: show_tips=true]
;
; Some helpful information when you hover your mouse over controls..
;
show_tips=true

Fortunately, I don't need to read/write ini files with long paths, so I am currently using the built-in functions for now. However, when I did use these ones, I simply changed the line that reads..

If $aFileLines[$i] <> '' ...

Omitting that condition, and it worked fine! (basically writing back whatever blank lines are already there) It's okay to write blank lines, lots of them!

Speaking about the internal functions, I haven't had time to properly play with the new AutoIt beta, but I had a quick try of one of the new so-called long-path aware functions and it failed instantly, so your UDF is still the only game in town, for now. Keep it up!

Note, _LFN_FileGetShortName() needs a second parameter to mimic the built-in function faithfully.

Still on the subject of FileGetShortName(), I note that it fails sometimes. It's not just your UDF, but also the built-in function - I rewrote it to use a pure DLL call (using GetShortPathNameW) and this also failed. It seems some paths simply cannot be made short! Perhaps you can throw some expertise at it

Another modifications I made was to have _LFN_FileGetLongName() prefix the return path with the UNC-style prefix when the path was over 259 characters, which makes sense, at least to me! You might know better.

Keep up the good work!

;o) Cor

[orbs]

thanks corz for your feedback. i only had a few minutes to overview your comments, i'll have a deeper look when i have the time.

Edited July 22, 2014 by orbs

[orbs]

ok, now be the time... 

regarding blank lines in ini files: the main reason i chose to eliminate those, is that i'm dealing with a specific application that does not handle blank lines well in its ini files (which often goes well over the 32kb limit). and frankly, i don't see the point in them either. but i can understand that people may prefer having lots of blank lines for readability, so i'll see how i can make this happen.

early attempts showed an issue when adding new values to existing section that ends with blank lines - the new values were added after the blank lines. i abandoned that at the time, but i'll get to it when i can.

regarding native support for extended lengths: would you mind reporting the issues you encountered to the Developer Chat forum? there are threads to the beta releases.

regarding second optional parameter for _LFN_FileGetShortName(): silly me, how could i have missed that   ? i'll get it fixed.

regarding FileGetShortName() not always working: i made this simple test, regardless of LFN:

#include <Array.au3> ; included only for a call to _ArrayDisplay
#include <File.au3>

Local $aFiles=_FileListToArrayRec('C:\','*', 0, 1, 0,2)
For $i=1 To $aFiles[0]
    $aFiles[$i]=FileGetShortName($aFiles[$i])
Next
_ArrayDisplay($aFiles)

the results were indeed puzzling:

this may relate to the registry settings for 8.3 naming, as described here:

http://technet.microsoft.com/en-us/library/cc778996(WS.10).aspx

however i did not try to meddle with this. although this is a puzzling behavior, the LFN function is compatible with the native one, so for now i'm ok with that.

regarding _LFN_FileGetLongName() prefix for unicode: that may be useful, however i will not implement this in the UDF, for the following reasons:

1) compatibility with native function.

2) not needed. if you submit the result of _LFN_FileGetLongName() as is - without the unicode prefix - to any of the other LFN functions, they already add the prefix as needed (as mentioned in the "note #1" at the first post).

3) i feel that in concept, the calling script - and the coder who wrote it - does not need to be aware of hardcore programming mechanisms, like DllStruct,ObjCreate and the likes. and i feel the same way about the unicode prefix. internally handled be it should.

4) despite of the above, if insisting, the calling script is in full liberty to call the internal function __LFN_Target_SetUnicodePrefix().

that's it for now... updates to follow.

Edited July 22, 2014 by orbs

[corz]

Yes, the blank lines are for readability. I use ini files extensively for preferences and usually leave lots of notes and examples and such right above each setting.

As for the Long Path handling in the beta, I do plan to report bugs when I get a chance to properly test the new stuff. Not yet!

I do understand your thinking about _LFN_FileGetLongName(). And for sure, I have found myself using __LFN_Target_SetUnicodePrefix()directly quite a bit! (as well as my own version which adds it regardless of whether or not it's a root dir!)

Onwards.. another bug report!

Moving an empty directory to a different drive with _LFN_DirMove() deletes the directory! Ouch!

MoveFileExW never does anything when moving directories to a different drive, as per spec, so your fall-back routine always kicks in. But with empty dirs, _LFN_DirCopy() fails but returns 1 (so that's another bug!), so _LFN_DirMove()continues with the delete operation. And WHAM! It's gone!

;o) Cor

[orbs]

updated at the first post:

modified:

 

to copy empty directories - also when moving across drives:

_LFN_DirCopy()

 

for compatibility - added optional 2nd parameter:

_LFN_FileGetShortName()

[corz]

Works great!

;o) Cor

[orbs]

updated at first post:

modified ini functions to preserve blank lines:

_LFN_IniDelete()

_LFN_IniRenameSection()

_LFN_IniWrite()

_LFN_IniWriteSection()

 

removed function no longer needed due to the above:

__LFN_IniSetSectionsApart()

Edited July 29, 2014 by orbs

[corz]

Very nice. Thanks!

;o) Cor

[orbs]

no update, just heads-up on this detail, which may seem minor but may be script-breaking:

until now, FileCopy() would fail when trying to copy an encrypted file into an unencryptable path.

_LFN_FileCopy() introduced a flag to overcome this.

in >AutoIt v3.3.13.2 Beta, Jon modified the native FileCopy() to perform the copy instead of failing (still to unencrypted form, of course). so, you don't need to specify the flag for _LFN_FileCopy() to do it.

however, the flag is useful to force copy an encrypted file to unencrypted form in an encryptable target.

why would you want that? i'm sure plenty of reasons exist. as for myself, i'm using it as part of a backup program, where the target is a server with 3rd-party encryption - so i don't need to worry about that - and the web is filled with people who lost the ability to decrypt their backup for various reasons (no DRA, no backup of EFS keys, corrupted user profile, etc.). the answer to those is usually "you're screwed", and i don't want to go there.

 

(i also added this comment to Note #2 in the first post.)

[orbs]

updated the conversion script at first post:

reminder: the conversion script is intended to replace native functions, e.g. FileCopy(), to their LFN equivalent, i.e. _LFN_FileCopy().

if you have any customized functions, similar to native ones but with a suffix, e.g. FileCopyEx(), then the conversion script would ignore them.

now it also ignores customized functions similar to native ones but with a prefix, e.g. MyFileCopy(), which it was not ignoring before this update.

Edited November 6, 2014 by orbs

[orbs]

many updates - most of them internal. here's what's important: _LFN_FileSetAttrib did not behave correctly on recursion - subfolders were not processed. also top folder is not processed in some cases.

modified:

 

__LFN_FilesListToArray to list only folders that comply with mask, but still recurse to all subfolders.

 

_LFN_FileSetAttrib - several changes:

- incorporate the above to change attributes to folders as well as files

- reversion rules to accommodate recursion

- deprecate @error 3 ("No files found")

- handle top folder in recursive mode

 

added: internal functions, but useful for general purposes. pretty much do what they are named after:

 

__LFN_Target_IsExistingFile

__LFN_Target_IsExistingFolder

 

renamed: (internal functions. take care if you were using them directly):

 

__LFN_Target_LastElement      => __LFN_Target_LastElement_Get

__LFN_Target_UntilLastElement => __LFN_Target_LastElement_Trim

 

here's the complete list of changes:

 

modified for convenience: Null => 0
    _LFN_FileSetTime
modified for convenience: False => 0 for declaration of $nOverwrite
    _LFN_FileMove
added (mostly for readability, but useful for general purpose):
    __LFN_Target_IsExistingFile
    __LFN_Target_IsExistingFolder
modified to incorporate the above:
    _LFN_DirCopy
    _LFN_DirCreate
    _LFN_DirMove
    _LFN_DirRemove
    _LFN_FileCopy
    _LFN_FileMove
    _LFN_FileSetAttrib
    __LFN_FileCopyRaw
renamed:
    __LFN_Target_LastElement => __LFN_Target_LastElement_Get
    __LFN_Target_UntilLastElement => __LFN_Target_LastElement_Trim
modified function: __LFN_FilesListToArray
    incorporate the above
    list only folders that comply with mask (but still recurse to all subfolders):
modified function: _LFN_FileSetAttrib
    incorporate the above to change attributes to folders as well as files
    reversion rules to accommodate recursion
    deprecate @error 3 ("No files found")
    handle top folder in recursive mode
many functions:
    modified comments syntax for parameters - added whitespace where needed
    modified comments syntax for functions that require only the unicode prefix - added parameters

 

[orbs]

two functional updates, and many internal ones for readability and usability.

modified:

 

reversion also for destination path (only single destination file allowed reversion until now):

_LFN_FileCopy (reversion allowed also for flag 64 on a non-encrypted source file)

_LFN_FileDelete

_LFN_FileMove

_LFN_FileSetAttrib

_LFN_FileSetTime

 

support for parent folder pointer ("..") in path - applies to practically all functions, but the root cause is amended in:

__LFN_Target_SetUnicodePrefix

 

here's the complete list of changes:

modified for better reversion: also for dst given as path and for flag 64 on a non-encrypted src
    _LFN_FileCopy
modified for readability & usability of reversion:
    _LFN_FileDelete
    _LFN_FileMove
    _LFN_FileSetAttrib
    _LFN_FileSetTime
modified to incorporate __LFN_GetNthOccuranceOfCharInString:
    _LFN_DirCreate
removed due to the above:
    __LFN_GetNthOccuranceOfCharInString:
modified for simpler use - redim and add the new data in a single call:
    __LFN_ReDim
    __LFN_ReDim2D
modified to incorporate the above:
    _LFN_FileSetAttrib
    _LFN_IniReadSectionNames
    __LFN_FilesListToArray
NOT modified to incorporate the above:
    _LFN_IniReadSection - default delimiter is a valid character!
modified to allow a delimiter:
    __LFN_Target_LastElement_Get
    __LFN_Target_LastElement_Trim
modified to support relative elements ("\..") in path:
    __LFN_Target_SetUnicodePrefix

[orbs]

updated - brand new, actually - GUI conversion script!

features:

choose where to include the LFN UDF from: built-in UDF folder, user-specific UDF folder, or target script folder:

choose your preferences: display font (monospace or proportional), create or do not create backup, include or do not include the UDF:

show details of the UDF, analyze the target script, and convert:

display all lines to be converted - the original line side-by-side with the converted line. hover a line for a tooltip display of the line context - shows 10 lines before and after:

enjoy!

[orbs]

UDF v5.2 released for AutoIt v3.3.14.2. list of changes:

 

removed (as "Encrypted" attribute is now returned by _LFN_FileGetAttrib)
    __LFN_FileGetAttribEx
modified to incorporate the above (using _LFN_FileGetAttrib instead of __LFN_FileGetAttribEx)
    _LFN_Filecopy
added
    _LFN_FileSetEnd
modified to using native functions, just applying the unicode prefix where needed
    _LFN_FileOpen
    _LFN_Filecopy
    _LFN_FileMove
    _LFN_FileDelete
removed (no longer in use, as _LFN_FileCopy was nativized)
    __LFN_FileCopyRaw

 

also released a compatible demo script.

 

important note for _LFN_FileCopy

in versions 3.x/4.x of this UDF, _LFN_FileCopy used some specific flags, and was quite an elaboration on the functionality of the native FileCopy. this is no longer the case.

 

 

Edited January 14, 2016 by orbs

[Deye]

Thanks orbs

I have used the converter to include LFN.au3 properties.
Didn't get a chance to test out a real case of an exceeding MAX_PATH yet
But I trust it will work just fine. I didn't find any problems so far
So in away, I'm setting it and forgetting it, for now

still, I do have one question:
will a generated array from _FileListToArrayRec, have a problem holding an exceeding MAX_PATH ?
And if it Will, what other ways can be used also to find where a long path exists, so I can Log that.

TIA