FileGetTime() uses an improper TZ+DST shift at today's date

[LaurentC]

Hi

FileGetTime() uses an improper TZ+DST shift at today's date ...
... and not a TZ+DST shift at the date of the file itself !
Thus, +/-1h error on the time...

Here a sample :
Filename                                   : 20190115_180000_test.jpg
FileGetTime                              : 20190115190000

And the display file properties inside Windows 10 shows the correct 18:00 for Date Modified and not the AutoIt erroneous 19:00...   🙄  
That causes errors when setting faraway date/time with a string to a file with FileSetTime !

Regards

[jchd]

Are you sure?

Local $sFile = _TempFile()
FileWrite($sFile, "grnbuigrenpim")
ConsoleWrite(FileGetTime($sFile, 0, 1) & @LF)

Works as intended for me.

[Melba23]

jchd,

As I read the OP the problem is that the function uses the current timezone difference and not the difference at the time of file creation. The timestamps he shows (2019...) offer support to this interpretation.

M23

[Jos]

1 hour ago, LaurentC said:

FileGetTime() uses an improper TZ+DST shift at today's date ...
... and not a TZ+DST shift at the date of the file itself !

The FileGetTime() retrieves the Date/Time stamp of the creation time and then convert to Local time with FileTimeToLocalFileTime().
I tested with a picture created in the USA EST and the returned time is indeed 6 hours later as I am in CET.

So in your case: the picture was taking during Daylight saving so this all makes perfect sense as the function returns the current localtime never the original time at the time the picture was taken. 

Jos

Edited May 25, 2021 by Jos

[jchd]

Ah OK, didn't see where the issue was.

Unfortunately TZ+DST offsets are a huge can of worms. Not only DST are changing over time due to political decisions based on thin air, they also change over places which decide (again due to "administrative" reasons) to belong this side or the other side of a TZ boundary, which themselves are moved from times to times after "international" consensus or disagreement.

Furthermore, OS versions evolving, they aren't always consistent: DOS file times are not consistent with current Windows versions.

Hence computing the correct TZ+DST in the past is a nightmare.

The page https://docs.microsoft.com/en-us/windows/win32/sysinfo/file-times explains what a Windows filetime is:

 

A file time is a 64-bit value that represents the number of 100-nanosecond intervals that have elapsed since 12:00 A.M. January 1, 1601 Coordinated Universal Time (UTC). The system records file times when applications create, access, and write to files.

The NTFS file system stores time values in UTC format, so they are not affected by changes in time zone or daylight saving time. The FAT file system stores time values based on the local time of the computer. For example, a file that is saved at 3:00pm PST in Washington is seen as 6:00pm EST in New York on an NTFS volume, but it is seen as 3:00pm EST in New York on a FAT volume.

Time stamps are updated at various times and for various reasons. The only guarantee about a file time stamp is that the file time is correctly reflected when the handle that makes the change is closed.

Not all file systems can record creation and last access times, and not all file systems record them in the same manner. For example, the resolution of create time on FAT is 10 milliseconds, while write time has a resolution of 2 seconds and access time has a resolution of 1 day, so it is really the access date. The NTFS file system delays updates to the last access time for a file by up to 1 hour after the last access.

Read the full MS page for details. At the end of the day, you can see why it's almost impossible for any OS to go back in the past and compute what TZ and DST were in force when an event occured in the past.

Edited May 25, 2021 by jchd

[jchd]

Look at how complex things are in practice: the tz database is modified several times a year and it's really a huge thing!

https://en.wikipedia.org/wiki/Tz_database

[Jos]

Edited my comments a bit:

25 minutes ago, Jos said:

So in your case: the picture was taking during Daylight saving so this all makes perfect sense as the function returns the current localtime never the original time at the time the picture was taken. 

 

[LaurentC]

Hi Jos — Jchd — Melba23

 

I did not imagine that so many answers will come so quickly with so much insight : incredible !

 

1/

The fact is that before W7 with WinXP and older OS, the DST date/time was in accordance to the Windows properties displayed, as said in comments above

With W7 to W10, there is a change that corrects the properties displayed by Windows to be in coherency with the date of the filename, but yet not true for the old DOS DIR command...   (I don't know for PowerShell...)  

Thus, Windows 10 displays 18:00 for a JPG tagged 1800 = Good from the user point of view now !

 

2/

Quote

Jos : So in your case: the picture was taken during Daylight saving so this all makes perfect sense as the function returns the current localtime never the original time at the time the picture was taken.

 

Yes, it's exactly the subject   😉  

This was, let's say, OK in the past with old DOS/Windows settings, but NOK with regards to Windows new behavior

So, Jos, you're definitely right in your explanations

The fact is at time of the 20190115_180000_test.jpg picture, the time was really 18:00, and now, Windows displays 18:00, but FileGetTime shows 19:00 using current DST

Not understandable from the user's point of view since W7...

 

3/

But, as said above in the link proposed : https://docs.microsoft.com/en-us/windows/win32/sysinfo/file-times

I understand this is certainly a pure Windows mismatch !

These AutoIt functions only reflect Windows functions : these are not original written AutoIt libraries, but a useful mapping to Windows API (?).

If FileGetTime is a pure mapping of Win32 DLL functions and shall be kept as this, Jos, I understand your proposition is becoming even the rule...   😏  

 

4/ Question : Jos, does FileGetTime is a pure Windows library mapping in AutoIt or is that adjustable ?

 

5/ Question : Without even discussing code correction, would it be possible, at least, to add a remark in the AutoIt Help AutoIt.chm ?

Quote

 

https://docs.microsoft.com/en-us/windows/win32/sysinfo/file-times

...

FindFirstFile retrieves the local time from the FAT file system and converts it to UTC by using the current settings for the time zone and daylight saving time. Therefore, if it is daylight saving time, FindFirstFile takes daylight saving time into account, even if the file time you are converting is in standard time.

...

 

 

This will help AutoIt users not to wrongly manipulates dates with FileSetTime()...  

I would have been happy last week to read these forum  explanations in the documentation of AutoIt take care of DST...

 

Regards

[jchd]

Absolute time doesn't exist and every piece of software chooses it's starting point, e.g. Windows, Unix, Excel, you-name-it. And as shown, local time poison gets in the way. Even zulu time isn't perfect: not well defined before 1960 and even then the question remains: how to deal with leap seconds?

I agree that things aren't simple for the unsuspecting casual user and as always the devil is in the details.

Granted, a short note or warning could be beneficial in the help but that's as true for almost any language/system where the details are implied or just untold.

To make things even more subtle, time and time differences (durations) aren't what most people think they are: time can only be well defined inside a referential in space-time. Due to relativistic effects, your time isn't mine when one of us changes position in space. Two very precise synchronized cesium fountain atomic clocks get out of sync if one is raised only 50cm and reset at it's former position! The latest atomic clocks are very precise: a research institute in Spain has built a clock whose max drift is 0.263 second in 16.8 billion years. GPS wouldn't work if relativity wasn't taken into account: max precision would then be around 15km, making the whole system essentially useless!

So yes, time is "simple" for most of us in everyday's life (as teached in kindergardens), but once you start looking closer under the hood, horrible complexity shows its face.

[LaurentC]

Hi Jchd

 

I like science and I just can say how right you are ; the time is very subtle and even the time essence itself is not really known in physics

 

As far are today's software is concerned with the existing API, it is yet possible to give simple and easy ways to manage time in 99.9% of cases

 

What is disconcerting for a not-expert user is to take the date/time of 20190115_180000_test.jpg ; to put it in FileSetTime() that is the principal function provided in AutoIt, and to see 19:00 inside Windows in the file properties !?

In my case, this is 50% of errors to change date/time of files in the year for the user...

This is also not consistent now with the Windows user's experience...

 

I'm not a Windows developer, so I didn't know for example that AutoIt was giving an exact mapping of all internal Windows functions in its CORE Library

For example, with FileFindFirstFile and FileGetTime that are found in the Win32 API

And it's very interesting to go to the Windows detailed explanations when needed

I will have discovered the DST mess with : https://docs.microsoft.com/en-us/windows/win32/sysinfo/file-times

Thus, just a short note in this case in the FileGetTime AutoIt Help with a link to Win32 API File Times about DST would have been very helpful 

The AutoIt Function Reference could also refer to Win32 API with a link to propose additional information for those who could need it

Furthermore that AutoIt is already a very very well documented dev tool   😊  

 

I know now how to bypass with complex functions the "not-really-bug" I encountered with FileGetTime(), but the top of the top will be a function added to the CORE Library named FileGetTimeDST() or FileGetTime2() to propose the adjusted DST fix in a very simple way   😄  

 

What do you think ?

 

Regards

[jchd]

It won't work in the general case.

Pictures taken with your smartphone during your 2007 hollidays in Australia will be marked with the UTC. Same for those taken during your stay in Chile in 2013. Which filetime will you show?

To show a local time you have to decide which one: the local time of the picture at the place it was taken or the then local time of the device you use to watch them? Anyway, you have to embark a tz database and maintain it several times a year, plus know which place each picture was taken. Most Unices do that for you but Windows won't take care of all this by itself.

We human face a dilemna: suppose you take a photo with your smartphone in London at some time UTC (GMT+0). You fly to Australia and look at your photo. Your smartphone is smart enough and well connected, so it has changed timezone and knows which local time shift it has to compensate for in Australia. Which time should it show? The London time or the Aussie backward time?

In both cases there is an issue. Either you have to remember for every file where it was created to interpret time correctly (locally from the place of origin) or you can have situations where time goes "backward", where file A created before file B in UTC is displayed as more recent than B in current local timeframe where you have moved. That's a very strong argument for those who advocate dropping TZ and DST to use UTC only everywhere.

[LaurentC]

Yes

Right

But I didn't mean to correct all date/time possible issues, even for distant travelers

I said, as far are today's software is concerned with the existing API : that means TZ in Windows for the current user in its country zone

Unless Windows is able to get EXIF geo-info from pictures to choose the traveled TZ in the API : not possible just now

That way, even in our own local time zone, the FileGetTime does not display a time in coherency with what is displayed locally in Windows

As a user in my own country, even my local time is not correct with FileGetTime...

Regards

 

PS : That's a very strong argument for those who advocate dropping TZ and DST to use UTC only everywhere = That's true, but it is not used/displayed that way in smartphones and Windows = For a future implementation   😏   by Microsoft & all...

 

[LaurentC]

  

 

On 5/25/2021 at 1:43 PM, Jos said:

The FileGetTime() retrieves the Date/Time stamp of the creation time and then convert to Local time with FileTimeToLocalFileTime().
I tested with a picture created in the USA EST and the returned time is indeed 6 hours later as I am in CET.

So in your case: the picture was taking during Daylight saving so this all makes perfect sense as the function returns the current localtime never the original time at the time the picture was taken. 

Jos

 

Hi Jos

After all the previous discussions, we must forget my sample 20190115_180000_test.jpg : just a sample and not an essay on picture geo-localization

It's not a question for photos, but for all files on my Windows !

If I want to set 2021 15 01 18:00 to my files with FileSetTime(), it will put 19:00 visible in the file properties with Windows : that's the real bad trick here...

Ask me if all my peregrinations are unclear   🤔  

Regards

Edited May 26, 2021 by LaurentC

[JockoDundee]

I think it’s worth recognizing all the outstanding effort in time zone accommodations that various operating systems have put forward over the years, even if not perfect, to try to help us humans with our peculiar need to localize time based on space.

Aliens are laughing at us right now

[LaurentC]

Yes, aliens are laughing 

FileGetTimeDST() could stop the sniggering... 

Edited June 1, 2021 by LaurentC

[LaurentC]

No votes or remarks for an adjusted FileGetTimeDST() to fix the W10 new display method ? 

[Jos]

Just write a UDF for that and you make everybody happy.  

[LaurentC]

You're not the forum master for nothing 😀 !

Difficult to say something else than well noted and I shall try !! 

I appreciate your answer to position myself for the next steps... 

Anyway, I need to finalize my own function to achieve the result 😉 

Regards 

(Sorry for the English = I'm just a froggy French...)

[JockoDundee]

52 minutes ago, LaurentC said:

Sorry for the English = I'm just a froggy French...

In that case, I would just mention that the word peregrination, is not very common in modern English, and therefore its usage might leave the recipient at least fartumult if not outright farmished; or worse yet invite a retaliatory strike of comparable  inscrutables, due to the embarrassment of the recipient’s ignorance of the word peregrination.  Like in my case.

OTOH, if you were trying to be funny - good job

[LaurentC]

Too funny 

Interesting for me 

I like old school terms, that's true 

Really, I'm a little old school, exact 

And, i am funny, well I hope so, at least, I try to be funny 

Regards