Writing to the Beginning of a file.

[Fossil Rock]

I'm not sure if this is possible but I want to write a short piece of code to the beginning of a file as opposed to appending it to the end (same concept opposite end).

It's safe to assume that the target file can be any size so file size may have an impact on performance.

Any suggestions ?

[water]

Write your code to file B, copy existing file A plus B to new file C, erase A and B, rename C to A.

[Fossil Rock]

Thanks water. Considering this may be done 100's or even thousands of times (on the same file), is there no more direct method to just 'inject' the code to the beginning?

If I'm working with a file that could potentially be in the 100's of MB's (even GB's) then this method would take a very, very long time.

[water]

Can you please elaborate why you need to write lines at the top of such large files? Maybe there is a better solution than moving GB's of data
What do you mean by "code"?

[Fossil Rock]

It's just an experiment I'm testing, but if I can get it streamlined to be as fast as possible then it may have some useful purposes down the road.

[AutoBert]

Use fileread, filewrite: 

$sFilePath='output.txt' ;here fullpath of your file to be changed
$hFile=FileOpen($sFilePath,0)
$sOld=FileRead($hFile)
FileClose($hFile)
$hFile=FileOpen($sFilePath,2)
$sNew='New line 1'&@CRLF&$sOld
FileWrite($hFile,$sNew)
FileClose($hFile)

 

[orbs]

@Fossil Rock,

theoretically it should be possible, although with considerable limitations. but you must be extremely proficient with file tables, streams and the rest of the gang. no-one, i dare say even most MS filesystem engineers, is not in that level. either go with the rewrite method, or rethink your goal entirely.

if it's of any consolation to you, even UNIX does not offer that feature.

[Fossil Rock]

That works @AutoBert, thanks. My initial test produced a speed of 80-90 seconds for 10,000 iterations (roughly 160KB).

The speed issue is with having to write the file each time. For each iteration that time gets longer and longer.

The $FO_APPEND method in AutoIt can do 10,000,000 iterations in 60 seconds (roughly 152MB).

Now if there were just a $FO_PREPEND that would be awesome.

Edited July 4, 2016 by Fossil Rock

[AutoBert]

_FileWriteLog can write at begin of a file. So i suggest copy this func and rename the copy. After that edit to your needs, maybe this func is faster than my solution.

[orbs]

_FileWriteLog does exactly the same - read the source, then write it back.

if insisting on the rewrite approach, then the best optimization would be to skip all unnecessary pre/post-checks. in that sense, the solution suggested in post #6 by @AutoBert is probably the fastest possible.

[jchd]

I don't know of a filesystem able to prepend data to a flat file.

If your use case is going to repeatedly prepend data to large files, then flat files aren't suitable. You'd be much, much happier with a database, like SQLite. We can assist you getting started with this.

[orbs]

at this point this thread is no more than a Gedankenexperiment, as mentioned in post #5: "It's just an experiment I'm testing..."

if the OP wishes a more practical advice, he is probably aware of that other means and measures exist to serve whatever practical purpose he is after. if he chooses to share his purpose, he may get more concrete ideas.

 

[water]

Unfortunately _FileWriteLog first reads the file into memory. That can lead to storage problems when dealting with GB files  

[water]

I think the OP doesn't provide enough information to provide a working AND fast solution:

Quote

Considering this may be done 100's or even thousands of times (on the same file)  ...

I do not see a use case where this would be sensible.

Quote

'inject' the code to the beginning?

Why would anyone inject code into a GB file?

He tells us that injecting lines of code at the top of a file is the solution. He wants to implement it using AutoIt.
But he tells us nothing about the problem that he thinks can be solved by injecting lines of code at the top of a file

We need to know more about the problem before we can suggest a proper solution.

[jguinch]

What about using FileSetPos ?

#Include <FileConstants.au3>

Local $h = FileOpen("file.txt", 1)
_FileWriteAtTheTop($h, "second line" & @CRLF)
_FileWriteAtTheTop($h, "first line" & @CRLF)
FileClose($h)


Func _FileWriteAtTheTop($hFile, $sText)
    FileSetPos($hFile, 0, $FILE_BEGIN)
    Local $sNewText = $sText & FileRead($hFile)
    FileSetPos($hFile, 0, $FILE_BEGIN)
    FileWrite($hFile, $sNewText)
    FileFlush($hFile)
EndFunc

 

[water]

Has the same problem as orbs' solution. It reads the whole file into memory. Will take forever for a GB file

[jchd]

Not only the actual use case would be good to know, but writing to a file is only one side. There is no point doing that if the data is never processed. How it should be further processed (read) is the key point: there must be a reason why the OP inists on prepending data (I read "code" as data).

A common case where you'd like to have newest entries at top is with log files, where it's unpractical to have to read giant Gb or even Tb files to just search within a few last entries, which make flat files unsuitable in practice. Again, a well-designed DB is a much better choice, even if writing entries incurs a little time penalty.

Let's see if/when the OP tells us more.

[Danyfirex]

I think the way to perform this is divide files in chunks and create a new file to append all.

 

For Example

 

File A.data 1gb

File B.data 1gb

Create my new file C.data Create+Write+Append

Get file a.data 

DIvide in chunks 1gb/10

Loop and read 100Mbs (To avoid full file in memory) and append to C.data

Do the same with B.data File.

 

Saludos

 

Edited July 4, 2016 by Danyfirex