Splitting File to a few files

[Kayage]

Dear script gurus,

I have a text file as big as 1GB++ (too big to be opened by notepad and would consume too much memory if opened up under wordpad. What I would like to do now, is to write a scrip to break that big file into a few text files (eg. to split that 1GB file to 4 250MB files or even more) for it to be managable.

I am new in AutoIT scripting and from what I read (the help function and sampels of scripts posted), I can either choose to read the 1GB file, line by line and write it to another file, setting criteria to save, close and open up new file to populate, thus breaking the file down to a few chunks; or

Read the whole file into an array and then break it out into mane smaller files.

Not too sure the latter method would work as my PC runs on 2GB of RAM (and from experience, the last time I attempted to open up the 1GB file, my RAM utilisation went up to 2.5GB!! almost crashing my OS!) Where else, the former method, could take ages to complete.

Is there another method to work on this?... pls advise. thank you.

[MiserableLife]

$f = Fileopen('file')
Do
FileRead($f,4096);4096 means read 4096 characters(or bytes in binary mode) each time
Until @error
FileClose($f)

So if

file.txt <-- "This is a string."

$f = Fileopen('file.txt')
$read = FileRead($f,5);$read = "This "
$read = FileRead($f,5);$read = "is a "
$read = FileRead($f,5);$read = "strin"
$read = FileRead($f,5);$read = "g."
$read = FileRead($f,5);<---- This will produce an @error,bec. it's end of file.
FileClose($f)

[James]

Kayage,

rasim made a neat file splitter a while back. Take a look here.

James

[martin]

Dear script gurus,

I have a text file as big as 1GB++ (too big to be opened by notepad and would consume too much memory if opened up under wordpad. What I would like to do now, is to write a scrip to break that big file into a few text files (eg. to split that 1GB file to 4 250MB files or even more) for it to be managable.

I am new in AutoIT scripting and from what I read (the help function and sampels of scripts posted), I can either choose to read the 1GB file, line by line and write it to another file, setting criteria to save, close and open up new file to populate, thus breaking the file down to a few chunks; or

Read the whole file into an array and then break it out into mane smaller files.

Not too sure the latter method would work as my PC runs on 2GB of RAM (and from experience, the last time I attempted to open up the 1GB file, my RAM utilisation went up to 2.5GB!! almost crashing my OS!) Where else, the former method, could take ages to complete.

Is there another method to work on this?... pls advise. thank you.

You could just copy a chunk at a time rather than read a line at a time or read the whole file to an array.

#include <winapi.au3>
Global $nbytes = 0, $chunk = 2000;2^28 for 250Mb
$tBuffer = DllStructCreate("byte[" & $chunk & "]")
$sFile = @ScriptFullPath
$hFile = _WinAPI_CreateFile ($sFile, 2, 2)
$size = _WinAPI_GetFileSizeEx ($hFile)

_WinAPI_SetFilePointer ($hFile, 0)
Global $Index = 1
While 1
    _WinAPI_ReadFile ($hFile, DllStructGetPtr($tBuffer), $chunk, $nbytes)
    
    If $nbytes = 0 Then ExitLoop
    ConsoleWrite("Read " & $nbytes & @CRLF)
    $tBuffer2 = DllStructCreate("byte[" & $nbytes & "]", DllStructGetPtr($tBuffer))
    $hF = FileOpen("File" & $Index & ".doc", 2)
    FileWrite($hF, DllStructGetData($tBuffer2, 1))
    FileClose($hF)
    $Index += 1
WEnd
_WinAPI_CloseHandle ($hFile)