How to progressively store encrypted text in a file

[masvil]

A way to progressively store encrypted text in a file, then easy decrypt it.
 

expandcollapse popup

#include <Crypt.au3>

Local $Encrypted = @ScriptDir & "\Encrypted.txt"
Local $DecryptedJoined = @ScriptDir & "\DecryptedJoined.txt"
Local $password = "any"

_Crypt_Startup()

$hFileOpen = FileOpen($Encrypted, 10)
FileWrite($hFileOpen, String(_Crypt_EncryptData("One", $password, $CALG_AES_256) & " "))
FileClose($hFileOpen)

$hFileOpen = FileOpen($Encrypted, 1)
FileWrite($hFileOpen, String(_Crypt_EncryptData("Two", $password, $CALG_AES_256) & " "))
FileClose($hFileOpen)

$hFileOpen = FileOpen($Encrypted, 1)
FileWrite($hFileOpen, String(_Crypt_EncryptData("Three", $password, $CALG_AES_256) & " "))
FileClose($hFileOpen)

MsgBox (0,"", "Original strings have been encrypted, joined and separated by space in " & $Encrypted)

$hFileOpen = FileOpen($Encrypted, 0)
$EncryptedContent = StringTrimRight(FileRead($hFileOpen), 1) ; StringTrimRight is to cut out the last separator
FileClose($hFileOpen)

$SplittedString = StringSplit($EncryptedContent, " ")

FileDelete($DecryptedJoined)
For $i = 1 To $SplittedString[0]
    ; MsgBox(0, "Encrypted", $SplittedString[$i])
    ; MsgBox(0, "Decrypted", BinaryToString(_Crypt_DecryptData($SplittedString[$i], $password, $CALG_AES_256)))
    $hFileOpen = FileOpen($DecryptedJoined, 1)
    FileWrite($hFileOpen, BinaryToString(_Crypt_DecryptData($SplittedString[$i], $password, $CALG_AES_256)))
    FileClose($hFileOpen)
Next

MsgBox (0,"", "Encrypted Strings have been splitted in a array, one-by-one decrypted, and re-joined in " & $DecryptedJoined)

_Crypt_Shutdown()

$hFileOpen = FileOpen($DecryptedJoined, 0)
$DecryptedJoinedContent = FileRead($hFileOpen)
FileClose($hFileOpen)

MsgBox(0, "", "So the final result is: " & $DecryptedJoinedContent)

 

Edited November 24, 2016 by masvil

[jchd]

The ciphertext is binary so you should Fopen the file for binary write mode.

Then the encryption/decryption passwords don't match!

Finally what are you trying to achieve by appending data? It won't work like this.

[masvil]

@jchd I followed your suggestions and fixed the code, as you can see. The decryption still doesn't work.

I have to progressively append data because in my project I have the data progressively available. I thought the 4th _Crypt_EncryptData optional parameter ([, $bFinal = True]) it's useful to do so.

Edited November 23, 2016 by masvil

[j0kky]

EDIT: my first one was a stupid advice (and maybe this one too).

Have you tried to memorize all data in a single buffer and then write it to the file?

Edited November 23, 2016 by j0kky

[masvil]

1 hour ago, j0kky said:

Have you tried to memorize all data in a single buffer and then write it to the file?

Good advice, but I need to continuously store data to avoid loss from PC power failure or something.

[jchd]

When invoking CryptEncrypt() the data passed must adhere to the underlying Windows function requirements (https://msdn.microsoft.com/en-us/library/windows/desktop/aa379924(v=vs.85).aspx):

" When a block cipher is used, this data length must be a multiple of the block size unless this is the final section of data to be encrypted and the Final parameter is TRUE. "

Since you embed to encryption function call inside a FileWrite you have exactly zero chance to remark that an error was raised inside.

So your alternatives are either to use a stream cipher or to postpone encrypting stuff until you gather enough data (multiple of AES block).

[j0kky]

Just an idea, what about:

- filewrite the content encrypted by _Crypt_EncryptData (with last param setted to True)

- write to an .ini file the lenght, in bytes, written

- filewrite another content encrypted by _Crypt_EncryptData (with last param setted to True)

- write to an .ini file the lenght, in bytes, written

- ......

Then you can FileRead and decrypt each block of data (because you can retrieve its lenght by the .ini file)

[jchd]

Why not, but then you can spare the .INI files altogether by prepending the block length before the block itself, an int32 would do.

Note that AES uses a fixed 128-bit blocksize, so dealing with multiples of blocksize shouldn't be very hard indeed!

[j0kky]

13 hours ago, jchd said:

Note that AES uses a fixed 128-bit blocksize, so dealing with multiples of blocksize shouldn't be very hard indeed!

How do you apply it to the user case? He said he has data progressively available and he must continuosly store data to avoid loss...

It means if he sets Final param to False, he should manually pad each block of data he currently has to make it multiple of 128 bit blocksize and make algorithm works... And when he has another block of data he will do the same thing and so on until he has the final block which he will crypt with Final param setted to True (so the function automatically pads his data).

It works with no error, but when you try to decrypt this data, you get something like:

DATA pad DATA pad DATA pad

Which is not exatly useful, right?

[masvil]

@jchd and @j0kky you pointed me right, thank you. I updated my code, take a look.

I opted to simply use a space as separator while progressively storing the encrypted text, so no need to care about size of blocks (it really doesn't matter).

Then I use StringSplit (based on that space separator) to catch strings while decrypting and re-joining.

What do you think about this method?

And thank you to @Melba23 for his example on how to do some things.

Edited November 24, 2016 by masvil

[j0kky]

10 minutes ago, masvil said:

Then I use StringSplit (based on that space separator) to separate strings while decrypting.

What if an encrypted byte is 0x20 (which is ASCII white space)?

In my opinion my solution (improved by jchd's suggestion about Int32) will fit for you.

[masvil]

9 minutes ago, j0kky said:

What if an encrypted byte is 0x20 (which is ASCII white space)?

It works as well (if I well understood your question). You can replace one of the three original strings with " " in my code to check it out.

Edited November 24, 2016 by masvil

[j0kky]

I'm not saying your code fails when you have a whitespace in plaintext, I'm saying I'm not sure it will work if an encrypted byte (a plaintext byte which has been encrypted) results equal to 0x20, which (I repeat, I'm not sure) is interpreted as a whitespace.

Edited November 24, 2016 by j0kky

[masvil]

2 hours ago, j0kky said:

I'm not saying your code fails when you have a whitespace in plaintext, I'm saying I'm not sure it will work if an encrypted byte (a plaintext byte which has been encrypted) results equal to 0x20, which (I repeat, I'm not sure) is interpreted as a whitespace.

You mean maybe StringSplit could interprete an eventual 0x20 string in the text (clear or encrypted should be the same) as a white space (so a separator), right?

If so, I've coded an example (grabbed from the Help) to test it. It doesn't detect 0x20 as a white space separator:

Example()

Func Example()
    Local $aDays = StringSplit("Mon0x20Tues0x20Wed0x20Thur0x20Fri0x20Sat0x20Sun", " ") ; Split the string of days using the delimiter " " and the default flag value.

    For $i = 1 To $aDays[0] ; Loop through the array returned by StringSplit to display the individual values.
        MsgBox(0, "", "$aDays[" & $i & "] - " & $aDays[$i])
    Next
EndFunc   ;==>Example

 

[j0kky]

My doubt was about String() interpretation of binary data, but I noticed it translates "as is" into a string, adding the classical 0x in front of the string, so there should not be problems with white spaces.

[jchd]

51 minutes ago, masvil said:

"Mon0x20Tues0x20Wed0x20Thur0x20Fri0x20Sat0x20Sun"

These "0x20" are substrings, not characters!

[masvil]

1 hour ago, jchd said:

These "0x20" are substrings, not characters!

So how can I test if @j0kky's suspicion is reasonable?

[jchd]

An implementation of what I recommended in previous post, immune to character misinterpretation:

expandcollapse popup

#include <Crypt.au3>
#include <FileConstants.au3>

Local $Encrypted = @ScriptDir & "\Encrypted.txt"
Local $Decrypted = @ScriptDir & "\Decrypted.txt"
Local $password = "any"

_Crypt_Startup()

Local $vBlock
Local $hCipherText = FileOpen($Encrypted, $FO_OVERWRITE + $FO_CREATEPATH + $FO_BINARY)
$vBlock = _Crypt_EncryptData("One; ", $password, $CALG_AES_256)
FileWrite($hCipherText, BinaryLen($vBlock))
FileWrite($hCipherText, $vBlock)
FileClose($hCipherText)

$hCipherText = FileOpen($Encrypted, $FO_APPEND + $FO_BINARY)
$vBlock = _Crypt_EncryptData("Two but with text longer than AES blocksize; ", $password, $CALG_AES_256)
FileWrite($hCipherText, BinaryLen($vBlock))
FileWrite($hCipherText, $vBlock)
FileClose($hCipherText)

$hCipherText = FileOpen($Encrypted, $FO_APPEND + $FO_BINARY)
$vBlock = _Crypt_EncryptData("Three.", $password, $CALG_AES_256)
FileWrite($hCipherText, BinaryLen($vBlock))
FileWrite($hCipherText, $vBlock)
FileClose($hCipherText)

MsgBox (0,"", "Original strings have been encrypted separately, with their binary length prepended, in " & $Encrypted)

Local $iFileSize = FileGetSize($Encrypted), $iBlockLength

$hCipherText = FileOpen($Encrypted, $FO_BINARY)
$hRecoverText = FileOpen($Decrypted, $FO_OVERWRITE + $FO_CREATEPATH)

While $iFileSize > 0
    $iBlockLength = Int(FileRead($hCipherText, 4))
    $iFileSize -= 4
    $EncryptedContent = FileRead($hCipherText, $iBlockLength)
    $iFileSize -= $iBlockLength
    FileWrite($hRecoverText, BinaryToString(_Crypt_DecryptData($EncryptedContent, $password, $CALG_AES_256)))
WEnd
FileClose($hRecoverText)

MsgBox (0,"", "Encrypted Strings have been decrypted and re-joined in " & $Decrypted)

Two points to consider:
  1) this works as long as plaintext strings are less than MaxInt32
  2) if you may have input strings containing Unicode characters beyond ANSI, they should be converted to UTF8 before encryption and from UTF8 after decryption.
 

[j0kky]

1 hour ago, masvil said:

So how can I test if @j0kky's suspicion is reasonable?

$t = DllStructCreate("byte element[1]")
$t.element = 0x20
ConsoleWrite(String($t.element) & @CRLF)

As I stated in post #15, it is not (instead the suspect is valid for BinaryToString function).

Anyhow jchd's code is better than yours, because it uses n bytes + 4 for each block of data of n bytes, while yours uses 2 * n bytes + 3 for each block.

Edited November 24, 2016 by j0kky