Native API Compression

[slota]

Hi. With this example I want to show how using the plugin MemoryEx you can compress and decompress binary data using LZ1(LZ77) algorithm. Thus you reduce the file size. After processing Base64 you can use the compressed data in your project to dynamically retrieve. For example, if you do not want to store some files in the clear.

[slota]

New function Base64 decoding. Very fast.

[font=Courier New][size=2]Crypto[color=#FF0000].[/color]Base64Decode [color=#FF0000]=[/color] [color=#0000FF][b]function[/b][/color] (sData)
    [color=#0000FF][b]local[/b][/color] pcbBinary [color=#FF0000]=[/color] MemoryEx[color=#FF0000].[/color]AllocateEx([color=#FF6600]4[/color]);
    [color=#0000FF][b]local[/b][/color] sRet [color=#FF0000]=[/color] DLL[color=#FF0000].[/color]CallFunction([color=#800080]"Crypt32.dll"[/color][color=#FF0000],[/color] [color=#800080]"CryptStringToBinaryA"[/color][color=#FF0000],[/color] [color=#800080]"\""[/color][color=#FF0000].[/color][color=#FF0000].[/color]sData[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]"\", 0, 1, 0, "[/color][color=#FF0000].[/color][color=#FF0000].[/color]pcbBinary[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]", 0, 0"[/color][color=#FF0000],[/color] DLL_RETURN_TYPE_INTEGER[color=#FF0000],[/color] DLL_CALL_STDCALL);
    [color=#0000FF][b]if[/b][/color] (sRet [color=#FF0000]=[/color][color=#FF0000]=[/color] [color=#800080]'0'[/color]) [color=#0000FF][b]then[/b][/color] pcbBinary[color=#FF0000]:[/color]Free(); [color=#0000FF][b]return[/b][/color] [color=#800080]""[/color][color=#FF0000],[/color] [color=#800080]'Error calculating the length of the buffer needed.'[/color]; [color=#0000FF][b]end[/b][/color]
    [color=#0000FF][b]local[/b][/color] pbBinary [color=#FF0000]=[/color] MemoryEx[color=#FF0000].[/color]AllocateEx(pcbBinary[color=#FF0000]:[/color]Integer());
    [color=#0000FF][b]local[/b][/color] sRet [color=#FF0000]=[/color] DLL[color=#FF0000].[/color]CallFunction([color=#800080]"Crypt32.dll"[/color][color=#FF0000],[/color] [color=#800080]"CryptStringToBinaryA"[/color][color=#FF0000],[/color] [color=#800080]"\""[/color][color=#FF0000].[/color][color=#FF0000].[/color]sData[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]"\", 0, 1, "[/color][color=#FF0000].[/color][color=#FF0000].[/color]pbBinary[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]","[/color][color=#FF0000].[/color][color=#FF0000].[/color]pcbBinary[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]", 0, 0"[/color][color=#FF0000],[/color] DLL_RETURN_TYPE_INTEGER[color=#FF0000],[/color] DLL_CALL_STDCALL);
    [color=#0000FF][b]if[/b][/color] (sRet [color=#FF0000]=[/color][color=#FF0000]=[/color] [color=#800080]'0'[/color]) [color=#0000FF][b]then[/b][/color] pcbBinary[color=#FF0000]:[/color]Free(); pbBinary[color=#FF0000]:[/color]Free(); [color=#0000FF][b]return[/b][/color] [color=#800080]""[/color][color=#FF0000],[/color] [color=#800080]'Error decoding.'[/color]; [color=#0000FF][b]end[/b][/color]
    [color=#0000FF][b]local[/b][/color] sRet [color=#FF0000]=[/color] pbBinary[color=#FF0000]:[/color]LString(pbBinary[color=#FF0000]:[/color]Size());
    pcbBinary[color=#FF0000]:[/color]Free(); pbBinary[color=#FF0000]:[/color]Free();
    [color=#0000FF][b]return[/b][/color] sRet;
[color=#0000FF][b]end[/b][/color][/size][/font]

[jakabo]

would it also make base64 decoding to a database fast ?
It is slow to decode amd encode image file to a database...

[slota]

Check it out.

[jakabo]

would it be the same way to encode with Crypt32.dll ?

Crypto.Base64Encode = function (sData)

[slota]

would it be the same way to encode with Crypt32.dll ?

I have a couple of hours working on it, but not coming out. :( Can someone tell me why this code does not convert to BASE64 encoded?

[font=Courier New][size=2]Crypto[color=#FF0000].[/color]Base64Encode [color=#FF0000]=[/color] [color=#0000FF][b]function[/b][/color] (sData)
    [color=#0000FF][b]local[/b][/color] cbBinary [color=#FF0000]=[/color] MemoryEx[color=#FF0000].[/color]AllocateEx(sData[color=#FF0000]:[/color]len());
    cbBinary[color=#FF0000]:[/color]LString(cbBinary[color=#FF0000]:[/color]Size());
    [color=#0000FF][b]local[/b][/color] pcchString [color=#FF0000]=[/color] MemoryEx[color=#FF0000].[/color]AllocateEx([color=#FF6600]4[/color]);
    [color=#0000FF][b]local[/b][/color] sRet [color=#FF0000]=[/color] DLL[color=#FF0000].[/color]CallFunction([color=#800080]"Crypt32.dll"[/color][color=#FF0000],[/color] [color=#800080]"CryptBinaryToStringA"[/color][color=#FF0000],[/color] cbBinary[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]","[/color][color=#FF0000].[/color][color=#FF0000].[/color]cbBinary[color=#FF0000]:[/color]Size()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]", 1, 0, "[/color][color=#FF0000].[/color][color=#FF0000].[/color]pcchString[color=#FF0000]:[/color]GetPointer()[color=#FF0000],[/color] DLL_RETURN_TYPE_INTEGER[color=#FF0000],[/color] DLL_CALL_STDCALL);
    [color=#0000FF][b]if[/b][/color] (sRet [color=#FF0000]=[/color][color=#FF0000]=[/color] [color=#800080]'0'[/color]) [color=#0000FF][b]then[/b][/color] pcchString[color=#FF0000]:[/color]Free(); cbBinary[color=#FF0000]:[/color]Free(); [color=#0000FF][b]return[/b][/color] [color=#800080]""[/color][color=#FF0000],[/color] [color=#800080]'Error calculating the length of the buffer needed.'[/color]; [color=#0000FF][b]end[/b][/color]
    [color=#0000FF][b]local[/b][/color] pszString [color=#FF0000]=[/color] MemoryEx[color=#FF0000].[/color]AllocateEx(pcchString[color=#FF0000]:[/color]Integer());
    [color=#0000FF][b]local[/b][/color] sRet [color=#FF0000]=[/color] DLL[color=#FF0000].[/color]CallFunction([color=#800080]"Crypt32.dll"[/color][color=#FF0000],[/color] [color=#800080]"CryptBinaryToStringA"[/color][color=#FF0000],[/color] cbBinary[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]","[/color][color=#FF0000].[/color][color=#FF0000].[/color]cbBinary[color=#FF0000]:[/color]Size()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]", 1, "[/color][color=#FF0000].[/color][color=#FF0000].[/color]pszString[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]", "[/color][color=#FF0000].[/color][color=#FF0000].[/color]pcchString[color=#FF0000]:[/color]GetPointer()[color=#FF0000],[/color] DLL_RETURN_TYPE_INTEGER[color=#FF0000],[/color] DLL_CALL_STDCALL);
    [color=#0000FF][b]if[/b][/color] (sRet [color=#FF0000]=[/color][color=#FF0000]=[/color] [color=#800080]'0'[/color]) [color=#0000FF][b]then[/b][/color] pszString[color=#FF0000]:[/color]Free(); pcchString[color=#FF0000]:[/color]Free(); cbBinary[color=#FF0000]:[/color]Free(); [color=#0000FF][b]return[/b][/color] [color=#800080]""[/color][color=#FF0000],[/color] [color=#800080]'Error encoding'[/color]; [color=#0000FF][b]end[/b][/color]
    sRet[color=#FF0000]=[/color]MemoryEx[color=#FF0000].[/color]LString(pszString[color=#FF0000]:[/color]GetPointer()[color=#FF0000],[/color] pszString[color=#FF0000]:[/color]Size());
    pszString[color=#FF0000]:[/color]Free(); pcchString[color=#FF0000]:[/color]Free(); cbBinary[color=#FF0000]:[/color]Free();
    [color=#0000FF][b]return[/b][/color] sRet;
[color=#0000FF][b]end[/b][/color][/size][/font]

The function returns a string from the repeating letter "A".

[slota]

Issue resolved. Missed one argument to the function. Is highlighted in red. Working version:

[font=Courier New][size=2]Crypto[color=#FF0000].[/color]Base64Encode [color=#FF0000]=[/color] [color=#0000FF][b]function[/b][/color] (sData)
    [color=#0000FF][b]local[/b][/color] cbBinary [color=#FF0000]=[/color] MemoryEx[color=#FF0000].[/color]AllocateEx(sData[color=#FF0000]:[/color]len());
    cbBinary[color=#FF0000]:[/color]LString(cbBinary[color=#FF0000]:[/color]Size()[color=#FF0000],[/color] sData);
    [color=#0000FF][b]local[/b][/color] pcchString [color=#FF0000]=[/color] MemoryEx[color=#FF0000].[/color]AllocateEx([color=#FF6600]4[/color]);
    [color=#0000FF][b]local[/b][/color] sRet [color=#FF0000]=[/color] DLL[color=#FF0000].[/color]CallFunction([color=#800080]"Crypt32.dll"[/color][color=#FF0000],[/color] [color=#800080]"CryptBinaryToStringA"[/color][color=#FF0000],[/color] cbBinary[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]","[/color][color=#FF0000].[/color][color=#FF0000].[/color]cbBinary[color=#FF0000]:[/color]Size()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]", 1, 0, "[/color][color=#FF0000].[/color][color=#FF0000].[/color]pcchString[color=#FF0000]:[/color]GetPointer()[color=#FF0000],[/color] DLL_RETURN_TYPE_INTEGER[color=#FF0000],[/color] DLL_CALL_STDCALL);
    [color=#0000FF][b]if[/b][/color] (sRet [color=#FF0000]=[/color][color=#FF0000]=[/color] [color=#800080]'0'[/color]) [color=#0000FF][b]then[/b][/color] pcchString[color=#FF0000]:[/color]Free(); cbBinary[color=#FF0000]:[/color]Free(); [color=#0000FF][b]return[/b][/color] [color=#800080]""[/color][color=#FF0000],[/color] [color=#800080]'Error calculating the length of the buffer needed.'[/color]; [color=#0000FF][b]end[/b][/color]
    [color=#0000FF][b]local[/b][/color] pszString [color=#FF0000]=[/color] MemoryEx[color=#FF0000].[/color]AllocateEx(pcchString[color=#FF0000]:[/color]Integer());
    [color=#0000FF][b]local[/b][/color] sRet [color=#FF0000]=[/color] DLL[color=#FF0000].[/color]CallFunction([color=#800080]"Crypt32.dll"[/color][color=#FF0000],[/color] [color=#800080]"CryptBinaryToStringA"[/color][color=#FF0000],[/color] cbBinary[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]","[/color][color=#FF0000].[/color][color=#FF0000].[/color]cbBinary[color=#FF0000]:[/color]Size()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]", 1, "[/color][color=#FF0000].[/color][color=#FF0000].[/color]pszString[color=#FF0000]:[/color]GetPointer()[color=#FF0000].[/color][color=#FF0000].[/color][color=#800080]", "[/color][color=#FF0000].[/color][color=#FF0000].[/color]pcchString[color=#FF0000]:[/color]GetPointer()[color=#FF0000],[/color] DLL_RETURN_TYPE_INTEGER[color=#FF0000],[/color] DLL_CALL_STDCALL);
    [color=#0000FF][b]if[/b][/color] (sRet [color=#FF0000]=[/color][color=#FF0000]=[/color] [color=#800080]'0'[/color]) [color=#0000FF][b]then[/b][/color] pszString[color=#FF0000]:[/color]Free(); pcchString[color=#FF0000]:[/color]Free(); cbBinary[color=#FF0000]:[/color]Free(); [color=#0000FF][b]return[/b][/color] [color=#800080]""[/color][color=#FF0000],[/color] [color=#800080]'Error encoding.'[/color]; [color=#0000FF][b]end[/b][/color]
    sRet [color=#FF0000]=[/color] pszString[color=#FF0000]:[/color]LString(pszString[color=#FF0000]:[/color]Size());
    pszString[color=#FF0000]:[/color]Free(); pcchString[color=#FF0000]:[/color]Free(); cbBinary[color=#FF0000]:[/color]Free();
    [color=#0000FF][b]return[/b][/color] sRet;
[color=#0000FF][b]end[/b][/color][/size][/font]

[slota]

[font=Courier New][size=2]cbBinary[color=#FF0000]:[/color]LString(cbBinary[color=#FF0000]:[/color]Size()[color=#FF0000],[/color] [COLOR="red"][B]sData[/B][/COLOR]);[/size][/font]

[jakabo]

thanks slota...

why not making an action plugin with this ? if it is possible...

[kingzooly]

So what will this let us do can we load in to a image object without flat file or is this just a faster means if base64?

[jakabo]

image or file to base64 - base64 to image or file

[slota]

why not making an action plugin with this ? if it is possible...

It is possible. But is it necessary?

@kingzooly
In this case, the BASE64 only need to ensure that you can insert a compressed binary data in your project as a script. Nothing more ..

[jakabo]

You will reach more people with Plugin than just a code...
But ofcourse it is still your choice...

Thanks anyway

[kingzooly]

It is possible. But is it necessary?

@kingzooly
In this case, the BASE64 only need to ensure that you can insert a compressed binary data in your project as a script. Nothing more ..

yes but when loading back in to AMS does it still need to be a flat file lets say it was a image?

[jakabo]

for example ;
if you want to store an image to db, then you can compress as text file (script) and if you want to load it back to AMS as an image then just uncompress it... so you can make it load faster to your project from db...