OpenCV v4 UDF

[malcev]

On 2/12/2022 at 1:09 PM, smbape said:

I have not looked at the source code of mediapipe.
If the python binding is also manually written, it will be a problem similar to dlib.

If You are interested here is dll

https://github.com/HW140701/GoogleMediapipePackageDll

[smbape]

Thanks @malcev

I saw that project but I was not satisfied.
I wrote a mediapipe binding that is similar to the python binding. It is not published yet because it is in the testing phase.
Once done, it will be published.

[smbape]

Update opencv to 4.7.0
COM Registration is no longer required

[mLipok]

@smbape can you move developing your UDF to github ?
I think then we can start involving.

[mLipok]

oops.
Is it here:
https://github.com/smbape/node-autoit-opencv-com

?

[malcev]

smbape, I think You have bug here:

$sManifest = EnvGet("OPENCV_ACTCTX_MANIFEST")

https://github.com/smbape/node-autoit-opencv-com/blob/main/autoit-opencv-com/udf/opencv_udf.au3#L123

Why user will have this environment variable?

May be it is easier just use direct path to manifest?

[smbape]

Hi @malcev

The environment variable is intended to override the embeded manifest.
It is not needed for scripting languages, but for compiled languages.
That being said, I kept it just in case.

For example, to use registration-free COM at compile time for opencv and mediapipe, the manifest used for activation context must have all the dependencies.
Since each of the libraries has its own manifest, environment varaibles are a way to reduce the complexity of specifying a manifest per library.

The following manisfest was put in a file
 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
    <assemblyIdentity type="win32" name="mediapipe.sxs" version="0.3.0.0" />

    <dependency>
        <dependentAssembly>
            <assemblyIdentity type="win32" name="autoit_opencv_com470.sxs" version="2.3.1.0" />
        </dependentAssembly>
    </dependency>

    <dependency>
        <dependentAssembly>
            <assemblyIdentity type="win32" name="autoit_mediapipe_com-0.9.1-470.sxs" version="0.3.0.0" />
        </dependentAssembly>
    </dependency>
</assembly>

and the environment variables point to that file, like in this c# example

https://github.com/smbape/node-autoit-mediapipe-com/blob/main/autoit-mediapipe-com/test/test.cs#L242

[malcev]

Thank You for explanation.

Can We activate manifest only 1 time at start with DllActivateManifest, and deactivate it only before exit script with DllDeactivateActCtx?

Can be some memory leaks?

Also if We exit script without calling DllDeactivateActCtx, will be some memory leaks?

[smbape]

I thought about it, but there are cases where the current activation context changes and that's because the activation contexts are per thread.
When a callback registered with DllCallbackRegister is called, the activation context changes.
Therefore, creating objects (in a c++ method or in autoit) will no longer work since the current activation context is not the one that contains the information about the COM objects.

There is also the case where different activation contexts are needed for different objects. For example if you use opencv and mediapipe at the same time.
To handle such a case, the manifest used for activation must have all dependencies, i.e. opencv and mediapipe.
But this is not enough, because the use in callbacks will not work.

From Microsoft about Isolating Components

Quote

Well-authored components do not perturb the environment of the hosting application, nor do they leak activation contexts. Well-authored components perform their own context management rather than relying on the hosting application's environment.

Activating before creating an object (_OpenCV_ObjCreate) or before calling a method/property (c++ GetTypeInfo/GetIDsOfNames/Invoke) ensures that the library objects always work.
Deactivating the context ensures that the library does not perturb the the environment of the hosting application.

[smbape]

1 hour ago, malcev said:

Can be some memory leaks?

I don't understand the question.
What makes you think of memory leaks.

 

1 hour ago, malcev said:

Also if We exit script without calling DllDeactivateActCtx, will be some memory leaks?

No, when your script dies, the memory associated with your script dies too.
Perhaps you mean something else.

[skijve]

Thx for the work.

At first sight, it seems soooo complicated to install :-)

[primetime]

Dunno if you'll ever read this, but I wanted to let you know that I'm testing this out and so far it's looking really great - wanted to commend you for the hard work on this - very incredible stuff, kudos!

[eumarcelino]

is there any way to run this compiled project on x86 systems?

[smbape]

There is no prebuilt opencv 32 bit.
If you really need it, you have to compile opencv and the com library for 32bit yourself.

[konya]

Share some information :

Bitmap to mat :

$cv.createMatFromBitmap($hImage2_o) 

 

Resolve other language code page path issue :

Local $STextHex = StringToBinary($_MODEL_FILE,1)
Local $tString = DllStructCreate("byte txt[" & BinaryLen($STextHex) & "]")
$tString.txt = $STextHex
Local $stringcc = _WinAPI_MultiByteToWideChar($tString, 1252, Default, True)

Local $base_options = $autoit.BaseOptions(_Mediapipe_Params("model_asset_path", $stringcc))

[smbape]

Thanks @konya

Can you give more information, a test case for example, about the code page path issue?
I would have tough that Local $base_options = $autoit.BaseOptions(_Mediapipe_Params("model_asset_path", $_MODEL_FILE)) would have worked.

[konya]

When I use chinese path open image or mediapipe ,it is will crash,convet code page to 1252 resolve issue.

 

$_MODEL_FILE = "c:\測試\test.jpg"

Local $STextHex = StringToBinary($_MODEL_FILE,1)

Local $tString = DllStructCreate("byte txt[" & BinaryLen($STextHex) & "]")

$tString.txt = $STextHex

Local $stringcc = _WinAPI_MultiByteToWideChar($tString, 1252, Default, True)

 

 

 

Edited May 28 by konya

[smbape]

Update OpenCV to 4.10.0

[smbape]

@konya There was actually a bug with non ascii strings.

I have tested language detector with mediapipe without issues.

Could you please retry with the new version without your workaround and tell me if it solves your issue?

[konya]

@smbape I'm have test new version but cant load dll,so still use old version 4.7.0 ,it is stable.

Edited July 12 by konya