DirectShow Frame grabber plugin

[monoceres]

Hi!

Very simple plugin to grab frames from any kind of video (anything your computer can play to be specific) using the DirectShow api.

I implemented this in a plugin because it sucks to work with any of the DirectX api's in autoit.

The plugin only exports this function:

GetVideoFrame($filename,$time[,$width,$height])

But who knows? Maybe there will be more.

Example shows how to use it.

AU3_FrameGrabber.dll

example.au3

Enjoy!

Edited May 31, 2009 by monoceres

[AdmiralAlkex]

Tested and works great

[trancexx]

I like it.

And I like the way you name your functions (Happy Easter from monoceres!)

[monoceres]

I like it.

And I like the way you name your functions (Happy Easter from monoceres!)

Thanks, and omg. This is truly awesome. I was just telling AdmiralAlkex that if anyone is going to find it, it would be you. You're awesome. Will you marry me?

[AdmiralAlkex]

Thanks, and omg. This is truly awesome. I was just telling AdmiralAlkex that if anyone is going to find it, it would be you. You're awesome. Will you marry me?

I think he was a little too quick... He must have hacked our conversation

[lordicast]

Thanks ! Was way in over my head on this plugin!

[trancexx]

Ok let's not get all that gayish.

I'm gonna have to take a raincheck on that proposal. My feminist side is not that strong.

So what was your motivation?

Nice job on frame job.

[Iuli]

Tested it, and even though i change the frame number, it grabs the same frame.

[monoceres]

Tested it, and even though i change the frame number, it grabs the same frame.

The second parameter is time (in seconds) not frame number. Adding an option to get the exact frame is on the todo.

[Iuli]

Still the same result. Changing the second, it will always return the same frame.

[monoceres]

Still the same result. Changing the second, it will always return the same frame.

I don't have that much control over seeking the stream. It's all handled automatically by DirectShow so my guess is that the filter that is handling the seeking is failing for some reason.

[Iuli]

Yep, i tried real-time seeking too, with the same result. Anyway, congrats for the plugin

[monoceres]

It's trivial to get the duration of the stream. I'll add it later.

[Roman9]

So what was your motivation?

Lol

I'd like to think we both know the answer to this.

Nice script, will definitely use it in the future!

[Splash]

It's possible use this to get a frame from a DirectX game?

[monoceres]

It's possible use this to get a frame from a DirectX game?

With this plugin - no. What in the world made you think that? >_<

[Splash]

I search about DirectX with AutoIt and

this appear. >_<

With this plugin - no. What in the world made you think that?

These:

http://gpalem.web.officelive.com/screencap.html

http://docs.google.com/gview?a=v&q=cache%3A9GtDBkXEyT8J%3Awww.dragonflydas.com%2FDocumentation%2FDASTool.pdf+inactive+window+capture&hl=pt-BR&gl=br&pli=1

http://delphi.about.com/od/delphitips2008/qt/print_window.htm

Sorry poor english...

[QED]

Hi monocres

Firstly, thank you for making this excellent plugin. I’m new to the forum and new to AutoIt.

I’ve been working on a small script to make “video contact sheets” – where you have say 30 shots of a movie laid out on one jpg image (with optional header info and timestamp on each shot). Media Player Classic (and some others) has this feature if you need an example, but I’m sure you know what I’m talking about.

Anyway, all the other programs never quite worked exactly as I want them to, so I decided to finally try writing my own. With AutoIt it couldn’t be easier – and that is coming from a non programmer! It’s plugins like yours that make it all possible!

At the moment I am using your “dsengine.dll” to get the necessary parameters from the media file (width, height and duration) then invoking your “GetVideoFrame()” in the FOR loop. The method works very well.

I’m now trying to optimise this because, as you can see it’s not very “elegant”. In your post above you said that it would be possible to get the Duration of the steam using FrameGrabber. Is it also possible to get Width and Height at the same time? That way I won’t have to use DSengine at all.

Another observation is that the frame grabbing seems to be quite slow compared to other programs that do the same thing (both directshow and ffmpeg based). Having had a quick look at the MSDN documentation (and not understanding much of it) I thought perhaps there is a lot of overhead in your plugin eg. initialising and closing which would be unnecessary when the function is called repetitively in a FOR loop. I’m no expert, so forgive me if I’m off track.

I’m also getting blank frames for mkv and mp4 files using FrameGrabber and I get corrupt blocky three colour pics with DSengine (with the same files). I’ve tweaked all the codec settings to no avail. WMP 11 and 12 play them no problems. My current work around is to create an AVISynth script on the fly (using DirectShowSource()) and passing that to your plugin. It works, but slower and also not elegant. Any thoughts on why mkv and mp4 don’t work natively in your plugin?

I also tried to use the frame grabbing feature in DSengine. It’s not always reliable and probably wasn’t designed to be used not-realtime. I’ve noticed it always syncs the audio (probably huge overhead) when it seeks. It got me thinking that perhaps there are similar unnecessary realtime restrictions in FrameGrabber. Is that a possible way to speed it up (disable audio, reference clocks etc.) or perhaps that has already been done?

My coding is not exemplary but I’d be happy to share any concepts of my little script if anybody is interested.

QED

[monoceres]

Hi monoceres

Hi!

I’m now trying to optimise this because, as you can see it’s not very “elegant”. In your post above you said that it would be possible to get the Duration of the steam using FrameGrabber. Is it also possible to get Width and Height at the same time? That way I won’t have to use DSengine at all.

Well it's not that bad, tha would be how mpc does it. Changing how the frame grabber works would be, uhm, hard, since I just opened the project folder just to realize it was empty (WTF?) However it's not a real loss since IMediaDet (the interface this is built upon is old and not really meant to be used anymore). I only made this plugin before I knew how to implement the more powerfull features (as you can see in dsengine.dll).

Another observation is that the frame grabbing seems to be quite slow compared to other programs that do the same thing (both directshow and ffmpeg based). Having had a quick look at the MSDN documentation (and not understanding much of it) I thought perhaps there is a lot of overhead in your plugin eg. initialising and closing which would be unnecessary when the function is called repetitively in a FOR loop. I’m no expert, so forgive me if I’m off track.

You are correct. The plugin will reopen the file every time. Extremely inefficient.

I’m also getting blank frames for mkv and mp4 files using FrameGrabber and I get corrupt blocky three colour pics with DSengine (with the same files). I’ve tweaked all the codec settings to no avail. WMP 11 and 12 play them no problems. My current work around is to create an AVISynth script on the fly (using DirectShowSource()) and passing that to your plugin. It works, but slower and also not elegant. Any thoughts on why mkv and mp4 don’t work natively in your plugin?

Sorry, I have no idea, I'm not a directshow guru in any way, the code will just initialize the preferred filter and try to use it.

I also tried to use the frame grabbing feature in DSengine. It’s not always reliable and probably wasn’t designed to be used not-realtime. I’ve noticed it always syncs the audio (probably huge overhead) when it seeks. It got me thinking that perhaps there are similar unnecessary realtime restrictions in FrameGrabber. Is that a possible way to speed it up (disable audio, reference clocks etc.) or perhaps that has already been done?

I think you're correct when you're saying that the audio is syncing, however I just looked at msdn and it doesn't seem to be possible to disable audio seeking during seeking. I can however add a flag when opening the file that makes it skip adding an audio filter.

[QED]

Thanks for your quick reply and hope you manage to find your project files soon.

My understanding now is that FrameGrabber is probably not capable of doing much more than it already does (eg. getting width and height) and in any case, it will still be inherently inefficient for repetitive FOR loops. As you said, perhaps it's better to focus on DSEngine.

I think you're correct when you're saying that the audio is syncing, however I just looked at msdn and it doesn't seem to be possible to disable audio seeking during seeking. I can however add a flag when opening the file that makes it skip adding an audio filter.

This, I think, is the reason why DSEngine is unreliable for me and the reason I turned to FrameGrabber. With DSengine I had to introduce a sleep() (horrible thought) after Engine_SetPosition() to allow enough time to seek before grabbing. The time needs to be longer depending on how complex the audio is (I guess more audio tracks = more latency). Your proposal to “skip adding an audio filter” in DSengine sounds very promising in this regard. This might be the key to fast and reliable frame grabbing with DSengine.

I think this is the “elegant solution” I’ve been looking for – just using DSEngine to do all the work. It grabs my mkv and mp4 files fine and I solved that “blocky” picture problem (accidently had jpg quality set to 1).

I look forward to trying out your next version of DSEngine.