fruits of spare-time labour
own deeds:
After the last post regarding my OpenGL/gstreamer video-player I was contacted by a few people asking me for the source. I refused these requests sofar… almost. It was in a sorry state and still isn’t what I would be proud of, although it is better now. Nevertheless some people from Tandberg asked for it, in order to demonstrate the stylish power of mixing OpenGL with gstreamer for video-conferencing applications… and to add some extra bling-punch to a presentation they needed to give for their managers. So I gave in and send them the tarball. They hooked my code up with their webcam stuff (some of their equipment is able to capture 720p at 60Hz 30Hz!) and had a pretty successful presentation. The result of this you see in the left picture (this is using a more modest setting of 640×480@30Hz). I hope they will obey the LGPL I put my code under. Just received their tarball. I will see to sort and clean things as good as I can and upload the stuff this coming weekend at my cozy f.d.o spot.
 (click to view fullsize) |
 (click to view fullsize) |
The right picture shows the current state of affairs (playing back one of Microsoft’s WMV9/HD-example videos, which are publicly available *g*). gl-gst-player is now able to play back any video I can throw at it, e.g. ogg/Dirac (yes Dirac using the Schroedinger implementation! I tested that with a dirac-encoded version of “Elephants Dream”), ogg/Theora, mov/H.264, wmv/VC1 and all the other things gstreamer can digest. By now I was able to make some progress and have fragment-shader-based (only using the ARB-extensions, 4 multi-texture units, one texture for the yuv-frame and one for the mask-texture) YUV2RGB colorconversion (I420 to RGB to be more precise) running on my i915. This is on the application-side not at the driver-level. Now I can even watch HD-video on my Vaio which e.g. totem cannot play back smoothly or at all (mainly due to XV failing). Before starting to polish the code and put it in my public git-repository at f.d.o I want to finish the other filter-effects I need (gloom and a special blur). Sadly I don’t have FBOs or pbuffers available on my i915 and am restricted to glCopyTexSubImage2D()… and I wish I had GLSL available for the i915 *sigh* If all goes well and I find the needed time to do all that, this might still happen before GUADEC.
deeds at Fluendo:
Here is a nice example of the kind of the abstraction level Python-developers feel right at home at 
The blue thing (PowerBook) controlls the green thing (Nokia cell-phone) to issue commands within a python-session triggering bluetooth-commands, that are sent to the red thing (Vaio) acutally controlling the current upstream version of elisa. In this particular case, which is captured on the photo, is it selecting/playing/pausing a video. By now - two days later after the photo was taken - the bluetooth input-provider in elisa works good enough to just use the cell-phone (currently only tested on one special Nokia phone) as a bluetooth-remote for elisa running on a bluetooth-enabled computer.
 (click to view fullsize) |
This is the deed of Arek and Florian, two employees at Fluendo, working late at night on the weekend… abusing Arek’s PowerBook, Florians cell-phone and my Vaio. Too bad my own cell-phone isn’t bluetooth-capable and lacks a built-in python interpreter. BTW, on Florian’s cell-phone you can even write OpenGL|ES programs via python. That’s seriously nifty!
June 25th, 2007 at 7:42 am
Impressive. Keep up the good work!
June 25th, 2007 at 10:52 am
It\’s not a Macbook, it\’s a PowerBook (PPC, not Intel)!
June 25th, 2007 at 10:54 am
@ Erik: Thanks!
@ Movi: Ah right, corrected now.
June 25th, 2007 at 2:12 pm
looks interesting (both elisa and the bluetoothremote). If you need some help with testing I got a bluetooth enabled notebook and a nokia n80, so I should also be able to use this setup - and I’d love to try a new (hopefully working) mediacenter solution.
June 25th, 2007 at 3:20 pm
@ Pharao: Sure you can grab the elisa and pigment code from svn and see to get it working with your bluetooth-hardware. Afaik Arek and Florian only worked on getting it working with Florian’s Nokia smart-phone (forget the model number). But feel free to join #elisa on irc.freenode.org and bug Kaleon (Florian) and arkadini (Arek) about details.
To check out the current rewrite versions of pigment and elisa and compile and run them in place, use this…
cd /tmp
svn co https://core.fluendo.com/pigment/svn/branches/rewrite-1 pigment-rewrite
cd pigment-rewrite
./autogen.sh
make
./misc/pgm-uninstalled
cd ..
svn co https://core.fluendo.com/elisa/svn/branches/rewrite-1 elisa-rewrite
cd elisa-rewrite
python elisa.py sample_config/classic.conf
You can do that as normal user and don’t pollute your filesystem with in-development software. This is a nice "sandbox" to toy around with.
June 27th, 2007 at 12:55 am
nice work, is there any tutorials about implementing a gstreamer/openGL video player…also is there a way i could view you source? thanks.
June 27th, 2007 at 10:16 am
"Nevertheless some people from Tandberg asked for it, in order to demonstrate the stylish power of mixing OpenGL with gstreamer for video-conferencing applications"
Sound exciting - being profoundly deaf - I use videophone exclusively - as it’s a natural way of being able to use my sign language to converse with other people over a phone. The lack of a decent video-conf client for Linux (sure there’s Ekiga but it’s only H.261 at most and that’s very poor video, and Wengophone is another that I *could* use, but most of my friends are using Skype instead…) is a pain.
Cheers
June 28th, 2007 at 12:00 pm
@Collin: In a few days. I’ll blog about it when I polished it a bit more and have put it online.
@JGJones: What a cool use-case this is! Well, sorry… I mean not cool you being deaf, but you know what I mean… I hope
I don’t know much about the sign-language (btw is it generic across countries… like a deaf italian person can communicate with a deaf french person?), but I can imagine what you would want is the best image-quality and framerate you can possibly get, in order to correctly recognize all the subtle hand-movements and gestures. That being said you probably would want your video/audio stream to be handled by a more powerful codec than H.261, right? Better H.264 or Dirac I would guess. Maybe GL-acceleration not only makes totem nicer, but also provides a real-world benefit for Ekiga. BTW, I phoned with the Tandberg just yesterday and they don’t seem to offer any affordable or OpenSource solutions at all. I did my best to make them seriously consider it, though 
June 29th, 2007 at 8:13 am
Hi Mac MacSlow, what are the differences between pigment-0.1.5 and pigment-rewrite ? Thanks
Tony
July 16th, 2007 at 5:40 am
Hello MacSlow. Congratulations for your work. Would you give me a clue about how to implement OpenGL Textures output from a movie?
please!, just a little clue. When I look for a "how to" with Google your blog is the first result. Bye!
October 29th, 2007 at 10:15 pm
Canyou post a video of the OpenGL/gstreamer video-player in action?
October 30th, 2007 at 4:00 am
@ Paul B: You can have a look at it either here… http://macslow.thepimp.net/clips/gl-gst-player-1.ogg or here… http://www.youtube.com/watch?v=674lAWn9CSs and aside from that the code of it is in my git-repo located at http://gitweb.freedesktop.org/?p=users/macslow/gl-gst-player;a=summary
December 7th, 2007 at 9:47 pm
Hi! I noticed you mention you\’re using glCopyTexSubImage2D on intel hardware. How come it\’s working? I\’m told that isn\’t accelerated on the intel driver. (It\’s used, for instance, by Compiz\’ alpha blur plug-in, which isn\’t working for me.)
December 7th, 2007 at 10:00 pm
@ Bogdan: It works but is not fast. Better than not working at all. But in the (hopefully) near future we will get framebuffer-objects supported with the intel-driver and all will be good.
July 30th, 2008 at 11:57 am
Hi!
I would like to ask you if you can point me where to find a tutorial to use opengl with gstreamer or if you can give me some help.
I’m developing a quicklook for linux (I have it in a very good way http://launchpad.net/gloobus and I would like to develop a video plugin so I think that using gstreamer is the best way but I don’t know how to bind it with a gl texture…
I really apreciate your help!