digital home of MacSlow

I care about:
Yesterday was my last day at Fluendo. I have finished my internship and will return to Germany next week. In between is GUADEC’07, which I will attend. There I will give two talks. My second GUADEC and I give talks… not just one, but two! It feels so unreal… but rock&roll!

During my internship I learned a lot (not necessarily about gstreamer, as one might expect *g*) and certainly grew due to this experience. Also I believe to have left my mark upon Fluendo. This sounds bold and maybe arrogant I can imagine, but is true to some extend. All in all it will benefit the free desktop in the long run. That is my sincere hope at least!

The time I chose to come to Barcelona for my internship turned out to be very eventful. In general in the IT-world, at Fluendo and for me personally. For example the result of the Ubuntu/Dell deal went online and turned into reality. That’s a major turning point in my opinion. Fluendo lost some of their - in my opinion - most valuable employees. That’s very sad! Myself, I had a couple of very interesting interviews, phone-calls and chats with numerous people, which make my head spin in excitement. Moreover I was invited to become member of a judging jury for OpenSource-projects at an event in Portugal… I mean come one… how flattering is that?

you might care about:
Carl Worth is going after bottlenecks and non-optimized paths in EXA puts a broad smile of anticipation on my face.

In recent times I had lowfat running on a Samsung Q1 Ultra (photo of device from flickr) and a ThinkPad X60 (photo of device from flickr). Sadly these machines are just single-touch and where not mine. Still these short periods of access to this type of hardware clearly showed me that I could already do very nice stuff with just single-touch, if I would have permanent access to this kind of hardware. I have to find a way to grab hold of one of those newer X61 tablets and a Q1 Ultra! Not having this for working on lowfat makes me feel I’m missing the future.

There were some first MultiTouch-efforts from some folks over at Denmark using lowfat. Super sweet stuff! (Sorry Jens, to not have blogged about this earlier!) *sigh* I still need to build my own multi-touch setup.

Having fun with my new gstreamer/OpenGL-fu.

(click to play back, ogg/theora, ~2.4 MBytes)

The code for gl-gst-player I uploaded to f.d.o in my cozy git-repo here. It is unfinished, unpolished, undocumented and probably will not work on anything but i915 or i945 graphics hardware (only tested on Ubuntu 7.04 and Fedora 7 sofar). There are tons of remaining issues, missing features and other things I want to do with. Be aware of that before you grab it. After all, this is my gstreamer-playground for lowfat so never expect gl-gst-player to ever be something that can be considered finished.

The reason for me being very calm on my blog is due to a busy schedule in the recent weeks. There was my personal financial disaster that was CeBIT, where I did a talk on the state of things regarding multimedia on GNOME. Sorry, no slides for this, because it was 80% demonstrating applications (or at least pretending to be using them). On the side of end-user-ready applications there is still a lot to do, in order to provide well-integrated native multimedia-applications for GNOME. Folks, PiTiVi needs you, seriously… give Edward Hervey a hand!

Next was a small brainstorming weekend in Hamburg with a few core gtk+-developers from Imendio. We stayed at rambokid’s place and did talking and tossing around ideas for the most part. My take from that weekend is this… if gtk+ gets the resources (developers, designers, funding) the sky’s the limit. I was invited to that meeting by Imendio after some of the Imendio-bunch saw my talk “bling it up - make it sexy” at FOSDEM 2007. The role I had at the meeting could be best described as “API-user stating wishes for gtk+’s future”. I also showed a few examples of stuff I’ve written, that offers things currently not possible within gtk+ itself today.

Upcoming is my internship at Fluendo in Barcelona starting this friday… rock&roll! During that three month period I’ll also be at the LGM2 and UDS Sevilla. I’m equally thrilled and intimidated by this intense road ahead.

On the software-development side of things I’m trying my best with getting stuff done that’s on my mind/TODO. Currently this means some base-work I need for lowfat and a few other things. One could call it a side-effect of those projects. Since I’m not in the mood for writing a whole essay on the topic, here’s a glimpse of that in unnarrated screencast-form…

(click to play back, ogg/theora, ~5.6 MBytes)

While feisty fawn herd5 is ruining my nights sleep, due to wifi not working with my prism54 card, I wanted to drop some notes of other stuff that’s cooking in my head and on my hard-disk for some time now.

lowfat will get an updated page with a proper design-document, a “10000 miles from above”-view, project goals and an additional long-term vision. I also plan to do a talk about it at Guadec’07. Apart from that I have to do a ton of hacking and further research on it (e.g. I want such a multitouch-display setup like Jeff Han has… somehow I got to get my hands on much a setup or build one myself!). lowfat is far from being finished! And yes it is OpenSource for all you sceptics. See here. But please remember, due to my lack of time (read: need to accept offers for paid work) I cannot waste any time to do handholding for end-users or review patches you might want to send in for a project that’s still so young and whichs code-base and design has not settled yet. Furthermore do not even think about packaging it for any distro. It would suggest a finished product to end-users, which is definetly not the case yet. In addition to that such actions would give it a bad reputation before it’s reached a stable state.

A humble attempt to avoid further misunderstandings comes now. lowfat is not meant or going to replace compiz, f-spot, elisa or similar projects. It is a thing of its own. You will see. If you think Apple does cool stuff, I can just mildly smirk.

Please remember, I’m a student, always suffering form a chronic lack of cash, eager to get my degree. As much as I would love to do nothing but work on putting my ideas for Gnome 3.0/4.0 into working code, real life asks other things from me.

I still need to finish the talk/demonstration about “multimedia on GNOME” for CeBIT next week, prepare for my internship at Fluendo (that’s a requirement for cs-studies) next month, maybe even a trip to the LGM2, continue and finish some contracted work. I try to do as much as I possibly can.

For those of you who where not able to come to FOSDEM and are interested in the eye-catching parts of my talk, I provide you with a screencast of the demo I hacked up to accompany my talk. This one is the second of the three demos (one, two, three) I showed in my talk to underline the facts I chatted about. To all trolls out there… this is about the effects and techniques used, not the content itself. So please spare me the “copycat”-reprovals.

(click to play back, ogg/theora, ~1.1 MBytes)

(click to play back, avi/h.264, ~2.6 MBytes)

Here is something I wanted to show also (at least in video-form as it does currently not work on a i915 graphics-chip with current drivers) during my talk, but was not able to finish it up properly. You can clearly see the remaining glitches. Running on a GeForce 7900GT using nvidia’s proprietary driver. It provides a modest glimpse of what a future gtk+ might be able to do if it is fully composite-aware…

(click to play back, ogg/theora, ~0.9 MBytes)

(click to play back, avi/h.264, ~2.0 MBytes)

All shown screencasts do not do the smoothness justice you see when running for real without the screen-recording going on. These kind of things will get much slicker once we have full EXA-accelerating drivers (paths for Copy, Solid and Composite) available in common distributions running on the latest and greatest X.org can/will provide. The screencast are all done under compiz.

BTW, the gtk+-developers are absolutely aware of missing features, shortcomings and expectations of users (both for end-users and for application developers). They have the drive and urge to improve and change things. So to the trolls… please just die! To everybody else… let go of everything and start helping out now. The more help they get form you, the faster we reach UI-toolkit heaven!

Any questions that might arise you may put in the comments-section or send me via email. And I have not forgotton about lowfat.

People have asked me for the slides of my talk. Note, that I generally don’t see much sense in putting just the slides of a talk online. In general a video-recording of a talk would make more sense. But this is not available, as there was no recording done in the GNOME dev-room at FOSDEM 2007. Be sure to read the license information once you grabbed the tarball of the talk from here. From the audience reactions I got during the talk and after it, I conclude that it was well received and people enjoyed it (and hopefully learnt something they did not know yet). I was really very nervous especially since some of the maintainers and contributors of projects I stressed and “criticized” sat in the audience. In retrospect, considering what I got to know about X.org and cairo at this FOSDEM-weekend, my talk should have had a revamp before presenting it. But this flexibility-presentation-fu I do not master yet. Keith Packard clearly put this into one sentence going something like: “You have to live with these kind of things, when dealing with the bleeding edge.”. Who am I to argue with the Packard *g*

FOSDEM itself was great. I almost managed to meet all the people I planned to meet and talk with them about the things that are on my mind. Sadly the brainstorming I wanted to do with Sven and Philip totally did not take place. I blame myself for not trying hard enough and being too distracted by the amount of other interesting things at FOSDEM. But at least Sven and Tim made sure I remember to stick around the gtk-devel mailing-list for discussing the things I wish would be available in future versions of gtk+. BTW, did you know that gtk+ and X.org need contributing man-power? Step in, contribute now and become world-famous!

As somehow expected, I mainly hang around with the X.org and gstreamer people as those projects are among the most interesting ones for me… from a desktop-graphics-stack perspective. I was glad to be able to meet Carl Worth, Keith Packard (the stuff X.org is made of), Michel Dänzer again and was lucky to meet Kristian Høgsberg (his talk on AIGLX… lots of very crucial information) and Øyvind Kolås (check out his talk on GEGL… I had no idea!) in person for the first time. Furthermore I also met Stéphane Marchesin again this year and enjoyed his talk about “X.org Myths”. Kristian’s call to arms “Let’s finish COMPOSITE” and Matthias thorough lecture titled “Video on dope” were also great. Sadly I didn’t make it in time for Stéphane’s update on nouveau and Keith’s summary on video drivers.

I also could not make it to Edward’s talk about “What’s new in gstreamer” and I could barely make it to Miguel’s Mono workshop. FOSDEM was really packed with a lot of things worth seeing… too much to see it all. The schedule made choosing talks to attend very hard for me this year.

I had good chunks of time chatting with Thomas and Philippe about Elisa. I’m looking forward to my time at Fluendo in a few weeks. As part of my requirements for an degree in computer science I need to do an internship. Finally the talks with Christian at Vilanova last year turn out to become reality. I’m thankful for this opportunity! My semester abroad, also needed for the degree, is hopefully going to turn into reality in 2008 as well. Then I aim for Bosten, USA.

I was pleasantly surprised to see Nat Friedman attending FOSDEM and being able to meet and talk with him in person for the first time ever. Also a great opportunity was meeting Michael Meeks, right after I gave my talk. I think Michael was around GUADEC’06 and LRL’06, but only this FOSDEM-weekend it worked out to be possible to chat with him a bit. A modest attempt to file a bug with OpenOffice in real life failed. So here is the result of doing things the correct way *g*

A large portion of people I saw again this time around are not mentioned here. Nevertheless meeting up with them again was as enjoyable as meeting folks I just mentioned. But I draw the line here as I don’t want to bore readers to death *g*

Finally, thanks to Thomas for his great guidance to the restaurant landscape in Brussels this weekend, and thanks to intel for paying for the great dinner on sunday evening.

Oh, there is one more thing… within the next 24h lowfat will start its breathing in the open at f.d.o, but I still have to do some preparations. It’s tough for me as the amount of and work (and noise) I expect is going to be huge. It will get a dedicated blog-entry.

It looks like I’m up against some really tough competition (manpower- and/or cash-wise) with lowfat. There’s the multi-touch interaction research project by people of New York University’s Media Research Lab. That project already made some news a few months ago. Of more recent date is the project called BumpTop by people from the University of Toronto. While both of these projects target slightly different (from lowfat) goals and do not seem to strive for platform-independence, I still regard them as my “closest competition”. I love the multi-pointer interaction of the first one and like the approach of mouse-gestures in the second one.
Sort of related to my interests and plans for lowfat I found out about MPX (a multi-pointer X11 Server). That’s something I would like to be able to investigate at some point. Preferably on a tablet-PC with a spiffy OpenGL-chip and a well-accommodated Linux. Could this be a start of a multi-pointer extension for X11? I would definitely welcome such an outcome!
Damn, where are the lowfat-interested and filthyly rich VC-investors when you need them? Ah yeah right, I live in the wrong country for this… Germany here? *sigh*

Over the last couple of weeks I received some flak regarding my way with going about lowfat. Some bits I cover in the added FAQ-section of it. Thank god I also got encouraging notes from people, but the negative parts always weight heavier than the positive parts. This takes the motivation to spend your spare-time on it down considerably. Trying to see only the positive feedback and mask out the negative never works so well… at least for me. How do others deal with that “dark side” of community-reactions?

First of all a bunch of greetings from my side go to the readers of planet-gstreamer. By chance I figured out that probably Christian “Uraeus” Schaller added my blog to the sink (is this pun?) that is planet-gstreamer. Thanks for that! With being visible at planet-gnome and planet-gstreamer, does this mean I’ve to behave like an adult now? *g*

While still fooling around and learning gstreamer, I’ve built me a special pipleline that let’s me use H.264-encoded quicktime clips as video-textures. For some faad-related bug (gstreamer-bugzilla 340762 or launchpad 3236) I currently cannot use the vanilla playbin or decodebin under ubuntu dapper. Here you see a 480p HD trailer of “Superman Returns” rendered in an odd way:

(click to play back, ogg/theora, ~5.1 MBytes)

(click to play back, avi/h.264, ~4.6 MBytes)

The code-sample doing this can be downloaded from here. It has major shortcomings, but still might be useable for someone. I’m putting this online mainly because I hope that some gstreamer-savvy people will point me to solutions for the still inefficient parts that are found in this code. Thanks in advance! Some facts about the provided code:

• conversion to RGB done via pad-filter (slow), should be done with fragment-shader (fast)… too lazy right now
• too much hardcoded values, way too many globals… but hey it’s try-stuff-time
• superfluous copy of decoded video-frame to main-thread-local memory, no clue how to avoid that yet
• almost zarro comments

Yipiee! After some hard work and bugging the experts in #gstreamer on irc.freenode.net now and then, I’ve managed to implement video-textures mixing OpenGL and gstreamer. I know that togra already achieves something like this, but it is python-based and a whole scripting-framework. Thus it is of no use to me, because I will need video-textures in lowfat, which is not using python but C++. Also I would not want to pull a complete additional framework into lowfat.

(click to play back, ogg/theora, ~3.2 MBytes)

I’ll upload another example screencast (showing off a bit more… like some H.264-quicktime twirling around) when I find the time.