Sunday, March 23, 2014

Why the display server doesn't matter

Display servers are the component in the display stack that seems to hog a lot of the limelight. I think this is a bit of a mistake, as it's actually probably the least important component, at least to a user.

In the modern display stack there are five main components:
  • Hardware
  • Driver
  • Display Server / Shell
  • Toolkit / Platform API
  • Applications
The hardware we have no control over. We just get to pick which hardware to buy. The driver we have more control over - drivers range from completely closed source to fully open source. There's a tug of war between the hardware manufacturers who are used to being closed (like their hardware) and the open source community which wants to be able to modify / fix the drivers.

For (too) many years we've lived with the X display server in the open source world. But now we are moving into next generation display servers (as Apple and Microsoft did many years ago). At the moment there are two new classes of contender for X replacement, Mir and a set of Wayland based compositors (e.g. weston, mutter-wayland etc).

Applications use toolkits and platform APIs to access graphical functionality. There are plenty of toolkits out there (e.g. GTK+, Qt) and existing libraries are growing more broad, consistent and stable to be considered as a complete platform API (which is great for developers).

If you read the Internet you would think the most important part in this new world is the display server. But actually it's just a detail that doesn't matter that much.
  • Applications access the display server via a toolkit. All the successful toolkits support multiple backends because there's more than one OS out there today. In general you can take a GTK+ application and run it in Windows and everything just works.
  • The hardware and drivers are becoming more and more generic. Video cards used to have very specialised functionality and OpenGL used to provide only a fixed function function. Now video cards are basically massively parallel processors (see OpenCL) and OpenGL is a means of passing shaders and buffer contents.
The result of this is the display server doesn't matter much to applications because we have pretty good toolkits that already hide all this information from us. And it doesn't matter much to drivers as they're providing much the same operations to anything that uses them (i.e. buffer management and passing shaders around).

So what does matter now?
  • It does matter that we have open drivers. Because there will be different things exercising them we need to be able to fix drivers when display server B hits a bug but A doesn't. We saw this working with Mir on Android drivers. Since these drivers are only normally used by SurfaceFlinger there are odd bugs if you do things differently. Filing a bug report is no substitute to being able to read and fix the driver yourself.
  • The shell matters a lot more. We're moving on from the WIMP paradigm. We have multiple form factors now. The shell expresses what an application can do and different shells are likely to vary in what they allow.
I hope I've given some insight into the complex world of display stacks and shown we have plenty of room for innovation in the middle without causing major problems to the bits that matter to users. 

Tuesday, March 12, 2013

Ubuntu GNOME

Thanks to the hard work of Jeremy Bicha and others Ubuntu GNOME is now an official Ubuntu flavour. Flavours get some infrastructure and support benefits such as ISO creation that make it easier to release and support.

More information on Mir

With the recent announcement of Mir there's been some concern about what this means for Ubuntu and the wider Linux ecosystem. Christopher Halse Rogers who is on the Mir team has written some excellent posts covering some of the major questions: why Mir and not Wayland/Weston, what does this mean for other desktops on Ubuntu and what does this mean for Linux graphics drivers.

Well worth the read.

Tuesday, March 05, 2013


Today we go public with the Ubuntu graphics stack for the post X world. Since the beginning Ubuntu has relied on the X server to support the user experience and while it has worked generally well; it’s time for something new. My team is working on a big new component for this - Mir. Mir is a graphics technology that allows us to implement user experience we want for Ubuntu across all devices we support.

In many ways, Mir will be completely transparent to the user. Applications that use toolkits (e.g. Qt, GTK+) will not need to be recompiled. Unity will still look like Unity. We will support legacy X applications for the foreseeable future.

This is a big task. A lot of work has already been done and there’s a lot more to go. We’re aiming to do incremental improvements, and you can find more about this on the Wiki page and in the blueprints. You can help. From today our project is public, it’s GPL licensed and you’re welcome to use the source and propose changes.

It’s exciting times, and I hope you enjoy the results of this work!

Thursday, January 10, 2013

Vala support for Protocol Buffers

Recently I was playing around with Protocol Buffers, a data interchange format from Google. In the past I have spent quite a bit of time working with ASN.1 which is a similar format that has been around for many years. Protocol buffers seem to me to be a nice distillation of the useful parts of efficient data interchange and a welcome relief to the enormous size of the ASN.1 specifications.

With Vala being my favourite super-productive language I felt the need to add support to it. Solution: protobuf-vala.

Let's see it in action. Say you have the following protocol in rating.proto:

message Rating {
  required string thing = 1;
  required uint32 n_stars = 2 [default = 3];
  optional string comment = 3;

Run it through the protocol buffer compiler with:

$ protoc rating.proto --vala_out=.

This will create a Vala file rating.pb.vala with a class like this:

public class Rating

  string thing;
  uint32 n_stars;
  string comment;

You can use this class to encode a rating, e.g. for storing to a file or sending over a network protocol:

var rating = new Rating ();
rating.thing = "Vala";
rating.comment = "Vala is super awesome!";
rating.n_stars = 5;
var buffer = new Protobuf.EncodeBuffer ();
rating.encode (buffer);
do_something_with_data (;

And decode it:

var data = get_data_from_somewhere ();
var buffer = new Protobuf.DecodeBuffer (data);
var rating = new Rating ();
rating.decode (buffer);
stderr.printf ("%s is %d stars\n", rating.thing, rating.n_stars);

That's pretty much it!

If you're using Ubuntu (12.04 LTS, 12.10 or 13.04) then you can install Vala protocol buffer support with:

$ sudo apt-add-repository ppa:protobuf-vala-team/ppa
$ sudo apt-get update
$ sudo apt-get install protobuf-compiler-vala libprotobuf-vala-dev

Have fun!

Thursday, December 27, 2012

A script for supporting multiple Ubuntu releases in a PPA

Something I find time consuming is uploading to a PPA when you want to support multiple Ubuntu releases. For my projects I generally want to support the most recent LTS release, the current stable release and the current development release (precise, quantal and raring when this was written).

I have my program in a branch and release that with make distcheck and lp-project-upload. The packaging is stored in another branch.

For each release I update the packaging with dch -i and add a new entry, e.g.

myproject (0.1.5-0ubuntu1) precise; urgency=low

  * New upstream release:
    - New exciting stuff

 -- Me <>  Thu, 27 Dec 2012 16:52:22 +1300

I then run and this generates three source packages and uploads them to the PPA:

RELEASES="raring quantal precise"

VERSION=`head -1 debian/changelog | grep -o '[0-9.]*' | head -1`
ORIG_RELEASE=`head -1 debian/changelog | sed 's/.*) \(.*\);.*/\1/'`
  cp debian/changelog debian/changelog.backup
  sed -i "s/${ORIG_RELEASE}/${RELEASE}/;s/0ubuntu1/0ubuntu1~${RELEASE}1/" debian/changelog
  bzr-buildpackage -S -- -sa
  dput ${PPA} ../${NAME}_${VERSION}-0ubuntu1~${RELEASE}1_source.changes
  mv debian/changelog.backup debian/changelog

Hope this is useful for someone!

Note I don't use source recipes as I want just a single package uploaded for each release.

Wednesday, November 21, 2012

Testing Kerberos in Ubuntu

In fixing a LightDM bug recently I needed to set up Kerberos authentication for testing. Now, Kerberos comes with quite a reputation for complexity so this was not a task I was looking forward to. And googling around to get some simple Ubuntu instructions only ended up confirming my expectations. But in the end, I was able to get it to work [1] and here is what I did. You should probably not rely on this information for an actual Kerberos implementation.

I start with two machines running Ubuntu, one as the Kerberos server [2] and one as a client. The client is already installed with a user account called test.

Server configuration

Edit /etc/krb5.conf to set the default realm [3]:
default_realm = TEST

Install the Kerberos server:

$ sudo apt-get install krb5-kdc krb5-admin-server

Create the realm. You will be prompted for a master password for the realm:

$ sudo krb5_newrealm

Add a new user (called a principal in Kerberos language) into the realm with the same username as on the client. You will be prompted for a password for this user [4]:

$ sudo kadmin.local
kadmin.local:  add_principal test

And now the server should be running. You can check things are working by watching the log:

$ tail -f /var/log/auth.log

Client configuration

The client is a lot easier, as the packages do most of the work for you:

$ sudo apt-get install krb5-user

You will be prompted for the following information:
  • Set "Default Kerberos version 5 realm" to TEST
  • Set "Kerberos server for your realm" to address / hostname of your server
  • Set "Administrative server for your Kerberos realm" to address / hostname of your server
Now you can test by getting a ticket [5] from the server. You will be prompted for the password you set when running kadmin.local on the server:

$ kinit
$ kdestroy

If that worked then you're ready to go. Have a look at the auth.log on the sever if it didn't work (the error messages are a bit cryptic though).

The next step is to setup PAM [6] to allow authentication with Kerberos. There's no configuration required, just install it:

$ sudo apt-get install libpam-krb5

Now you can log into your client machine (e.g. from LightDM/Unity Greeter) using the Kerberos password you setup on the server. Remember if something went wrong you can still use the local password to get in [7].

The reason I set all this up was to test Kerberos accounts which need password changes. You can control this feature from the server using the following:

$ sudo kadmin.local
kadmin.local:  modify_principal +needchange test

[1] on Ubuntu 13.04 (server) and 12.04 (client). I don't know which other combinations will work.
[2] Called a Key Distribution Centre in Kerberos jargon.
[3] Kerberos calls different authentication domains realms. I've used the realm TEST though in proper usage this would be a domain name e.g. EXAMPLE.COM to avoid name collision.
[4] You will already have a password set for this user on the client machine. Pick a different password as this allows you log in with either Kerberos or local passwords - both passwords will work.
[5] A ticket is the name for an authentication token provided by the server. In a real implementation this ticket will allow you to access services without re-entering your password.
[6] PAM is the library that does authentication when logging into Ubuntu.
[7] The PAM configuration that the packages setup first tries your password with the Kerberos server, then the local passwords (/etc/shadow) if that fails.

Thursday, February 09, 2012

So You Want to Write a LightDM Greeter…

Matt Fischer wrote a great post about writing a greeter for LightDM.  Runs through an example of a Python greeter and how it works.