Interesting things about the GIF image format

I recently took a deep dive into the GIF format. In the process I learnt a few things by reading the specification.

A GIF is made up of multiple images

 

I thought the GIF format would just contain a set of pixels. In fact, a GIF is made up of multiple images. So a simple example like:

 Could actually be made up of multiple images like this:

 

GIF has transparency, but that doesn't mean you have transparent GIFs

 

In the above example the sun and house images have the background in them. If the background was very detailed then this would be inefficient. So instead you can set a transparent colour index for each image. Pixels with this index don't replace the background pixels when the images are composited together.

That's the only transparency in the specification. The background colour is actually encoded in the file so technically a GIF picture has all pixels set to a colour. However at some point renderers decided they wanted transparency and ignored the background colour and set it to transparent instead. It's not in the spec, but it's what everyone does. This is the reason that GIF transparency looks bad - there's no alpha channel, just a hack abusing another feature.

You can have more than 256 colours

 

GIFs are well known for having a palette of only up to 256 colours. However, you can have a different palette for each image in the GIF. That means in the above example you could use a palette with lots of greens and blues for the background, lots of reds for the house and lots of yellows for the sun. The combined image could have up to 768 colours! With some clever encoding you can have a GIF file that uses up to 24 million colours.

Animation is just delaying the rendering 

GIFs are most commonly used for small animations. This wasn't in the original specification but at some point someone realised if you inserted a delay between each image you could make an animation! In the above example we could animate by adding more images of the sun that were rotated from the previous frame with a delay before them:

 

Why we can't have nice things

With all of the above GIF is both a simple but powerful format. You can make an animation that is made up of small updates efficiently encoded.

Sadly however someone decided that all images inside a GIF file should be treated as animation frames. And they should have a minimum delay time (including zero delays being rounded up to 20ms or so). So if you want you GIF to look as you intended you're stuck with one image per frame and only 256 colours per frame unless the common decoders are fixed. It seems the main reason they continue to be like this is there are badly encoded GIF files online and they don't want them to stop working.

GIF, you are a surprisingly beautiful format and it's a shame we don't see your full potential!

GIFs in GNOME

Here is the story of how I fell down a rabbit hole and ended up learning far more about the GIF image format than I ever expected...

We had a problem with users viewing a promoted snap using GNOME Software. When they opened the details page they'd have huge CPU and memory usage. Watching the GIF in Firefox didn't show a problem - it showed a fairly simple screencast demoing the app without any issues.

I had a look at the GIF file and determined:

• It was quite large for a GIF (13Mb).
• It had a lot of frames (625).
• It was quite high resolution (1790×1060 pixels).
• It appeared the GIF was generated from a compressed video stream, so most of the frame data was just compression artifacts. GIF is lossless so it was faithfully reproducing details you could barely notice.

GNOME Software uses GTK+, which uses gdk-pixbuf to render images. So I had a look a the GIF loading code. It turns out that all the frames are loaded into memory. That comes to 625×1790×1060×4 bytes. OK, that's about 4.4Gb... I think I see where the problem is. There's a nice comment in the gdk-pixbuf source that sums up the situation well:

 /* The below reflects the "use hell of a lot of RAM" philosophy of coding */

They weren't kidding. 🙂

While this particular example is hopefully not the normal case the GIF format has has somewhat come back from the dead in recent years to be a popular format. So it would be nice if gdk-pixbuf could handle these cases well. This was going to be a fairly major change to make.

The first step in refactoring is making sure you aren't going to break any existing behaviour when you make changes. To do this the code being refactored should have comprehensive tests around it to detect any breakages. There are a good number of GIF tests currently in gdk-pixbuf, but they are mostly around ensuring particular bugs don't regress rather than checking all cases.

I went looking for a GIF test suite that we could use, but what was out there was mostly just collections of GIFs people had made over the years. This would give some good real world examples but no certainty that all cases were covered or why you code was breaking if a test failed.

If you can't find what you want, you have to build it. So I wrote PyGIF - a library to generate and decode GIF files and made sure it had a full test suite. I was pleasantly surprised that GIF actually has a very well written specification, and so implementation was not too hard. Diversion done, it was time to get back to gdk-pixbuf.

Tests plugged in, and the existing code actually has a number of issues. I fixed them, but this took a lot of sanity to do so. It would have been easier to replace the code with new code that met the test suite, but I wanted the patches to be back-portable to stable releases (i.e. Ubuntu 16.04 and 18.04 LTS).

And with a better foundation, I could now make GIF frames load on demand. May your GIF viewing in GNOME continue to be awesome.

Counting Code in GNOME Settings

I've been spending a bit of time recently working on GNOME Settings. One part of this has been bringing some of the older panel code up to modern standards, one of which is making use of GtkBuilder templates.

I wondered if any of these changes would show in the stats, so I wrote a program to analyse each branch in the git repository and break down the code between C and GtkBuilder. The results were graphed in Google Sheets:

This is just the user accounts panel, which shows some of the reduction in C code and increase in GtkBuilder data:

Here's the breakdown of which panels make up the codebase:

I don't think this draws any major conclusions, but is still interesting to see. Of note:

• Some of the changes make in 3.28 did reduce the total amount of code! But it was quickly gobbled up by the new Thunderbolt panel.
• Network and Printers are the dominant panels - look at all that code!
• I ignored empty lines in the files in case differing coding styles would make some panels look bigger or smaller. It didn't seem to make a significant difference.
• You can see a reduction in C code looking at individual panels that have been updated, but overall it gets lost in the total amount of code.

I'll have another look in a few cycles when more changes have landed (I'm working on a new sound panel at the moment).

GUADEC 2018 Almería

I recently attended the recent GNOME Users and Developers European Conference (GUADEC) in Almería, Spain. This was my fifth GUADEC and as always I was able to attend thanks to my employer Canonical paying for me to be there. This year we had seven members of the Ubuntu desktop team present. Almería was a beautiful location for the conference and a good trade for the winter weather I left on the opposite side of the world in New Zealand.

This was the second GUADEC since the Ubuntu desktop switched back to shipping GNOME and it’s been great to be back. I was really impressed how positive and co-operative everyone was; the community seems to be in a really healthy shape. The icing on the cake is the anonymous million dollar donation the foundation has received which they announced will be used to hire some staff.

The first talk of the week was from my teammates Ken VanDine, Didier Roche and Marco Treviño who talked about how we’d done the transition from Unity to GNOME in Ubuntu desktop. I was successful in getting an open talk slot and did a short talk about the state of Snap integration into GNOME. I talked about the work I’d done making snapd-glib and the Snap plugin in GNOME Software. I also touched on some of the work James Henstridge has been working on making Snaps work with portals. It was quite fun to see James be a bit of a celebrity after a long period of not being at a GUADEC - he is the JH in JHBuild!

After the first three days of talks the remaining three days are set for Birds of a Feather sessions where we get together in groups around a particular topic and discuss and hack on that. I organised a session on settings which turned out to be surprisingly popular! It was great to see everyone that I work with online in-person and allowed us to better understand each other. In particular I caught up with Georges Stavracas who has been very patient in reviewing the many patches I have been working on in GNOME Control Center.

I hope to see everyone again next year!