tag:blogger.com,1999:blog-201559162024-03-14T05:58:30.199+13:00Bob's development blogMusings on open-source, GNOME, Ubuntu etcRobert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.comBlogger203125tag:blogger.com,1999:blog-20155916.post-58511528282092473062019-09-05T11:34:00.000+12:002019-09-05T11:34:59.759+12:00GUADEC 2019 - Thessaloniki<div dir="ltr" style="text-align: left;" trbidi="on">
I recently attended <a href="https://2019.guadec.org/">GUADEC 2019</a> in Thessaloniki, Greece. This is the seventh GUADEC I've attended, which came as a bit of a surprise when I added it up! It was great to catch up in person (some again, and some new!) and as always the face to face communication makes future online interactions that much easier.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-SntU_JfGiGE/XXBBEq6lAjI/AAAAAAAAZzE/3EPbskKCPBQo9E_lfg3qqByMB_IcpP2MQCLcBGAs/s1600/IMG_20190824_123405-01-01.jpeg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="527" data-original-width="1600" height="210" src="https://1.bp.blogspot.com/-SntU_JfGiGE/XXBBEq6lAjI/AAAAAAAAZzE/3EPbskKCPBQo9E_lfg3qqByMB_IcpP2MQCLcBGAs/s640/IMG_20190824_123405-01-01.jpeg" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Photo by Cassidy James Blaede</td></tr>
</tbody></table>
<br />
This year we had seven people from Canonical Ubuntu desktop team in attendance. Many other companies and projects had representatives (including Collabora, Elementary OS, Endless, Igalia, Purism, RedHat, SUSE and System76). I think this was the most positive GUADEC I've attended, with people from all these organizations actively leading discussions and a general consideration of each other as we try and maximise where we can collaborate. <br />
<br />
Of course, the community is much bigger than a group of companies. In particular is was great to meet Carlo and Frederik from the <a href="https://github.com/ubuntu/yaru">Yaru</a> theme project. They've been doing amazing work on a new theme for Ubuntu and it will be great to see it land in a future release.<br />
<br />In the <a href="https://www.gnome.org/wp-content/uploads/2019/06/GNOMEAnnualReport-2018-final.pdf">annual report</a> there was a nice surprise; I made the most merge requests this year! I think this is a reflection on the step change in productivity in GNOME since switching to GitLab. So now I have a challenge to maintain that for next year...<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-NFBRaTdcRJE/XXA7YNU2S8I/AAAAAAAAZy4/dXJCpfyR20ALPdC-EYW2cxugq8Q5lx2RQCLcBGAs/s1600/Screenshot%2Bfrom%2B2019-09-05%2B10-31-29.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="197" data-original-width="1342" height="93" src="https://1.bp.blogspot.com/-NFBRaTdcRJE/XXA7YNU2S8I/AAAAAAAAZy4/dXJCpfyR20ALPdC-EYW2cxugq8Q5lx2RQCLcBGAs/s640/Screenshot%2Bfrom%2B2019-09-05%2B10-31-29.png" width="640" /></a></div>
<br />
If you were unable to attend you can watch the all talks on <a href="https://www.youtube.com/playlist?list=PLkmRdYgttscEuv9v2-H9P5FBj8-td_Nri">YouTube</a>. Two talks I'd like to highlight; the first one is by Britt Yazel from the <a href="https://wiki.gnome.org/Engagement">Engagement team</a>. In it he talks about <i>Setting a Positive Voice for GNOME</i>. He talked about how open source communities have a lot of passion - and that has good and bad points. The Internet being as it is can lead to the trolls taking over but we can counter that but highlighting positive messages and showing the people behind GNOME. One of the examples showed how Ubuntu and GNOME have been posting positive messages on their channels about each-other, which is great!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/S8OoacfLep4/0.jpg" src="https://www.youtube.com/embed/S8OoacfLep4?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
The second talk was by Georges Basile Stavracas Neto and he talked <i>About Maintainers and Contributors</i>. In it he talked about the difficulties of being a maintainer and the impacts of negative feedback. It resonated with Britt's talk in that we need to highlight that maintainers are people who are doing their best! As state in the GNOME Code of Conduct - <i>Assume people mean well</i> (they really do!).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/HsCp3gFhXko/0.jpg" src="https://www.youtube.com/embed/HsCp3gFhXko?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
Georges and I are co-maintainers of <a href="https://gitlab.gnome.org/GNOME/gnome-control-center/">Settings</a> and we had a productive GUADEC and managed to go through and review all the open <a href="https://gitlab.gnome.org/GNOME/gnome-control-center/">merge requests</a>.<br />
<br />
There were a number of discussions around <a href="https://snapcraft.io/">Snaps</a> in GNOME. There seemed a lot more interest in Snap technology compared to last GUADEC and it was great to be able to help people better understand them. Work included discussions about portals, better methods of getting the Freedesktop and GNOME stacks snapped, Snap integration in Settings and the GNOME publisher name in the Snap Store.<br />
<br />
I hope to be back next year!</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-50941765817828353132019-02-12T10:49:00.000+13:002019-02-12T11:47:21.509+13:00linux.conf.au 2019<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>
Along with a number of other Canonical staff I recently attended <a href="https://linux.conf.au/">linux.conf.au</a>
2019 in Christchurch, New Zealand. I consider this the major
Australia/New Zealand yearly conference that covers general open source
development. This year the theme of the conference was "Linux of Things"
and many of the talks had an IoT connection.</div>
<div>
<br /></div>
<div>
One
of the premium swag items was a Raspberry Pi Zero. It is unfortunate
that this is not a supported Ubuntu Core device (CPU a generation too
old) as this would have been a great opportunity to show an Ubuntu Core
device in action. I did prepare a lightning talk showing some Ubuntu
Core development on a Raspberry Pi 3, but this sadly didn't make the
cut. You can see it in <a href="http://bobthegnome.blogspot.com/2019/02/easy-iot-with-ubuntu-core-and-raspberry.html">blog form</a>.</div>
<div>
<br /></div>
<div>
LCA
consistently has high quality talks, so choosing what to attend is
hard. Mostly everything was recorded and is viewable on their <a href="https://youtube.com/linuxconfau2019">YouTube channel</a>. Here is some highlights that I saw:</div>
<div>
<br /></div>
<div>
<a href="https://linux.conf.au/schedule/presentation/179/">STM32 Development Boards (literally) Falling From The Sky</a> (<a href="https://www.youtube.com/watch?v=YBy-bXEWZeM">video</a>)
- This talk was about tracking and re-purposing hardware from weather
balloons. I found it interesting as it made me think about the amount of
e-waste that is likely to be generated as IoT increases and ways in
that it can be re-cycled, particularly with open source software.</div>
<div>
<a href="https://linux.conf.au/schedule/presentation/103/"><br /></a></div>
<div>
<a href="https://linux.conf.au/schedule/presentation/103/">Plastic is Forever: Designing Tomu's Injection-Molded Case</a> (<a href="https://www.youtube.com/watch?v=Br5Ieo8USIw">video</a>) and <a href="https://linux.conf.au/schedule/presentation/140/">SymbiFlow - The next generation FOSS FPGA toolchain</a> (<a href="https://www.youtube.com/watch?v=-xyAauPa__s">video</a>)
- FPGA development is something that has really struggled to break into
the mainstream. I think this is mostly down to two things - a lack of a
quality open source toolchain and cheap hardware. These talks make it
seem like we're getting really close with the <a href="https://symbiflow.github.io/">SymbiFlow</a> toolchain and hardware like the <a href="https://www.crowdsupply.com/sutajio-kosagi/fomu">Fomu</a>.
I think we'll get some really interesting new developments when we get
something close to the Rasberry Pi/Arduino experience and I'm looking forward to writing some code in the FPGA and IoT space, hopefully soon!</div>
<div>
<br /></div>
<div>
<a href="https://linux.conf.au/schedule/presentation/156/">The Tragedy of systemd</a> (<a href="https://www.youtube.com/watch?v=o_AIw9bGogo">video</a>) - It's the conflict that just keeps giving ðŸ˜
Benno talked about how regardless of how systemd came to exist the
value of modern middleware is valuable. I had thought the majority had
come to this conclusion but it seems this is still an idea that needs
selling. I think the talk was effective in doing that.</div>
<div>
<a href="https://linux.conf.au/schedule/presentation/133/"><br /></a></div>
<div>
<a href="https://linux.conf.au/schedule/presentation/133/">Sequencing DNA with Linux Cores and Nanopores</a> (<a href="https://www.youtube.com/watch?v=CHCAb-PAqUI">video</a>) - This was a live (!) demonstration of doing DNA sequencing on the speakers lunch. This was done using the <a href="https://nanoporetech.com/products/minion">MinION</a>
- a USB DNA sequencer. As well as being able to complete the task what
impressed me was this was done on a laptop and no special software was
required. Given this device costs something around $1000 and is easy to
use this opens up DNA analysis to the open source world.</div>
<div>
<a href="https://linux.conf.au/schedule/presentation/114/"><br /></a></div>
<div>
<a href="https://linux.conf.au/schedule/presentation/114/">Around the world in 80 Microamps - ESP32 and LoRa for low-power IoT</a> (<a href="https://www.youtube.com/watch?v=bu6IaOrPZKU">video</a>)
- This discussed real world cases of building IoT / automation
solutions using battery power (e.g. solar not suitable). It covered how
it's very hard to run a Linux based solution for a long time on a
battery, but technology is slowly improving. Turns out the popularity of
e-scooters is making bigger and cheaper batteries available.</div>
<div>
<br /></div>
<div>
Christchurch has recently started trialing <a href="https://www.li.me/electric-scooter">Lime scooters</a>.
These were super popular with a hacker crowd and quickly accumulated
around the venue. I planned to scooter from the airport to the venue but
sadly that day there weren't any nearby, so I walked half way and
scootered the rest. They're super fun and useful so I recommend you try
them if you are visiting a city that has them. 🙂</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/--zAsfEda7Qo/XGHs-m8-qbI/AAAAAAAAW7U/iKchrIo0BjEnCqpoCScYEOXHOBSh0lpRwCLcBGAs/s1600/IMG_20190124_101628.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://4.bp.blogspot.com/--zAsfEda7Qo/XGHs-m8-qbI/AAAAAAAAW7U/iKchrIo0BjEnCqpoCScYEOXHOBSh0lpRwCLcBGAs/s640/IMG_20190124_101628.jpg" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-65122884701687290522019-02-05T14:26:00.001+13:002019-02-05T14:26:24.908+13:00Easy IoT with Ubuntu Core and Raspberry Pi<div dir="ltr" style="text-align: left;" trbidi="on">
My current job involves me mostly working in the upper layers of the desktop software stack however I started out working in what was then called embedded engineering but now would probably be know as the Internet of Things (IoT). I worked on a number of projects which normally involved taking some industrial equipment (radio infrastructure, camera control system) and adding a stripped down Linux kernel and an application.<br />
<br />
While this was cutting edge at the time, there were a number of issues with this approach:<br />
<ul style="text-align: left;">
<li>You essentially had to make your own mini-distribution to match the hardware you were using. There were some distributions available at the time but they were often not light weight enough or had a financial cost.</li>
<li>You had to build your own update system. That comes with a lot of security risks.</li>
<li>The hardware was often custom.</li>
</ul>
The above issues meant a large overhead building and maintaining the platform instead of spending that time and money on your application. If you wanted to make a hobby project it was going to be expensive.<br />
<br />
But we live in exciting times! It's now possible to use cheap hardware and easily accessible software to make a robust IoT device. For around $USD60 you can make a highly capable device using <a href="https://www.ubuntu.com/core">Ubuntu Core</a> and <a href="https://www.raspberrypi.org/">Raspberry Pi</a>. I decided to make a device that showed a <a href="https://shop.pimoroni.com/products/scroll-phat-hd?variant=2380803801098">scrolling LED display</a>, but there are many other sensors and output devices you could attach.<br />
<br />
The <a href="https://www.raspberrypi.org/products/raspberry-pi-3-model-a-plus/">Raspberry Pi 3 A+</a> is a good choice to build with. It was just recently released and is the same as the B+ variant but on a smaller board. This means you save some money and space but only lose some connectors that you can probably live without in an IoT device.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-7-siBL4pwFE/XFjLhUILOBI/AAAAAAAAW2I/3Zy3Bx4bQTEQVxoUj-XGNQEZ6J4pHHkkwCLcBGAs/s1600/3A-1-462x322.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="322" data-original-width="462" height="444" src="https://1.bp.blogspot.com/-7-siBL4pwFE/XFjLhUILOBI/AAAAAAAAW2I/3Zy3Bx4bQTEQVxoUj-XGNQEZ6J4pHHkkwCLcBGAs/s640/3A-1-462x322.jpg" width="640" /></a></div>
<br />
I added an SD card and for protection put it in a <a href="https://shop.pimoroni.com/products/pibow-3-a-plus-coupe?variant=17988387930195">case</a>. I chose an nice Ubuntu orange colour.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-n_h9Rpdxihs/XFjMt7neE1I/AAAAAAAAW2c/UrtXSYq7uXwKSelW4_TDvgRyPo_X5j0wQCLcBGAs/s1600/IMG_20190114_142528.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="480" src="https://2.bp.blogspot.com/-n_h9Rpdxihs/XFjMt7neE1I/AAAAAAAAW2c/UrtXSYq7uXwKSelW4_TDvgRyPo_X5j0wQCLcBGAs/s640/IMG_20190114_142528.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
Next step was to connect up a <a href="https://shop.pimoroni.com/products/scroll-phat-hd?variant=2380803801098">display</a> (also in Ubuntu orange). Note this didn't need the wires - it should fit flat onto the case but I spent too much time photographing the process that I accidentally soldered on the connector backwards. So don't make that mistake... 😕<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-wQjoiaT59Fw/XFjOCBC6g3I/AAAAAAAAW2o/0Z2jQwjNwZo8LgPIcNwRC5AOZeXbj1yHwCLcBGAs/s1600/IMG_20190115_163753.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1600" data-original-width="1200" height="640" src="https://3.bp.blogspot.com/-wQjoiaT59Fw/XFjOCBC6g3I/AAAAAAAAW2o/0Z2jQwjNwZo8LgPIcNwRC5AOZeXbj1yHwCLcBGAs/s640/IMG_20190115_163753.jpg" width="480" /></a></div>
<br />
Final step was to connect a USB power supply (e.g. a phone charger). The hardware is complete, now for the software...<br />
<br />
Using Ubuntu Core 18 is as simple as <a href="https://www.ubuntu.com/download/iot/raspberry-pi-2-3">downloading a file and copying it onto the SD card</a>. Then I put the SD card into the Raspberry Pi, powered it on and all I had to do was:<br />
<ol style="text-align: left;">
<li>Select my home WiFi network.</li>
<li>Enter my email address for my Ubuntu SSO account.</li>
<li>Secure shell into the Raspberry Pi from my Ubuntu laptop.</li>
</ol>
The last step is magically easy. If you connect a screen to the Pi it shows you the exact ssh command to type to log into it (i.e. you don't have to work out the IP address) and it uses the SSH key you have attached to your Ubuntu SSO account - no password necessary!<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">$ ssh robert-ancell@192.168.1.210 </span><br />
<br />
Now to write my application. I decided to <a href="https://github.com/robert-ancell/little-orange-display">write it in C</a> so it would be fast and h<span style="font-family: inherit;">ave very few dependencies. The easiest way to quickly develop was to cross-compile it on my Ubuntu laptop, then ssh the binary over the the Pi. This just required installing the appropriate compiler:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: "Courier New", Courier, monospace;">$ sudo apt install gcc-arm-linux-gnueabihf</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ arm-linux-gnueabihf-gcc test.c -o test</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ scp test robert-ancell@192.168.1.210:</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ ssh robert-ancell@192.168.1.210 ./test</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Once I was happy my application worked the next step was to package it to run on Ubuntu Core. Core doesn't use .deb packages, instead the whole system is built using <a href="https://snapcraft.io/">Snaps</a>.</span><br />
<br />
<span style="font-family: inherit;">All that is required to generate a snap is to fill out the following metadata (running <span style="font-family: "Courier New", Courier, monospace;">snapcraft init</span> creates the template for you): </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><span style="font-family: "Courier New", Courier, monospace;">name: little-orange-display<br />base: core18<br />version: git<br />summary: Demonstration app using Ubuntu Core and a Raspberry Pi<br />description: |<br /> This is a small app used to demonstrate using Ubuntu Core with a Raspberry Pi.<br /> It uses a Scroll pHAT HD display to show a message.<br /><br />architectures:<br /> - build-on: all<br /> run-on: armhf<br /><br />grade: stable<br />confinement: strict<br /><br />apps:<br /> little-orange-display:<br /> daemon: simple<br /> command: display-daemon<br /> plugs:<br /> - i2c<br /><br />parts:<br /> little-orange-display:<br /> plugin: make<br /> source: .</span></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">This describes the following:</span><br />
<ul style="text-align: left;">
<li><span style="font-family: inherit;">Information for users to understand the app.</span></li>
<li><span style="font-family: inherit;">It is an armhf package that is stable and confined.</span></li>
<li><span style="font-family: inherit;">It should run as a daemon.</span></li>
<li><span style="font-family: inherit;">It needs a special access to I2C devices (the display).</span></li>
<li><span style="font-family: inherit;">How to build it (use the Makefile I wrote).</span></li>
</ul>
<span style="font-family: inherit;">To test the package I built it on my laptop and installed the .snap file on the Raspberry Pi:</span><br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">$ snapcraft</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ scp little-orange-display_0+git.aaa6688_armhf.snap robert-ancell@192.168.1.210:</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ ssh robert-ancell@192.168.1.210</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ snap install little-orange-display_0+git.aaa6688_armhf.snap</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ snap connect little-orange-display:i2c pi:i2c-1</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ snap start little-orange-display</span><br />
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: inherit;"><br /></span></span></span>
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: inherit;">And it ran!</span></span></span><br />
<div>
<span style="font-family: inherit;"> </span><div>
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/h9NxaKmYOIY/0.jpg" src="https://www.youtube.com/embed/h9NxaKmYOIY?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">The last stage was to upload it to the <a href="https://snapcraft.io/store">Snap store</a>. This required me to register the name (little-orange-display) and upload it:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: "Courier New", Courier, monospace;">$ snapcraft register little-orange-display</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ snapcraft push little-orange-display_0+git.aaa6688_armhf.snap</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">And with that <a href="https://snapcraft.io/little-orange-display">little-orange-display is in the store</a>. If I wanted to make more devices I can by installing Ubuntu Core and enter the following on each device:</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: "Courier New", Courier, monospace;">$ snap install little-orange-display</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ snap connect little-orange-display:i2c pi:i2c-1</span><br />
<span style="font-family: "Courier New", Courier, monospace;">$ snap start little-orange-display</span><br />
<span style="font-family: inherit;"></span><br />
<span style="font-family: inherit;"><br /></span>
And that's the end of my little project. I spent very little time installing Ubuntu Core and doing the packaging and the majority of the time writing the app, so it solved the issues I would have traditionally encountered building a project like this.<br />
<br />
Using Ubuntu Core and Snaps this project now has following functionality available:<br />
<ul style="text-align: left;">
<li>It automatically updates.</li>
<li>The application I wrote is confined, so any bugs I introduce are unlikely to break the OS or any other app that might be installed.</li>
<li>I can use <a href="https://docs.snapcraft.io/channels/551">Snap channels</a> to test software easily. In their simplest usage I can have a device choose to be on the <i>edge</i> channel which contains a snap built directly from the git repository. When I'm happy that's working I can move it to the <i>beta</i> channel for wider testing and finally to the <i>stable</i> channel for all devices. </li>
<li>I get metrics on where my app is being used. Apparently it has one user in New Zealand currently (i.e. me). 🙂</li>
</ul>
<span style="font-family: inherit;"><br /></span></div>
</div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-77499462535786563342018-12-14T17:07:00.000+13:002018-12-14T17:07:59.461+13:00Interesting things about the GIF image format<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: left;">
I recently took a <a href="http://bobthegnome.blogspot.com/2018/12/gifs-in-gnome.html">deep dive into the GIF format</a>. In the process I learnt a few things by reading the <a href="https://www.w3.org/Graphics/GIF/spec-gif89a.txt">specification</a>.</div>
<div style="text-align: left;">
<br /></div>
<h3 style="text-align: left;">
A GIF is made up of multiple images</h3>
<h3 style="text-align: left;">
</h3>
<div style="text-align: left;">
I thought the GIF format would just contain a set of pixels. In fact, a GIF is made up of multiple images. So a simple example like:</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-9-6Secv5dDY/XBMcHfbJYtI/AAAAAAAAVl4/OiaztOc6hPomc3DPoCQvUKBcQj7OWzFSwCLcBGAs/s1600/gif-example.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="360" data-original-width="360" height="320" src="https://1.bp.blogspot.com/-9-6Secv5dDY/XBMcHfbJYtI/AAAAAAAAVl4/OiaztOc6hPomc3DPoCQvUKBcQj7OWzFSwCLcBGAs/s320/gif-example.png" width="320" /></a></div>
<br />
Could actually be made up of multiple images like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-m0fZYzAOQaw/XBMh-ZJZ6XI/AAAAAAAAVmY/NSbXcjk7MC4jKboAwZZ0Daon8omuG5sUwCLcBGAs/s1600/gif-example-breakdown.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="421" data-original-width="758" src="https://1.bp.blogspot.com/-m0fZYzAOQaw/XBMh-ZJZ6XI/AAAAAAAAVmY/NSbXcjk7MC4jKboAwZZ0Daon8omuG5sUwCLcBGAs/s1600/gif-example-breakdown.png" /></a> </div>
<h3 style="text-align: left;">
</h3>
<h3 style="text-align: left;">
GIF has transparency, but that doesn't mean you have transparent GIFs</h3>
<h3 style="text-align: left;">
</h3>
In the above example the sun and house images have the background in them. If the background was very detailed then this would be inefficient. So instead you can set a transparent colour index for each image. Pixels with this index don't replace the background pixels when the images are composited together.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-r-3BeOm2hC4/XBMld6i2kiI/AAAAAAAAVmk/FhGGUPYljzUACo74rrdCwcJRdAlwOMYhgCLcBGAs/s1600/gif-example-sun-transparent.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="150" data-original-width="150" src="https://4.bp.blogspot.com/-r-3BeOm2hC4/XBMld6i2kiI/AAAAAAAAVmk/FhGGUPYljzUACo74rrdCwcJRdAlwOMYhgCLcBGAs/s1600/gif-example-sun-transparent.png" /></a></div>
<br />
That's the only transparency in the specification. The background colour is actually encoded in the file so technically a GIF picture has all pixels set to a colour. However at some point renderers decided they wanted transparency and ignored the background colour and set it to transparent instead. It's not in the spec, but it's what everyone does. This is the reason that GIF transparency looks bad - there's no alpha channel, just a hack abusing another feature.<br />
<br />
<h3 style="text-align: left;">
You can have more than 256 colours</h3>
<h3 style="text-align: left;">
</h3>
GIFs are well known for having a palette of only up to 256 colours. However, you can have a different palette for each image in the GIF. That means in the above example you could use a palette with lots of greens and blues for the background, lots of reds for the house and lots of yellows for the sun. The combined image could have up to 768 colours! With some clever encoding you can have a GIF file that uses up to 24 million colours.<br />
<br />
<h3 style="text-align: left;">
Animation is just delaying the rendering </h3>
<h3 style="text-align: left;">
<br /></h3>
GIFs are most commonly used for small animations. This wasn't in the original specification but at some point someone realised if you inserted a delay between each image you could make an animation! In the above example we could animate by adding more images of the sun that were rotated from the previous frame with a delay before them:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-2weloqqsNFs/XBMovG_mB9I/AAAAAAAAVmw/uRSTJarDaI4xDxr54wKEXo9er_JglkH4gCLcBGAs/s1600/gif-example-animation.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="360" data-original-width="360" height="320" src="https://1.bp.blogspot.com/-2weloqqsNFs/XBMovG_mB9I/AAAAAAAAVmw/uRSTJarDaI4xDxr54wKEXo9er_JglkH4gCLcBGAs/s320/gif-example-animation.gif" width="320" /></a></div>
<h4 style="text-align: left;">
</h4>
<h3 style="text-align: left;">
Why we can't have nice things</h3>
<h3 style="text-align: left;">
<br /></h3>
With all of the above GIF is both a simple but powerful format. You can make an animation that is made up of small updates efficiently encoded.<br />
<br />
Sadly however someone decided that all images inside a GIF file should be treated as animation frames. And they should have a <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1511298">minimum delay time</a> (including zero delays being rounded up to 20ms or so). So if you want you GIF to look as you intended you're stuck with one image per frame and only 256 colours per frame unless the common decoders are fixed. It seems the main reason they continue to be like this is there are badly encoded GIF files online and they don't want them to stop working.<br />
<br />
GIF, you are a surprisingly beautiful format and it's a shame we don't see your full potential!</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-91765881228740768312018-12-14T15:15:00.001+13:002019-02-19T01:55:30.772+13:00GIFs in GNOME<div dir="ltr" style="text-align: left;" trbidi="on">
<div>
<ul id="docs-internal-guid-22dea3fe-7fff-c8cd-91dc-5c97fc5579a0" style="margin-bottom: 0pt; margin-top: 0pt; text-align: left;">
</ul>
<span style="font-family: "arial";">Here is the story of how I fell down a rabbit hole and ended up learning far more about the <a href="https://en.wikipedia.org/wiki/GIF">GIF image format</a> than I ever expected...</span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-LRKlDIKnGRg/XBMRAaGUO1I/AAAAAAAAVlQ/1CfVvlZqV6UYguZbwRYMBIGxnTz7gmdigCLcBGAs/s1600/text-animation-1s-280x100px.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="100" data-original-width="280" src="https://1.bp.blogspot.com/-LRKlDIKnGRg/XBMRAaGUO1I/AAAAAAAAVlQ/1CfVvlZqV6UYguZbwRYMBIGxnTz7gmdigCLcBGAs/s1600/text-animation-1s-280x100px.gif" /></a></div>
<div>
<span style="font-family: "arial";">We had a problem with users viewing a promoted snap using GNOME Software. When they opened the details page they'd have <a href="https://gitlab.gnome.org/GNOME/gdk-pixbuf/issues/101">huge CPU and memory usage</a>. Watching the GIF in Firefox didn't show a problem - it showed a fairly simple screencast demoing the app without any issues.</span></div>
<div>
</div>
<div>
<span style="font-family: "arial";">I had a look at the GIF file and determined:</span></div>
<div>
<ul style="text-align: left;">
<li><span style="font-family: "arial";">It was quite large for a GIF (13Mb).</span></li>
<li><span style="font-family: "arial";">It had a lot of frames (625).</span></li>
<li><span style="font-family: "arial";">It was quite high resolution (1790</span><span style="font-family: "arial";">×1060 pixels).</span></li>
<li><span style="font-family: "arial";">It appeared the GIF was generated from a compressed video stream, so most of the frame data was just compression artifacts. GIF is lossless so it was faithfully reproducing details you could barely notice. </span></li>
</ul>
</div>
<div>
<span style="font-family: "arial";">GNOME Software uses <a href="https://www.gtk.org/">GTK+</a>, which uses <a href="https://gitlab.gnome.org/GNOME/gdk-pixbuf">gdk-pixbuf</a> to render images. So I had a look a the GIF loading code. It turns out that all the frames are loaded into memory. That comes to </span><span style="font-family: "arial";">625</span><span style="font-family: "arial";"><span style="font-family: "arial";">×</span>1790</span><span style="font-family: "arial";">×1060</span><span style="font-family: "arial";">×4 bytes. OK, that's about 4.4Gb... I think I see where the problem is. There's a nice <a href="https://gitlab.gnome.org/GNOME/gdk-pixbuf/blob/1b232af452e6812822164291c54655a0c1b12c3f/gdk-pixbuf/io-gif-animation.h#L153">comment in the gdk-pixbuf source</a> that sums up the situation well:</span></div>
<div>
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "arial";"><span style="font-family: "courier new" , "courier" , monospace;"> /* The below reflects the "use hell of a lot of RAM" philosophy of coding */</span></span></div>
<div>
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "arial";">They weren't kidding. 🙂</span></div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial";"><a href="https://2.bp.blogspot.com/-Cmz0TJyhDco/XBMRuDGTzSI/AAAAAAAAVlg/3YDHQbalmckmNL8pOr6CStMaG0MgIoBCgCLcBGAs/s1600/44GB1_63699.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="40" data-original-width="278" src="https://2.bp.blogspot.com/-Cmz0TJyhDco/XBMRuDGTzSI/AAAAAAAAVlg/3YDHQbalmckmNL8pOr6CStMaG0MgIoBCgCLcBGAs/s1600/44GB1_63699.gif" /></a></span></div>
<span style="font-family: "arial";">
</span></div>
<div>
</div>
<div>
<span style="font-family: "arial";">While this particular example is hopefully not the normal case the GIF format has has somewhat come back from the dead in recent years to be a popular format. So it would be nice if gdk-pixbuf could handle these cases well. This was going to be a fairly major change to make.</span></div>
<div>
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "arial";">The first step in refactoring is making sure you aren't going to break any existing behaviour when you make changes. To do this the code being refactored should have <a href="https://en.wikipedia.org/wiki/Code_refactoring#Testing">comprehensive tests around it</a> to detect any breakages. There are a good number of GIF tests currently in gdk-pixbuf, but they are mostly around ensuring particular bugs don't regress rather than checking all cases.</span></div>
<div>
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "arial";">I went looking for a GIF <a href="https://en.wikipedia.org/wiki/Test_suite">test suite</a> that we could use, but what was out there was mostly just collections of GIFs people had made over the years. This would give some good real world examples but no certainty that all cases were covered or why you code was breaking if a test failed.</span></div>
<div>
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "arial";">If you can't find what you want, you have to build it. So I wrote <a href="https://github.com/robert-ancell/pygif">PyGIF</a> - a library to generate and decode GIF files and made sure it had a full <a href="https://github.com/robert-ancell/pygif/tree/master/test-suite">test suite</a>. I was pleasantly surprised that GIF actually has a <a href="https://www.w3.org/Graphics/GIF/spec-gif89a.txt">very well written specification</a>, and so implementation was not too hard. Diversion done, it was time to get back to gdk-pixbuf.</span></div>
<div>
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "arial";">Tests plugged in, and the existing code actually has a number of issues. I <a href="https://gitlab.gnome.org/GNOME/gdk-pixbuf/merge_requests/28">fixed them</a>, but this took a lot of sanity to do so. It would have been easier to replace the code with new code that met the test suite, but I wanted the patches to be back-portable to stable releases (i.e. Ubuntu 16.04 and 18.04 LTS).</span></div>
<div>
<span style="font-family: "arial";"><br /></span></div>
<div>
<span style="font-family: "arial";">And with a better foundation, I could now make <a href="https://gitlab.gnome.org/GNOME/gdk-pixbuf/merge_requests/31">GIF frames load on demand</a>. May your GIF viewing in GNOME continue to be awesome.</span></div>
<div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-Ek78MlkoFbQ/XBMRHX9kOpI/AAAAAAAAVlU/g6kj-2fpxC4_6V1Y_zc62wv0eXtwLxH_gCLcBGAs/s1600/success.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="45" data-original-width="430" height="33" src="https://3.bp.blogspot.com/-Ek78MlkoFbQ/XBMRHX9kOpI/AAAAAAAAVlU/g6kj-2fpxC4_6V1Y_zc62wv0eXtwLxH_gCLcBGAs/s320/success.gif" width="320" /></a></div>
<div>
</div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com1tag:blogger.com,1999:blog-20155916.post-66911673649229100102018-11-15T12:05:00.001+13:002018-11-16T09:05:15.770+13:00Counting Code in GNOME Settings<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
I've been spending a bit of time recently working on <a href="https://gitlab.gnome.org/GNOME/gnome-control-center">GNOME Settings</a>. One part of this has been bringing some of the older panel code up to modern standards, one of which is making use of <a href="https://developer.gnome.org/gtk3/stable/GtkBuilder.html">GtkBuilder templates</a>.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
I wondered if any of these changes would show in the stats, so I <a href="https://gist.github.com/robert-ancell/b84df04768c526471f5f7d88f2e32243">wrote a program</a> to analyse each branch in the git repository and break down the code between C and GtkBuilder. The results were graphed in Google Sheets:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-560aXn8MEyc/W-3RDctLj2I/AAAAAAAAVNw/5aSfLDUuSSkVA5qGymlKM9WTr_2fGN6NQCLcBGAs/s1600/GNOME%2BSettings%2BCode%2BBreakdown.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="651" data-original-width="1053" height="394" src="https://3.bp.blogspot.com/-560aXn8MEyc/W-3RDctLj2I/AAAAAAAAVNw/5aSfLDUuSSkVA5qGymlKM9WTr_2fGN6NQCLcBGAs/s640/GNOME%2BSettings%2BCode%2BBreakdown.png" width="640" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
This is just the user accounts panel, which shows some of the reduction in C code and increase in GtkBuilder data:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-596YwCpY654/W-3RP0RIcYI/AAAAAAAAVN0/xGV3dmQ8VAEhl9K7wCGLXbIS0cNUf0ODgCLcBGAs/s1600/User%2BAccounts%2BPanel%2BCode%2BBreakdown.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="656" data-original-width="1062" height="394" src="https://1.bp.blogspot.com/-596YwCpY654/W-3RP0RIcYI/AAAAAAAAVN0/xGV3dmQ8VAEhl9K7wCGLXbIS0cNUf0ODgCLcBGAs/s640/User%2BAccounts%2BPanel%2BCode%2BBreakdown.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Here's the breakdown of which panels make up the codebase:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-OVD-5sLra7U/W-3RYOH3_BI/AAAAAAAAVN8/sSmyJ8ZC_S0Ve0vJ5-z49bys6zJLPNMhwCLcBGAs/s1600/GNOME%2BSettings%2BPanel%2BBreakdown.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="677" data-original-width="1095" height="394" src="https://1.bp.blogspot.com/-OVD-5sLra7U/W-3RYOH3_BI/AAAAAAAAVN8/sSmyJ8ZC_S0Ve0vJ5-z49bys6zJLPNMhwCLcBGAs/s640/GNOME%2BSettings%2BPanel%2BBreakdown.png" width="640" /></a></div>
<br />
<br />
I don't think this draws any major conclusions, but is still interesting to see. Of note:<br />
<ul style="text-align: left;">
<li>Some of the changes make in 3.28 did reduce the total amount of code! But it was quickly gobbled up by the new Thunderbolt panel. </li>
<li>Network and Printers are the dominant panels - look at all that code! </li>
<li>I ignored empty lines in the files in case differing coding styles would make some panels look bigger or smaller. It didn't seem to make a significant difference.</li>
<li>You can see a reduction in C code looking at individual panels that have been updated, but overall it gets lost in the total amount of code.</li>
</ul>
I'll have another look in a few cycles when more changes have landed (I'm working on a <a href="https://gitlab.gnome.org/GNOME/gnome-control-center/merge_requests/233">new sound panel</a> at the moment).</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-18826528860057448112018-07-16T14:38:00.004+12:002018-07-16T14:41:19.811+12:00GUADEC 2018 AlmerÃa<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" id="docs-internal-guid-12f860b7-a0e8-3cd7-1d8b-e790bd5d5dc0" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: left;">
<div style="text-align: left;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">I recently attended the recent <a href="https://2018.guadec.org/">GNOME Users and Developers European Conference (GUADEC)</a> in AlmerÃa, Spain. This was my fifth GUADEC and as always I was able to attend thanks to my employer Canonical paying for me to be there. This year we had seven members of the Ubuntu desktop team present. AlmerÃa was a beautiful location for the conference and a good trade for the winter weather I left on the opposite side of the world in New Zealand.</span></div>
</div>
<div dir="ltr" id="docs-internal-guid-12f860b7-a0e8-3cd7-1d8b-e790bd5d5dc0" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-0kz7NJ20s0M/W0wCcsrxUQI/AAAAAAAATwM/vMOLYJQjM1o9O3t7yewJBkvbXYAqNRqQQCLcBGAs/s1600/IMG_20180707_210632.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://1.bp.blogspot.com/-0kz7NJ20s0M/W0wCcsrxUQI/AAAAAAAATwM/vMOLYJQjM1o9O3t7yewJBkvbXYAqNRqQQCLcBGAs/s320/IMG_20180707_210632.jpg" width="320" /></a></div>
<div dir="ltr" id="docs-internal-guid-12f860b7-a0e8-3cd7-1d8b-e790bd5d5dc0" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">This was the second GUADEC since the Ubuntu desktop switched back to shipping GNOME and it’s been great to be back. I was really impressed how positive and co-operative everyone was; the community seems to be in a really healthy shape. The icing on the cake is the <a href="https://www.gnome.org/news/2018/05/anonymous-donor-pledges-1m-donation-over-two-years/">anonymous million dollar donation</a> the foundation has received which they announced will be used to <a href="https://www.gnome.org/news/2018/07/gnome-foundation-opens-recruitment-for-further-expansion/">hire some staff</a></span><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">.</span></div>
<br />
<div style="text-align: center;">
<a href="https://2.bp.blogspot.com/-5rq_0332FjA/W0wE694gmRI/AAAAAAAATws/_2Iv5mnzHnYscFBrx075ILQzDY3lkIsvACLcBGAs/s1600/DhatoVPUEAIimLC.jpg%253Alarge.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="720" data-original-width="1280" height="180" src="https://2.bp.blogspot.com/-5rq_0332FjA/W0wE694gmRI/AAAAAAAATws/_2Iv5mnzHnYscFBrx075ILQzDY3lkIsvACLcBGAs/s320/DhatoVPUEAIimLC.jpg%253Alarge.jpeg" width="320" /></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">The first talk of the week was from my teammates <a href="https://twitter.com/kenvandine">Ken VanDine</a>, <a href="https://didrocks.fr/">Didier Roche</a> and <a href="https://blog.3v1n0.net/">Marco Treviño</a> who talked about how we’d done the transition from Unity to GNOME in Ubuntu desktop. I was successful in getting an open talk slot and did a short talk about the state of Snap integration into GNOME. I talked about the work I’d done making <a href="https://github.com/snapcore/snapd-glib/">snapd-glib</a> and the Snap plugin in GNOME Software. I also touched on some of the work <a href="https://blogs.gnome.org/jamesh/">James Henstridge</a> </span><span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">has been working on making Snaps work with portals. It was quite fun to see James be a bit of a celebrity after a long period of not being at a GUADEC - he is the JH in JHBuild!</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-9NU5Spoz3-w/W0wEAzr_-GI/AAAAAAAATwc/lVxgiyEzAZgUZfsoOUuV2WIxBjnKq20JgCLcBGAs/s1600/IMG_20180707_215640.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://3.bp.blogspot.com/-9NU5Spoz3-w/W0wEAzr_-GI/AAAAAAAATwc/lVxgiyEzAZgUZfsoOUuV2WIxBjnKq20JgCLcBGAs/s320/IMG_20180707_215640.jpg" width="320" /></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">After the first three days of talks the remaining three days are set for <i>Birds of a Feather</i> sessions where we get together in groups around a particular topic and discuss and hack on that. I organised a session on settings which turned out to be surprisingly popular! It was great to see everyone that I work with online in-person and allowed us to better understand each other. In particular I caught up with <a href="https://feaneron.com/">Georges Stavracas</a> who has been very patient in reviewing the many patches I have been working on in GNOME Control Center.</span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-9RlCgPa0ERQ/W0wD1V0w2qI/AAAAAAAATwY/hdkmNaBJU38t-me0vNc_kLJkhe6QYKeVQCLcBGAs/s1600/IMG_20180711_123720.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1200" data-original-width="1600" height="240" src="https://2.bp.blogspot.com/-9RlCgPa0ERQ/W0wD1V0w2qI/AAAAAAAATwY/hdkmNaBJU38t-me0vNc_kLJkhe6QYKeVQCLcBGAs/s320/IMG_20180711_123720.jpg" width="320" /></a></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;"><br /></span></div>
<div dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: "arial"; font-size: 11pt; font-style: normal; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline;">I hope to see everyone again next year! </span></div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-17771288005641394302017-12-08T13:39:00.000+13:002017-12-08T13:40:27.720+13:00Setting up Continuous Integration on gitlab.gnome.org<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="https://gitlab.gnome.org/GNOME/simple-scan">Simple Scan</a> recently migrated to the new <a href="https://gitlab.gnome.org/">gitlab.gnome.org</a> infrastructure. With modern infrastructure I now have the opportunity to enable Continuous Integration (CI), which is a fancy name for automatically building and testing your software when you make changes (and it can do more than that too).<br />
<br />
I've used CI in many projects in the past, and it's a really handy tool. However, I've never had to set it up myself and when I've looked it's been non-trivial to do so. The great news is this is really easy to do in GitLab!<br />
<br />
There's lots of <a href="https://docs.gitlab.com/ee/ci/">good documentation</a> on how to set it up, but to save you some time I'll show how I set it up for Simple Scan, which is a fairly typical GNOME application.<br />
<br />
To configure CI you need to create a file called <a href="https://gitlab.gnome.org/GNOME/simple-scan/blob/master/.gitlab-ci.yml"><span style="font-family: "courier new" , "courier" , monospace;">.gitlab-ci.yml</span></a> in your git repository. I started with the following:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">build-ubuntu:<br /> image: ubuntu:rolling<br /> before_script:<br /> - apt-get update<br /> - apt-get install -q -y --no-install-recommends meson valac gcc gettext itstool libgtk-3-dev libgusb-dev libcolord-dev libpackagekit-glib2-dev libwebp-dev libsane-dev<br /> script:<br /> - meson _build<br /> - ninja -C _build install</span><br />
<br />
The first line is the name of the job - "build_ubuntu". This is going to define how we build Simple Scan on Ubuntu.<br />
<br />
The "image" is the name of a Docker image to build with. You can see all the available images on <a href="https://hub.docker.com/explore/">Docker Hub</a>. In my case I chose an official Ubuntu image and used the "rolling" link which uses the most recently released Ubuntu version.<br />
<br />
The "before_script" defines how to set up the system before building. Here I just install the packages I need to build simple-scan.<br />
<br />
Finally the "script" is what is run to build Simple Scan. This is just what you'd do from the command line.<br />
<br />
And with that, every time a change is made to the git repository Simple Scan is built on Ubuntu and tells me if that succeeded or not! To make things more visible I added the following to the top of the README.md:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">[![Build Status](https://gitlab.gnome.org/GNOME/simple-scan/badges/master/build.svg)](https://gitlab.gnome.org/GNOME/simple-scan/pipelines)</span><br />
<br />
This gives the following image that shows the status of the build:</div>
<br />
<a href="https://gitlab.gnome.org/GNOME/simple-scan/commits/master"><img alt="pipeline status" src="https://gitlab.gnome.org/GNOME/simple-scan/badges/master/pipeline.svg" /></a></div>
<br />
And because there's many more consumers of Simple Scan that just Ubuntu, I added the following to<span style="font-family: "courier new" , "courier" , monospace;">.gitlab-ci.yml</span>:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">build-fedora:<br /> image: fedora:latest<br /> before_script:<br /> - dnf install -y meson vala gettext itstool gtk3-devel libgusb-devel colord-devel PackageKit-glib-devel libwebp-devel sane-backends-devel<br /> script:<br /> - meson _build<br /> - ninja -C _build install</span><br />
<br />
Now it builds on both Ubuntu and Fedora with every commit!<br />
<br />
I hope this helps you getting started with CI and gitlab.gnome.org. Happy hacking.</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-19668397071265525662017-11-01T17:01:00.001+13:002017-11-01T17:01:36.762+13:00Retiring my Ubuntu Phone after 1000 days<div dir="ltr" style="text-align: left;" trbidi="on">
With some sadness I recently replaced my Ubuntu Phone with a Nexus 5. It lasted me just over 1000 days (almost three years) as my everyday phone, and I last wrote about it <a href="http://bobthegnome.blogspot.co.nz/2016/04/five-hundred-days-using-ubuntu-phone.html">at the 500 mark</a>.<br />
<br />
Even though this is the end for me and Ubuntu Phone the hope of a true open source phone platform continues on:<br />
<ul style="text-align: left;">
<li>The Ubuntu Phone project lives on <a href="https://ubports.com/">ubports</a>.</li>
<li>As I put my Ubuntu phone to rest the <a href="https://puri.sm/shop/librem-5/">Purism Librem 5</a> project was funded with over $2 million!</li>
</ul>
I wish both these projects all the best.<br />
<br />
My thoughts on my time with Ubuntu Phone:<br />
<ul style="text-align: left;">
<li>It worked!</li>
<li>While the hardware (Meizu MX4) was reasonable hardware, it would have been nice to see it on something newer/faster and have gone through some more iterations on software performance.</li>
<li>The apps I missed most were:</li>
<ul>
<li>An app for my bank and network provider (that I could use to quickly check balances).</li>
<li>Communication apps (e.g. Facebook messenger, WhatsApp)</li>
<li>Uber </li>
</ul>
<li>I used a reasonable amount of webapps, which mostly filled the gap where apps weren't available. I does appear that most companies put more effort into their mobile apps than mobile web.</li>
</ul>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-86868166060626395252017-07-07T15:41:00.000+12:002017-07-07T15:41:35.855+12:00Snappy Sprint - London June 2017<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
I recently attended a <a href="https://snapcraft.io/">Snappy</a> Sprint in London, UK. As well as the Canonical people attending (including me) with experience in the whole Snappy stack (Snapcraft, the Snap store, snapd, snapd-glib) we had great representation from the Elementary, Fedora, GNOME, MATE and KDE communities. My goal was to help improve the Snap experience for desktop apps both on Ubuntu and other distributions.<br />
<br />
We spent a lot of time working on improving snap metadata for use with desktop apps. Improvements included:<br />
<ul style="text-align: left;">
<li><a href="https://github.com/snapcore/snapd/commit/e77a3876d1570e68030c6f9b53a06cb5e555ea6c">Exposing the title field</a> from the store down to clients.</li>
<li><a href="https://forum.snapcraft.io/t/snap-license-metadata/856">A plan</a> to get standard license information (using <a href="https://spdx.org/license-list">SPDX</a>) attached to snaps.</li>
<li>We made progress on a solution for projects that use <a href="https://www.freedesktop.org/wiki/Distributions/AppStream/">AppStream</a> to be able to easily build snaps and provide some AppStream data that doesn't fit the Snap metadata model to pass through to clients.</li>
<li>Fixing of many small issues in GNOME Software so it is suitable to work in Fedora and other distributions.</li>
<li>Plans for a tool that allows <a href="https://forum.snapcraft.io/t/connecting-plugs-and-slots-in-a-gui/776">graphical configuration of snap interfaces</a>.</li>
<li>A plan to solve the limitation on desktop clients able to install / remove snaps <a href="https://forum.snapcraft.io/t/local-logins-without-store-auth/857">without a store login</a>.</li>
<li>Discussions around <a href="https://forum.snapcraft.io/t/snap-license-metadata/856">metadata translations</a>.</li>
</ul>
I helped the <a href="https://ubuntu-mate.community/t/sneak-peek-new-software-boutique-17-10/13891">MATE Software Boutique</a> and <a href="http://www.omgubuntu.co.uk/2017/07/snap-apps-kde-discover-improvements">KDE Discover</a> developers make use of <a href="https://github.com/snapcore/snapd-glib">snapd-glib</a> using GIR bindings in Python and the Qt bindings to make their stores work. It was great to see snapd-glib working in these different use cases and got back some great feedback and a few patches.<br />
<br />
Thanks to all the community for attending, I found it very productive to work in-person with them all. If you're interested in following Snappy development check out the <a href="https://forum.snapcraft.io/">Snapcraft Forum</a> where you'll find discussions about what I've described above and much more.<br />
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-33633838688987454052017-07-01T15:58:00.000+12:002017-07-02T13:38:54.480+12:00Back to GNOME<div dir="ltr" style="text-align: left;" trbidi="on">
With <a href="https://insights.ubuntu.com/2017/04/05/growing-ubuntu-for-cloud-and-iot-rather-than-phone-and-convergence/">recent changes in Ubuntu</a> I found myself suddenly swung back into the GNOME orbit. It’s been a long journey and my GNOME contributions had reduced over time but it’s good to be back! Iain has written an <a href="https://medium.com/@iain.lane/welcome-16c91c0a71fb">excellent post</a> about the challenges we face trying to balance the best possible experience for Ubuntu users while also having a mutually beneficial relationship with our upstreams.<br />
<br />
After missing the last five GUADECs, I will be at <a href="https://2017.guadec.org/">Manchester</a> this year. I hope to catch up with as many people as possible including many old friends I haven’t seen in person for quite some time. If you have any questions about Ubuntu please find me or others from our team, we’re excited to collaborate.<br />
<br />
I’ll be spending a lot of my time this development cycle working on <a href="https://wiki.gnome.org/Apps/Software">GNOME Software</a> particularly around snap support. We’re already delivered some good changes to upstream GNOME like reviews and paid application support. GNOME Software has changed from being a stop-gap solution in Ubuntu to being our permanent software management solution (and has been working really well for us).<br />
<br />
It is with some sadness that I say goodbye to the <a href="https://en.wikipedia.org/wiki/Unity_(user_interface)">Unity desktop</a>. In particular some things I will miss:<br />
<ul style="text-align: left;">
<li>The performance and stability of Unity. After some early teething troubles Unity was rock solid and reliable.</li>
<li>Fullscreen window management. Unity was super efficient at making use of screen space and reducing distraction. I hope we can get a similar solution into GNOME Shell .</li>
<li>Convervenge. While we weren’t able to make a commercial success of it in Ubuntu I hope it will return in the future when the time is right.</li>
<li>The development experience of Ubuntu phone and clicks. I hope we can get that experience (and better) soon as next generation packaging systems start to take over. Luckily the click packages I worked on for the phone are being taken over by others in the community (as is the whole Unity 8 project). So I wish these projects success in the future.</li>
<li>Unfortunately we have decided it’s <a href="https://lists.ubuntu.com/archives/ubuntu-desktop/2017-June/004969.html">not possible to continue</a> to use <a href="https://www.freedesktop.org/wiki/Software/LightDM/">LightDM</a> in the default Ubuntu install. This project has had wide support amongst many distributions and has a number of features that I will miss.</li>
</ul>
<br /></div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-30952643093905417282016-08-25T22:41:00.000+12:002016-09-09T12:02:42.326+12:00Introducing snapd-glib<div dir="ltr" style="text-align: left;" trbidi="on">
World, meet <a href="http://launchpad.net/snapd-glib">snapd-glib</a>. It's a new library that makes it easy for GLib based projects (e.g. software centres) to access the daemon that allows you to query, install and remove <a href="http://snapcraft.io/">Snaps</a>. If C is not for you, you can use all the functionality in Python (using GObject Introspection) and Vala. In the future <a href="https://bugs.launchpad.net/bugs/1614797">it will support Qt/QML</a> through a wrapper library.<br />
<br />
snapd uses a <a href="https://github.com/snapcore/snapd/blob/master/docs/rest.md">REST API</a> and snapd-glib very closely matches that. The behaviour is best understood by reading the documentation of that API. To give you a taste of how it works, here's an example that shows how to find and install the <a href="https://uappexplorer.com/app/vlc.videolan">VLC</a> snap.<br />
<br />
<b>Step 1: Connecting to snapd</b><span style="font-family: inherit;"></span><br />
<br />
<span style="font-family: inherit;">The connection to snapd is controlled through the SnapdClient object. This </span>object has all the methods required to communicate with snapd. Create and connect with:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> g_autoptr(SnapdClient) c = <span style="color: blue;">snapd_client_new</span> ();<br /> if (!<span style="color: blue;">snapd_client_connect_sync</span> (c, NULL, &error))<br /> // Something went wrong</span><br />
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b>Step 2: Find the VLC snap</b><span style="font-family: inherit;"> </span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: inherit;">Asking snapd to perform a find causes it to contact the remote Snap store. This </span>can take some time so consider using an asynchronous call for this. This is the synchronous version:</div>
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> g_autoptr(GPtrArray) snaps =</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: blue;">snapd_client_find_sync</span> (c,</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> SNAPD_FIND_FLAGS_NONE, "vlc",</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> NULL, NULL, &error);<br /> if (snaps == NULL)<br /> // Something went wrong<br /> for (int i = 0; i < snaps->len; i++) {<br /> SnapdSnap *snap = snaps->pdata[i];<br /> // Do something with this snap information<br /> }</span><br />
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b>Step 3: Authenticate</b> </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Some methods require authorisation in the form of a <a href="http://research.google.com/pubs/pub41892.html">Macaroon</a> (the link is quite complex but in practise it's just a couple of strings). To get a Macaroon you need to provide credentials to snapd. In Ubuntu this is your <a href="https://login.ubuntu.com/">Ubuntu account</a>, but different snapd installations may use another authentication provider.</div>
<br />
Convert credentials to authorization with: <br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> g_autoptr(SnapdAuthData) auth_data =</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: blue;">snapd_login_sync</span> (email, password, code,</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="font-family: "courier new" , "courier" , monospace;"></span>NULL, &error);<br /> if (auth_data == NULL)<br /> return EXIT_FAILURE;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: blue;">snapd_client_set_auth_data</span> (<span style="font-family: "courier new" , "courier" , monospace;">c, auth_<span style="font-family: "courier new" , "courier" , monospace;">data)</span></span> </span><br />
<br />
Once you have a Macaroon you can store it somewhere and re-use it next time you need it. Then the authorization can be created with:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> g_autoptr(SnapdAuthData) auth_data =</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: blue;">snapd_auth_data_new</span> (macaroon, discharges);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: blue;">snapd_client_set_auth_data</span> (<span style="font-family: "courier new" , "courier" , monospace;">c, auth_<span style="font-family: "courier new" , "courier" , monospace;">data);</span></span> </span><br />
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b>Step 4: Install VLC</b> </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
In step 2 we could determine the VLC snap has the name "vlc". Since this involves <span style="font-family: inherit;">downloading ~100Mb and is going to take some time the asynchronous method is used. There is a callback that gives updates on the progress of the install and one that is called when the operation completes:</span></div>
<span style="font-family: inherit;"> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: blue;">snapd_client_install_async</span> (c,<br /> "vlc", NULL,<br /> progress_cb, NULL,<br /> NULL,<br /> install_cb, NULL);</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">static void<br />progress_cb (SnapdClient *client,</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> SnapdTask *main_task, GPtrArray *tasks,</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> gpointer user_data)<br />{</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> // Tell the user what's happening </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}<br /><br />static void<br />install_cb (GObject *object, GAsyncResult *result,</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> gpointer user_data)<br />{<br /> g_autoptr(GError) error = NULL;<br /><br /> if (<span style="color: blue;">snapd_client_install_finish</span> (SNAPD_CLIENT (object),</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> result, &error))<br /> // It installed!<br /> else<br /> // Something went wrong...<br />}</span><br />
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: left;">
<b><span style="font-family: inherit;">Conclusion</span></b><span style="font-family: inherit;"> </span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: inherit;">With snapd-glib and the above code as a starting point you should be able to start integrating Snap support into your project. Have fun!</span></div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-67946424815155004172016-05-11T23:22:00.000+12:002016-05-11T23:22:29.667+12:00Developing for Ubuntu Phone without the SDK<div dir="ltr" style="text-align: left;" trbidi="on">
I'm not a fan of IDEs. So when developing apps for Ubuntu Phone, I went looking for a way of doing everything from the command line. I'll describe the method I'm currently using to do this. It's pretty rough and there are probably better ways of doing this, but it does work!<br />
<br />
For an example, I've make a <a href="https://code.launchpad.net/~robert-ancell/+junk/hello-example">small application</a> that:<br />
<ul style="text-align: left;">
<li>Has some C++ code</li>
<li>Has some QML code</li>
<li>Uses gettext for translations</li>
</ul>
I need to do the following things:<br />
<ul style="text-align: left;">
<li>Extract translatable strings from source files and merge in translations </li>
<li>Cross-compile for the target device (ARM CPU) from my laptop (AMD64 CPU)</li>
<li>Create a click package</li>
<li>Deploy the click package to my phone for testing</li>
</ul>
I'm using <a href="https://www.gnu.org/software/make/">make</a> for the build system. It's pretty basic, but it's fairly easy to understand what it does.<br />
<h3 style="text-align: left;">
The project</h3>
<div style="text-align: left;">
My example project has the following files:</div>
<div style="text-align: left;">
</div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">Makefile</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">hello.apparmor</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">hello.cpp</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">hello.desktop.in</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">hello.h</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">hello.png</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">main.qml</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">manifest.json</span></div>
<span style="font-family: "Courier New",Courier,monospace;">po/fr.po</span><div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">po/de.po<br />po/hello.robert-ancell.pot<br />share/locale/de/LC_MESSAGES/</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">share/locale/fr/LC_MESSAGES/</span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
If you've done some Ubuntu phone apps hopefully these should be familiar.</div>
<h3 style="text-align: left;">
Translations </h3>
<div style="text-align: left;">
To make my app translatable to other languages (French and German in this example) I've used gettext:</div>
<ul style="text-align: left;">
<li>The .desktop file needs translations added to it. This is done by using hello.dekstop.in and prefixing the fields that need translating with '_' (e.g. _Name). These are then combined with the translations to make hello.desktop.</li>
<li>In the .qml files translatable strings are marked with <a href="https://developer.ubuntu.com/api/apps/qml/sdk-15.04/Ubuntu.Components.i18n/">i18n.tr</a> ("text to translate").</li>
<li>The compiled message catalogues (.mo files) need to be in the click package as share/locale/(language)/(appid).mo.</li>
</ul>
<div style="text-align: left;">
Gettext / intltool are somewhat scary to use, but here's the magic rules that work for me:</div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">hello.desktop: hello.desktop.in po/*.po<br /> intltool-merge --desktop-style po $< $@<br /><br />po/hello.robert-ancell.pot: main.qml hello.desktop.in<br /> xgettext --from-code=UTF-8 --language=JavaScript --keyword=tr --keyword=tr:1,2 --add-comments=TRANSLATORS main.qml -o po/hello.robert-ancell.pot<br /> intltool-extract --type=gettext/keys hello.desktop.in<br /> xgettext --keyword=N_ hello.desktop.in.h -j -o po/hello.robert-ancell.pot<br /> rm -f hello.desktop.in.h<br /><br />share/locale/%/LC_MESSAGES/hello.robert-ancell.mo: po/%.po<br /> msgfmt -o $@ $<</span></div>
<h3 style="text-align: left;">
Cross-compiling for the target device</h3>
<div style="text-align: left;">
To compile our package I need to make a <i>chroot</i>: </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">$ sudo click chroot -a armhf -f ubuntu-sdk-15.04 create</span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The following Makefile rule runs make inside this chroot, then packages the results into a click file:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">click:<br /> click chroot -a armhf -f ubuntu-sdk-15.04 run ARCH_PREFIX=arm-linux-gnueabihf- make<br /> click build --ignore=Makefile --ignore=*.cpp --ignore=*.h --ignore=*.pot --ignore=*.po --ignore=*.in --ignore=po .</span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Note the ARCH_PREFIX variable in the above. I've used this to run the correct cross-compiler when inside the chroot. When compiling from my laptop this variable is not set so it uses the local compiler.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">hello_moc.cpp: hello.h<br /> moc $< -o $@</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">hello: hello.cpp hello_moc.cpp<br /> $(ARCH_PREFIX)g++ -g -Wall -std=c++11 -fPIC $^ -o $@ `$(ARCH_PREFIX)pkg-config --cflags --libs Qt5Widgets Qt5Quick`</span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Running 'make click' in this project will spit out hello.robert-ancell_1_armhf.click.</div>
<h3 style="text-align: left;">
Testing</h3>
<div style="text-align: left;">
I connect my phone with a USB cable and copy the click package over and install it locally:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">$ adb push hello.robert-ancell_1_armhf.click /tmp/hello.robert-ancell_1_armhf.click</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">$ phablet-shell</span></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">$ pkcon install-local --allow-untrusted /tmp/hello.robert-ancell_1_armhf.click</span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Then on the phone I quit any instances of this app running, refresh the app scope (pull down at the top) and run my test version.</div>
<h3 style="text-align: left;">
Summary</h3>
<div style="text-align: left;">
Just one more rule to add to the Makefile, then it all works:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">all: hello \<br /> hello.desktop \<br /> po/hello.robert-ancell.pot \<br /> share/locale/de/LC_MESSAGES/hello.robert-ancell.mo \<br /> share/locale/fr/LC_MESSAGES/hello.robert-ancell.mo</span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The whole example is available in Launchpad:</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<span style="font-family: "Courier New",Courier,monospace;">$ bzr branch lp:~robert-ancell/+junk/hello-example</span></div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Happy hacking!</div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-37423461218585945102016-04-05T20:24:00.001+12:002016-04-05T20:24:57.919+12:00Five hundred days using Ubuntu Phone<div dir="ltr" style="text-align: left;" trbidi="on">
Today is my five hundredth day of using the <a href="http://www.meizu.com/en/products/mx4ubuntu/summary.html">Meizu MX4 Ubuntu Edition</a> exclusively as my mobile phone. This is a nice piece of hardware (good power, good camera and simple but elegant design).<br />
<br />
Here's what I've learnt.<br />
<br />
<a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.euchre">I</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.animal-farm">have</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.dotty">written</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.five-letters">a</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.pairs">bunch</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.dice-roller">of</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.morse-sender">phone</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.yatzy">apps</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.mines">you</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.ubuntu-digg-reader">can</a> <a href="https://uappexplorer.com/app/com.ubuntu.developer.robert-ancell.ubuntu-newshub">install</a> and <a href="http://bobthegnome.blogspot.co.nz/2014/11/writing-apps-for-ubuntu-phone.html">blogged</a> <a href="http://bobthegnome.blogspot.co.nz/2015/02/i-wrote-some-more-apps-for-ubuntu-phone.html">it</a>. Writing for the Ubuntu phone is by far the easiest platform I've developed for. Click packaging works really well and the speed at which you can release to the Ubuntu store and get the update on your phone is incredible. QML allows you to build beautiful apps quickly however can be a challenge when apps get more complicated. Qt / C++ is functional, but feels lacking compared to more modern languages. If I could get <a href="https://developer.apple.com/swift/">Swift</a> and an improved QML working together I'd be very happy. I initially used the <a href="https://developer.ubuntu.com/en/start/ubuntu-sdk/">Ubuntu SDK</a> for building and deplying the apps but have now switched to doing everything on the command line (I've never found an IDE that doesn't feel over-engineered).<br />
<br />
There's more than enough apps in the store to keep me happy. In fact, I have installed far more apps than I ever did on my Android phone. I think that is because I really trust the Ubuntu store in a way I never did in Android (too much crap there).<br />
<br />
I initially thought webapps wouldn't be useful but they're a good option when there's no native app. I use webapps for social networking and news sites and am pretty happy with that. They're definitely not as good as a native app but feel slightly more integrated than just visiting using your web browser.<br />
<br />
<a href="https://developer.ubuntu.com/en/scopes/">Scopes</a>. I can see there's something there in the concept but even though I've tried I've never found them useful. The only scope I have is the app scope (i.e. the traditional grid of applications). I'm hoping a few more iterations and they will find a place on my phone.<br />
<br />
Love getting updates. Both system and app updates occur frequently and bring improvements. Unless you had a <a href="https://www.google.com/nexus/">Nexus</a> device you are more or less abandoned in the Android world - with Ubuntu the complete opposite.<br />
<br />
The polish is not quite there compared to Android, but it's getting better quickly. There's little quirks / crashes that are annoying but nothing that stops me from using it all day. A couple more releases and the unforgiving mainstream will be able to thrash it too.<br />
<br />
Here's to another five hundred days!</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-87066449456520799772015-12-10T20:56:00.000+13:002016-05-11T21:34:32.200+12:00Accessing a webservice using libsoup and liboauth<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="https://en.wikipedia.org/wiki/OAuth">OAuth</a> 1.0 (<a href="https://tools.ietf.org/html/rfc5849">RFC 5849</a>)
is a common method used by web services for authorization. If you are
working on a GLib based project there are two libraries that can help
with this - <a href="https://wiki.gnome.org/Projects/libsoup">libosoup</a> and <a href="https://code.google.com/p/oauth/">liboauth</a>. The bad news for liboauth is it's not being developed anymore. The good news is OAuth 2.0 (<a href="https://tools.ietf.org/html/rfc6749">RFC 6749</a>) is replacing OAuth 1.0 so I guess that wont matter into the future.<br />
<br />
There are a number of ways OAuth can be used, in this example I'll show how to obtain <a href="https://login.ubuntu.com/">Ubuntu One</a> credentials and use these to post a review for an Ubuntu application. This example will use JSON so refer to my <a href="http://bobthegnome.blogspot.co.nz/2015/11/accessing-json-webservice-using-libsoup.html">previous tutorial</a> on how to do this with JSON-GLib.<br />
<br />
The first step is to get the credentials from Ubuntu One. After reading the <a href="http://canonical-identity-provider.readthedocs.org/en/latest/resources/token.html#oauth-token">API documentation</a>
this is fairly easy to acomplish. For my application, I need to
create a named token (I'll use "test-oauth"). To do this, I send my email address and password in JSON form to the server:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">// Create JSON request</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">JsonBuilder *builder = <a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-new">json_builder_new</a> ()<span style="font-family: "courier new" , "courier" , monospace;">;</span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;"></span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-begin-object">json_builder_begin_object</a> (builder);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-set-member-name">json_builder_set_member_name</a> (builder, "email");</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-add-string-value">json_builder_add_string_value</a> (builder, "test@example.com");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-set-member-name">json_builder_set_member_name</a> (builder, "password");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-add-string-value">json_builder_add_string_value</a> (builder, "secret");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-set-member-name">json_builder_set_member_name</a> (builder, "token_name");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-add-string-value">json_builder_add_string_value</a> (builder, "test-oauth");</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-end-object">json_builder_end_object</a> (builder);</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">// Convert request into a string</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">JsonGenerator *generator = <a href="https://developer.gnome.org/json-glib/stable/JsonGenerator.html#json-generator-new">json_generator_new</a> ();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonGenerator.html#json-generator-set-root">json_generator_set_root</a> (generator, <a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-get-root">json_builder_get_root</a> (builder));</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gsize length;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gchar *data = <a href="https://developer.gnome.org/json-glib/stable/JsonGenerator.html#json-generator-to-data">json_generator_to_data</a> (generator, &length);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">// Send request to the server</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">SoupSession *session = <a href="https://developer.gnome.org/libsoup/stable/SoupSession.html#soup-session-new-with-options">soup_session_new_with_options</a> (SOUP_SESSION_USER_AGENT, "test-oauth", NULL);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">SoupMessage *message = <a href="https://developer.gnome.org/libsoup/stable/SoupMessage.html#soup-message-new">soup_message_new</a> (SOUP_METHOD_POST, "https://login.ubuntu.com/api/v2/tokens/oauth");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/libsoup/stable/SoupMessageHeaders.html#soup-message-headers-append">soup_message_headers_append</a> (message-<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">></span></span>request_headers, "Accept", "application/json");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/libsoup/stable/SoupMessage.html#soup-message-set-request">soup_message_set_request</a> (message, "application/json", SOUP_MEMORY_TAKE, data, length);</span> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">guint status_code = <a href="https://developer.gnome.org/libsoup/stable/SoupSession.html#soup-session-send-message">soup_session_send_message</a> (session, message);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">g_assert (status_code == SOUP_STATUS_CREATED);</span><br />
<br />
If everything is correct, then the server will send back OAuth information for this token like this:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><br />{</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "token_key": "vt81TzXi2DdNkS9WRa27FZOt4SMhrn4XlPI4YgyplS6TbxoCqnFZtVAWEDfn", </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "token_secret": "r232LlyogYQIN72AinyjGg32yKcEDP0w73Vwau0EpOzjC3XhQD9ID1X99OtaZcZk2yNUrG",</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "consumer_key": "K6Jl5et",</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> "consumer_secret": "U23tFbCv9HCZJvvVZiBDBxZ2qcidOx"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
The <span style="font-family: "courier new" , "courier" , monospace;">consumer_key</span> is basically your user ID (named weirdly for historical reasons). <span style="font-family: "courier new" , "courier" , monospace;">token_key</span> is what you use to mark requests with the "test-oauth" token. <span style="font-family: "courier new" , "courier" , monospace;">consumer_secret</span> and <span style="font-family: "courier new" , "courier" , monospace;">token_secret</span> are encryption keys you use to prove that <span style="font-family: "courier new" , "courier" , monospace;">consumer_key</span> and <span style="font-family: "courier new" , "courier" , monospace;">token_key</span> are yours. These must be kept secret (the hint is in the name)!<br />
<br />
Now I have my OAuth token I can use it to review an Ubuntu package. I should also save this information so I can re-use it without returning to login.ubuntu.com.<br />
<br />
I'll make a message to send to the reviews server:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">// Create JSON request</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">JsonBuilder *builder = <a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-new">json_builder_new</a> ();</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-begin-object">json_builder_begin_object</a> (builder);</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-set-member-name">json_builder_set_member_name</a> (builder, "<span style="font-family: "courier new" , "courier" , monospace;">pa<span style="font-family: "courier new" , "courier" , monospace;">ckage_name</span></span>"<span style="font-family: "courier new" , "courier" , monospace;">);</span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-add-string-value">json_builder_add_string_value</a> (builder, "</span><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">no-such-package</span>");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-set-member-name">json_builder_set_member_name</a> (builder, "<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">rating</span></span>"<span style="font-family: "courier new" , "courier" , monospace;">);</span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-add-int-value">json_builder_add_<span style="font-family: "courier new" , "courier" , monospace;">int</span>_value</a> (builder, <span style="font-family: "courier new" , "courier" , monospace;">5</span></span><span style="font-family: "courier new" , "courier" , monospace;">); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">// Add other <span style="font-family: "courier new" , "courier" , monospace;">required fields...</span></span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-end-object">json_builder_end_object</a> (builder);</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">// Convert request into a string</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">JsonGenerator *generator = <a href="https://developer.gnome.org/json-glib/stable/JsonGenerator.html#json-generator-new">json_generator_new</a> ();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/json-glib/stable/JsonGenerator.html#json-generator-set-root">json_generator_set_root</a> (generator, <a href="https://developer.gnome.org/json-glib/stable/JsonBuilder.html#json-builder-get-root">json_builder_get_root</a> (builder));</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gsize length;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gchar *data = <a href="https://developer.gnome.org/json-glib/stable/JsonGenerator.html#json-generator-to-data">json_generator_to_data</a> (generator, &length);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">// <span style="font-family: "courier new" , "courier" , monospace;">Prepare</span> request <span style="font-family: "courier new" , "courier" , monospace;">for</span> the server</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">SoupSession *session = <a href="https://developer.gnome.org/libsoup/stable/SoupSession.html#soup-session-new-with-options">soup_session_new_with_options</a> (SOUP_SESSION_USER_AGENT, "test-oauth", NULL);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">SoupMessage *message = <a href="https://developer.gnome.org/libsoup/stable/SoupMessage.html#soup-message-new">soup_message_new</a> (SOUP_METHOD_POST, "</span><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">https://reviews.ubuntu.com/reviews/api/1.0/reviews/</span>");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/libsoup/stable/SoupMessageHeaders.html#soup-message-headers-append">soup_message_headers_append</a> (message-<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">></span></span></span>request_headers, "Accept", "application/json");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;"><a href="https://developer.gnome.org/libsoup/stable/SoupMessage.html#soup-message-set-request">soup_message_set_request</a> (message, "application/json", SOUP_MEMORY_TAKE, data, length);</span></span><br />
<br />
If I send the message like this, the server will reject it because there's no valid HTTP Authorization header. To populate this I'll use liboauth.<br />
<br />
The liboauth API is a bit confusing, so I'll run through the functions. Firstly the URL needs splitting into the parts that are needed for signing:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">char **url_parameters = NULL;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">int url_parameters_length = oauth_split_url_parameters (</span><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">"</span><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "courier new" , "courier" , monospace;">https://reviews.ubuntu.com/reviews/api/1.0/reviews/</span>"</span>, &url_parameters);</span><br />
<br />
Next step is to generate the Oauth fields and the signature. These new fields are written back into <span style="font-family: "courier new" , "courier" , monospace;">url_parameters</span>. Note the NULL argument - this is required if the HTTP request has content in the form <span style="font-family: "courier new" , "courier" , monospace;">application/x-www-form-urlencoded</span>. In my case it's <span style="font-family: "courier new" , "courier" , monospace;">application/json</span> so I don't have to bother.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">oauth_sign_array2_process (&url_parameters_length, &url_parameters,</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> NULL, // No postargs<br /> O_HMAC,<br /> "POST",<br /> oauth_consumer_key, oauth_consumer_secret,<br /> oauth_token, oauth_token_secret);</span><br />
<br />
Finally all the OAuth fields in <span style="font-family: "courier new" , "courier" , monospace;">url_parameters</span> need to be combined into a valid OAuth field. The 6 is a bitfield (no defines in the liboauth API) that specifies how to correctly choose and format the required fields.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">char *p = oauth_serialize_url_sep (url_parameters_length, 1, url_parameters, ", ", 6);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">gchar *authorization_text = g_strdup_printf ("OAuth realm=\"Ratings and Reviews\", %s", p);</span><br />
<br />
And now <span style="font-family: "courier new" , "courier" , monospace;">authorization_text</span> contains a valid Authorization HTTP header! I can now add this to the SoupMessage and send it to the server:<span style="font-family: "courier new" , "courier" , monospace;"></span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">soup_message_headers_append (message->request_headers, "Authorization", authorization_text);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">guint status_code = <a href="https://developer.gnome.org/libsoup/stable/SoupSession.html#soup-session-send-message">soup_session_send_message</a> (session, message);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">g_assert (status_code == SOUP_STATUS_<span style="font-family: "courier new" , "courier" , monospace;">OK</span>);</span><br />
<br />
And that's all it takes to send an OAuth authenticated request.<br />
<br />
The full program:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">// gcc -g -Wall example-oauth.c -o example-oauth `pkg-config --cflags --libs libsoup-2.4 oauth json-glib-1.0`<br /><br />#include <stdlib.h><br />#include <stdio.h><br />#include <string.h><br />#include <time.h><br />#include <unistd.h><br />#include <libsoup/soup.h><br />#include <oauth.h><br />#include <json-glib/json-glib.h><br /><br />static void<br />add_string_member (JsonBuilder *builder, const gchar *name, const gchar *value)<br />{<br /> json_builder_set_member_name (builder, name);<br /> json_builder_add_string_value (builder, value);<br />}<br /><br />static void<br />add_int_member (JsonBuilder *builder, const gchar *name, gint64 value)<br />{<br /> json_builder_set_member_name (builder, name);<br /> json_builder_add_int_value (builder, value);<br />}<br /><br />static void<br />set_request (SoupMessage *message, JsonBuilder *builder)<br />{<br /> JsonGenerator *generator = json_generator_new ();<br /> json_generator_set_root (generator, json_builder_get_root (builder));<br /> gsize length;<br /> gchar *data = json_generator_to_data (generator, &length);<br /> soup_message_set_request (message, "application/json", SOUP_MEMORY_TAKE, data, length);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> g_<span style="font-family: "courier new" , "courier" , monospace;">objec<span style="font-family: "courier new" , "courier" , monospace;">t_unref (genera<span style="font-family: "courier new" , "courier" , monospace;">tor);</span></span></span><br />}<br /><br />static const char *<br />getfield (const char *prompt)<br />{<br /> static char value[1024];<br /> printf ("%s", prompt);<br /> fgets (value, 1024, stdin);<br /> g_strchomp (value);<br /> return value;<br />}<br /><br />static void<br />sign_message (SoupMessage *message, const gchar *realm, OAuthMethod method,<br /> const gchar *oauth_consumer_key, const gchar *oauth_consumer_secret,<br /> const gchar *oauth_token, const gchar *oauth_token_secret)<br />{<br /> gchar *url, *oauth_authorization_parameters;<br /> gchar **url_parameters = NULL;<br /> int url_parameters_length;<br /> gchar *authorization_text;<br /><br /> url = soup_uri_to_string (soup_message_get_uri (message), FALSE);<br /><br /> url_parameters_length = oauth_split_url_parameters (url, &url_parameters);<br /> oauth_sign_array2_process (&url_parameters_length, &url_parameters,<br /> NULL, // No postargs<br /> method,<br /> message->method,<br /> oauth_consumer_key, oauth_consumer_secret,<br /> oauth_token, oauth_token_secret);<br /> oauth_authorization_parameters = oauth_serialize_url_sep (url_parameters_length, 1, url_parameters, ", ", 6);<br /> authorization_text = g_strdup_printf ("OAuth realm=\"%s\", %s", realm, oauth_authorization_parameters);<br /> soup_message_headers_append (message->request_headers, "Authorization", authorization_text);<br /><br /> g_free (url);<br /> oauth_free_array (&url_parameters_length, &url_parameters);<br /> free (oauth_authorization_parameters);<br />}<br /><br />int main (int argc, char **argv)<br />{<br /> const gchar *token_name = "test-oauth";<br /><br /> SoupSession *session = soup_session_new_with_options (SOUP_SESSION_USER_AGENT, "test-oauth", NULL);<br /><br /> /* Load the OAuth key, token and secrets */<br /> GKeyFile *config = g_key_file_new ();<br /> gchar *path = g_build_filename (g_get_user_config_dir (), "oauth-test.conf", NULL);<br /> g_key_file_load_from_file (config, path, G_KEY_FILE_NONE, NULL);<br /> gchar *oauth_consumer_key = g_key_file_get_string (config, token_name, "consumer-key", NULL);<br /> gchar *oauth_consumer_secret = g_key_file_get_string (config, token_name, "consumer-secret", NULL);<br /> gchar *oauth_token = g_key_file_get_string (config, token_name, "token", NULL);<br /> gchar *oauth_token_secret = g_key_file_get_string (config, token_name, "token-secret", NULL);<br /> if (!oauth_consumer_key || !oauth_consumer_secret || !oauth_token || !oauth_token_secret)<br /> {<br /> /* Request a token from the Canonical Identity Provider */<br /> SoupMessage *message = soup_message_new (SOUP_METHOD_POST, "https://login.ubuntu.com/api/v2/tokens/oauth");<br /> soup_message_headers_append (message->request_headers, "Accept", "application/json");<br /> JsonBuilder *builder = json_builder_new ();<br /> json_builder_begin_object (builder);<br /> add_string_member (builder, "email", getfield ("Email address: "));<br /> add_string_member (builder, "password", getpass ("Password: "));<br /> add_string_member (builder, "otp", getfield ("Verification code: "));<br /> add_string_member (builder, "token_name", token_name);<br /> json_builder_end_object (builder);<br /> set_request (message, builder);<br /> g_object_unref (builder);<br /> guint status_code = soup_session_send_message (session, message);<br /> g_assert (status_code == SOUP_STATUS_CREATED);<br /><br /> /* Parse response */<br /> JsonParser *parser = json_parser_new ();<br /> gboolean result = json_parser_load_from_data (parser, message->response_body->data, -1, NULL);<br /> g_assert (result);<br /> JsonNode *root = json_parser_get_root (parser);<br /> g_assert (JSON_NODE_HOLDS_OBJECT (root));<br /> JsonObject *object = json_node_get_object (root);<br /> oauth_consumer_key = g_strdup (json_object_get_string_member (object, "consumer_key"));<br /> oauth_consumer_secret = g_strdup (json_object_get_string_member (object, "consumer_secret")); <br /> oauth_token = g_strdup (json_object_get_string_member (object, "token_key"));<br /> oauth_token_secret = g_strdup (json_object_get_string_member (object, "token_secret"));<br /><br /> /* Write config so we don't do this the second time */<br /> g_key_file_set_string (config, token_name, "consumer-key", oauth_consumer_key);<br /> g_key_file_set_string (config, token_name, "consumer-secret", oauth_consumer_secret);<br /> g_key_file_set_string (config, token_name, "token", oauth_token);<br /> g_key_file_set_string (config, token_name, "token-secret", oauth_token_secret);<br /> g_key_file_save_to_file (config, path, NULL);<br /><br /> g_object_unref (message);<br /> g_object_unref (parser);<br /> }<br /> g_free (path);<br /><br /> /* Make the request using a HTTP POST signed with OAuth */<br /> SoupMessage *message = soup_message_new (SOUP_METHOD_POST, "https://reviews.ubuntu.com/reviews/api/1.0/reviews/");<br /> g_assert (message != NULL);<br /> soup_message_headers_append (message->request_headers, "Accept", "application/json");<br /> JsonBuilder *builder = json_builder_new ();<br /> json_builder_begin_object (builder);<br /> add_string_member (builder, "package_name", "no-such-package");<br /> add_string_member (builder, "summary", "★★★★★");<br /> add_string_member (builder, "review_text", "This is the best non existant application ever!");<br /> add_string_member (builder, "language", "en");<br /> add_string_member (builder, "origin", "ubuntu");<br /> add_string_member (builder, "distroseries", "xenial");<br /> add_string_member (builder, "version", "42");<br /> add_int_member (builder, "rating", 5);<br /> add_string_member (builder, "arch_tag", "amd64");<br /> json_builder_end_object (builder);<br /> set_request (message, builder);<br /> g_object_unref (builder);<br /> sign_message (message, "Ratings and Reviews", OA_HMAC, oauth_consumer_key, oauth_consumer_secret, oauth_token, oauth_token_secret);<br /> guint status_code = soup_session_send_message (session, message);<br /> g_printerr ("%d '%s'\n", status_code, message->response_body->data);<br /> g_assert (status_code == SOUP_STATUS_OK);<br /><br /> /* Parse the data in JSON format */<br /> JsonParser *parser = json_parser_new ();<br /> gboolean result = json_parser_load_from_data (parser, message->response_body->data, -1, NULL);<br /> g_assert (result);<br /> JsonNode *root = json_parser_get_root (parser);<br /> g_assert (JSON_NODE_HOLDS_OBJECT (root));<br /> JsonObject *object = json_node_get_object (root);<br /> gint64 review_id = json_object_get_int_member (object, "id");<br /> g_print ("Review created with ID %" G_GUINT64_FORMAT "\n", review_id);<br /><br /> /* Clean up */<br /> g_object_unref (session);<br /> g_object_unref (message);<br /> g_object_unref (parser);<br /> g_free (oauth_consumer_key);<br /> g_free (oauth_consumer_secret);<br /> g_free (oauth_token);<br /> g_free (oauth_token_secret);<br /> <br /> return 0;<br />}</span></div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-19791952697529436752015-11-30T20:55:00.000+13:002016-05-11T21:34:27.821+12:00Accessing a JSON webservice using libsoup and JSON-GLib<div dir="ltr" style="text-align: left;" trbidi="on">
A lot of web services use the <a href="http://json.org/">JSON</a> format. If you are working a GLib based project and need to access a service like this there are two great libraries to help you - <a href="https://wiki.gnome.org/Projects/libsoup">libsoup</a> and <a href="https://wiki.gnome.org/Projects/JsonGlib">JSON-Glib</a>.<br />
<br />
For my example, I'm going to grab some <a href="https://reviews.ubuntu.com/reviews/api/1.0/review-stats/any/any/">review data</a> from Ubuntu (<a href="https://wiki.ubuntu.com/AppStore/Interfaces/RatingsAndReviews#API">API</a>) which looks something like this:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">[<br /> {<br /> "ratings_total": 229, <br /> "ratings_average": "3.84", <br /> "app_name": "", <br /> "package_name": "simple-scan", <br /> "histogram": "[35, 13, 22, 42, 117]"<br /> }, <br /> {<br /> "ratings_total": 546, <br /> "ratings_average": "4.66", <br /> "app_name": "", <br /> "package_name": "audacity", <br /> "histogram": "[17, 7, 17, 63, 442]"<br /> },</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ... </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">]</span><br />
<br />
The data is a single array of objects that contain the statistics for each package. For this example I'll print out the number of ratings for each package by getting the <span style="font-family: "courier new" , "courier" , monospace;">package_name</span> and <span style="font-family: "courier new" , "courier" , monospace;">ratings_total</span> members from each object.<br />
<br />
Firstly, I need to download the data. The data is retrieved using a HTTP GET request; in libsoup you can do this with:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> SoupSession <span style="font-family: "courier new" , "courier" , monospace;">*</span>session = <a href="https://developer.gnome.org/libsoup/stable/SoupSession.html#soup-session-new-with-options">soup_session_new_with_options</a> (SOUP_SESSION_USER_AGENT, "test-json", NULL);<br /> SoupMessage *message = <a href="https://developer.gnome.org/libsoup/stable/SoupMessage.html#soup-message-new">soup_message_new</a> (SOUP_METHOD_GET, "<a href="https://reviews.ubuntu.com/reviews/api/1.0/review-stats/any/any">https://reviews.ubuntu.com/reviews/api/1.0/review-stats/any/any</a>");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <a href="https://developer.gnome.org/libsoup/stable/SoupSession.html#soup-session-send-message">soup_session_send_message</a> (session, message);</span><br />
<br />
Now I have the server text in <span style="font-family: "courier new" , "courier" , monospace;">message->response_body->data</span> but it needs to be decoded. JSON-GLib can parse it with:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> JsonParser *parser = <a href="https://developer.gnome.org/json-glib/stable/JsonParser.html#json-parser-new">json_parser_new</a> ();<br /> <a href="https://developer.gnome.org/json-glib/stable/JsonParser.html#json-parser-load-from-data">json_parser_load_from_data</a> (parser, message-<span style="font-family: "courier new" , "courier" , monospace;">></span>response_body-<span style="font-family: "courier new" , "courier" , monospace;">></span>data, -1, NULL);<br /> JsonNode *root = <a href="https://developer.gnome.org/json-glib/stable/JsonParser.html#json-parser-get-root">json_parser_get_root</a> (parser);</span><br />
<br />
Now I have an in-memory tree of the JSON data which can be traversed. After checking the root node is an array as expected I'll iterate over each object:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> g_assert (<a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Node.html#JSON-NODE-HOLDS-ARRAY:CAPS">JSON_NODE_HOLDS_ARRAY</a> (root));<br /> array = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Node.html#json-node-get-array">json_node_get_array</a> (root);<br /> for (i = 0; i <span style="font-family: "courier new" , "courier" , monospace;"><</span> <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Array.html#json-array-get-length">json_array_get_length</a> (array); i++)<br /> {<br /> JsonNode *node = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Array.html#json-array-get-element">json_array_get_element</a> (array, i);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="font-family: "courier new" , "courier" , monospace;">/* do stuff... *<span style="font-family: "courier new" , "courier" , monospace;">/</span></span> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<br />
For each object, I extract the required data:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;"> <span style="font-family: "courier new" , "courier" , monospace;">g_asser<span style="font-family: "courier new" , "courier" , monospace;">t</span> </span>(<a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Node.html#JSON-NODE-HOLDS-OBJECT:CAPS">JSON_NODE_HOLDS_OBJECT</a> (node));<br /> JsonObject <span style="font-family: "courier new" , "courier" , monospace;">*</span>object = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Node.html#json-node-get-object">json_node_get_object</a> (node);<br /> const gchar *package_name = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Object.html#json-object-get-string-member">json_object_get_string_member</a> (object, "package_name");<br /> gint64 ratings_total = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Object.html#json-object-get-int-member">json_object_get_int_member</a> (object, "ratings_total");<br /> if (package_name)<br /> g_print ("%s: %" G_GUINT64_FORMAT "\n", package_name, </span><br />
<br />
Combined into a program, I can print out the number of reviews for each package:<br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">simple-scan: 229<br />audacity: 546<br />...</span><br />
<br />
The full program:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">// gcc -g -Wall example-json.c -o example-json <span style="font-family: "courier new" , "courier" , monospace;">`</span>pkg-config --cflags --libs libsoup-2.4 json-glib-1.0`<br /><br />#include <json-glib json-glib.h=""><br />#include <libsoup soup.h=""><br /><br />int main (int argc, char **argv)<br />{<br /> SoupSession *session;<br /> SoupMessage *message;<br /> guint status_code;<br /> JsonParser *parser;<br /> gboolean result;<br /> JsonNode *root;<br /> JsonArray *array;<br /> gint i;<br /><br /> /* Get the data using a HTTP GET */<br /> session = <a href="https://developer.gnome.org/libsoup/stable/SoupSession.html#soup-session-new-with-options">soup_session_new_with_options</a> (SOUP_SESSION_USER_AGENT, "test-json", NULL);<br /> message = <a href="https://developer.gnome.org/libsoup/stable/SoupMessage.html#soup-message-new">soup_message_new</a> (SOUP_METHOD_GET, "<a href="https://reviews.ubuntu.com/reviews/api/1.0/review-stats/any/any">https://reviews.ubuntu.com/reviews/api/1.0/review-stats/any/any</a>");<br /> g_assert (message != NULL);<br /> status_code = <a href="https://developer.gnome.org/libsoup/stable/SoupSession.html#soup-session-send-message">soup_session_send_message</a> (session, message);<br /> g_assert (status_code == SOUP_STATUS_OK);<br /><br /> /* Parse the data in JSON format */<br /> parser = <a href="https://developer.gnome.org/json-glib/stable/JsonParser.html#json-parser-new">json_parser_new</a> ();<br /> result = <a href="https://developer.gnome.org/json-glib/stable/JsonParser.html#json-parser-load-from-data">json_parser_load_from_data</a> (parser, message-<span style="font-family: "courier new" , "courier" , monospace;">></span>response_body-<span style="font-family: "courier new" , "courier" , monospace;">></span>data, -1, NULL);<br /> g_assert (result);<br /><br /> /* The data should contain an array of JSON objects */<br /> root = <a href="https://developer.gnome.org/json-glib/stable/JsonParser.html#json-parser-get-root">json_parser_get_root</a> (parser);<br /> g_assert (<a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Node.html#JSON-NODE-HOLDS-ARRAY:CAPS">JSON_NODE_HOLDS_ARRAY</a> (root));<br /> array = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Node.html#json-node-get-array">json_node_get_array</a> (root);<br /> for (i = 0; i <span style="font-family: "courier new" , "courier" , monospace;"><</span> <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Array.html#json-array-get-length">json_array_get_length</a> (array); i++)<br /> {<br /> JsonNode *node;<br /> JsonObject *object;<br /> const gchar *package_name;<br /> gint64 ratings_total;<br /><br /> /* Get the nth object, skipping unexpected elements */<br /> node = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Array.html#json-array-get-element">json_array_get_element</a> (array, i);<br /> if (!<a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Node.html#JSON-NODE-HOLDS-OBJECT:CAPS">JSON_NODE_HOLDS_OBJECT</a> (node))<br /> continue;<br /><br /> /* Get the package name and number of ratings from the object - skip if has no name */<br /> object = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Node.html#json-node-get-object">json_node_get_object</a> (node);<br /> package_name = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Object.html#json-object-get-string-member">json_object_get_string_member</a> (object, "package_name");<br /> ratings_total = <a href="https://developer.gnome.org/json-glib/stable/json-glib-JSON-Object.html#json-object-get-int-member">json_object_get_int_member</a> (object, "ratings_total");<br /> if (package_name)<br /> g_print ("%s: %" G_GINT64_FORMAT "\n", package_name, ratings_total);<br /> }<br /><br /> /* Clean up */<br /> g_object_unref (session); <br /> g_object_unref (message);<br /> g_object_unref (parser);<br /><br /> return 0;<br />}</libsoup></json-glib></span><br />
<br />
And to show you can do the same thing with GIR bindings, here's the same in Vala:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">// valac example-json.vala --pkg soup-2.4 --pkg json-glib-1.0</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">public int main (string[] args)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">{</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> /* Get the data using a HTTP GET */</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var session = new Soup.Session.with_options (Soup.SESSION_USER_AGENT, "test-json");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var message = new Soup.Message ("GET", "https://reviews.ubuntu.com/reviews/api/1.0/review-stats/any/any");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> assert (message != null);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var status_code = session.send_message (message);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> assert (status_code == Soup.Status.OK);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> /* Parse the data in JSON format */</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var parser = new Json.Parser ();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> try</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> parser.load_from_data ((string) message.response_body.data);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> catch (Error e)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> /* The data should contain an array of JSON objects */</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var root = parser.get_root ();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> assert (root.get_node_type () == Json.NodeType.ARRAY);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var array = root.get_array ();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> for (var i = 0; i <array .get_length="" i="" span=""></array></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> /* Get the nth object, skipping unexpected elements */</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var node = array.get_element (i);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> if (node.get_node_type () != Json.NodeType.OBJECT)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> continue;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> /* Get the package name and number of ratings from the object - skip if has no name */</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var object = node.get_object ();</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var package_name = object.get_string_member ("package_name");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> var ratings_total = object.get_int_member ("ratings_total");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> if (package_name != null)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> stdout.printf ("%s: %" + int64.FORMAT + "\n", package_name, ratings_total);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> return 0;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">}</span><br />
<br />
and Python:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">#!/usr/bin/python</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">from gi.repository import Soup</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">from gi.repository import Json</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># Get the data using a HTTP GET</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">session = Soup.Session.new ()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">session.set_property (Soup.SESSION_USER_AGENT, "test-json")</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">message = Soup.Message.new ("GET", "https://reviews.ubuntu.com/reviews/api/1.0/review-stats/any/any")</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">assert (message != None)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">status_code = session.send_message (message)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">assert (status_code == Soup.Status.OK)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># Parse the data in JSON format</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">parser = Json.Parser ()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">parser.load_from_data (message.response_body.data, -1)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"># The data should contain an array of JSON objects</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">root = parser.get_root ()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">assert (root.get_node_type () == Json.NodeType.ARRAY)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">array = root.get_array ()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">for i in xrange (array.get_length ()):</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> # Get the nth object, skipping unexpected elements</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> node = array.get_element (i)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> if node.get_node_type () != Json.NodeType.OBJECT:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> continue</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> # Get the package name and number of ratings from the object - skip if has no name</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> object = node.get_object ()</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> package_name = object.get_string_member ("package_name")</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ratings_total = object.get_int_member ("ratings_total")</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> if package_name != None:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> print ("%s: %d" % (package_name, ratings_total))</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-91457639290585462662015-02-22T21:09:00.001+13:002015-02-22T21:09:36.339+13:00I wrote some more apps for Ubuntu Phone<div dir="ltr" style="text-align: left;" trbidi="on">
In a <a href="http://bobthegnome.blogspot.co.nz/2014/11/writing-apps-for-ubuntu-phone.html">previous post</a> I talked about the experiences of writing my first five applications for Ubuntu Phone. Since then, I've written three more.<br />
<br />
As before, all these apps are GPL 3 licensed and available on Launchpad. What's new is now you can browse them online due to a great <a href="https://appstore.bhdouglass.com/">unofficial web appstore</a> made by <a href="http://bhdouglass.com/">Brian Douglass</a>. This solves one of my previous gripes about not being able to find new applications.<br />
<br />
One thing I have changed is I've started to use the standard page header. This makes it easy to add more pages (e.g. help, high scores). I initially wasn't a fan of the header due to the amount of space it took up but for simple apps it works well and it makes them predictable to use. I also went back and updated <a href="https://appstore.bhdouglass.com/app/com.ubuntu.developer.robert-ancell.dotty">Dotty</a> to use this style.<br />
<br />
Let me introduce...<br />
<br />
<div style="text-align: center;">
<a href="https://appstore.bhdouglass.com/app/com.ubuntu.developer.robert-ancell.dice-roller">Dice Roller</a> (<a href="https://launchpad.net/dice-roller">lp:dice-roller</a>)</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-dhwe-N2nd6c/VOmGiwH3GxI/AAAAAAAALQE/0NzkBffoo68/s1600/screenshot2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-dhwe-N2nd6c/VOmGiwH3GxI/AAAAAAAALQE/0NzkBffoo68/s1600/screenshot2.png" height="320" width="192" /></a></div>
<br />
It's a simple utility to simulate dice rolling. Not much more to say. 431 lines of QML. <br />
<br />
<div style="text-align: center;">
<a href="https://appstore.bhdouglass.com/app/com.ubuntu.developer.robert-ancell.morse-sender">Morse Sender</a> (<a href="https://launchpad.net/morse-sender">lp:morse-sender</a>)</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-SbT6hbzmrbs/VOmGpWHBHcI/AAAAAAAALQM/JWvRMZ7v6-M/s1600/screenshot.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-SbT6hbzmrbs/VOmGpWHBHcI/AAAAAAAALQM/JWvRMZ7v6-M/s1600/screenshot.png" height="320" width="192" /></a></div>
<br />
I was playing with <a href="https://appstore.bhdouglass.com/app/com.ubuntu.developer.majster-pl.utorch">uTorch</a> and watching a spy movie and thinking "hey, wouldn't it be cool to flash the camera and send morse code messages. I wonder if the phone responds fast enough to do that." Apparently it does. 584 lines of QML.<br />
<br />
<div style="text-align: center;">
<a href="https://appstore.bhdouglass.com/app/com.ubuntu.developer.robert-ancell.yatzy">Yatzy</a> (<a href="https://launchpad.net/yatzy">lp:yatzy</a>)</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-hJQKO5u27VI/VOmGvjGINWI/AAAAAAAALQU/kpH7DbiviGI/s1600/screenshot1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-hJQKO5u27VI/VOmGvjGINWI/AAAAAAAALQU/kpH7DbiviGI/s1600/screenshot1.png" height="320" width="192" /></a></div>
<br />
So I thought, now I have this nice dice code from <i>Dice Roller</i>, it would be cool to make a full dice game with it. I've played a fair bit of Yahtzee in my time and searching Wikipedia I found there was a similar public domain game called Yatzy. 999 lines of QML.<br />
<br /></div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-78947708663487938472014-12-13T10:13:00.000+13:002016-05-11T21:34:52.380+12:00Tips for contributing code to open source projects<div dir="ltr" style="text-align: left;" trbidi="on">
I've spent a lot of time over the years contributing to and reviewing code changes to open source projects. It can take a lot of work for the submitter and reviewer to get a change accepted and often they don't make it. Here are the things in my experience that successful contributions do.<br />
<b><br /></b>
<b>Use the issue tracker</b>. Having an open issue means there is always something to point to with all the history of the change that wont get lost. Submit patches using the appropriate method (merge proposals, pull requests, attachments in the issue tracker etc).<br />
<br />
<b>Sell your idea</b>. The change is important to you but the maintainers may not think so. You may be a 1% use case that doesn't seem worth supporting. If the change fixes a bug describe exactly how to reproduce the issue and how serious it is. If the change is a new feature then show how it is useful.<br />
<br />
<b>Always follow the existing coding style</b>. Even if you don't like it. If the existing code uses tabs, then use them too. Match brace style. If the existing code is inconsistent, match the code nearest to the changes you are making.<br />
<br />
<b>Make your change as small as possible</b>. Put yourself in the mind of the reviewer. The longer the patch the more time it will take to review (and the less appealing it will be to do). You can always follow up later with more changes. First time contributors need more review - over time you can propose bigger changes and the reviewers can trust you more.<br />
<br />
<b>Read your patch before submitting it</b>. You will often find bits you should have removed (whitespace, unrelated variable name changes, debugging code).<br />
<br />
<b>Be patient</b>. It's OK to check back on progress - your change might have be forgotten about (everyone gets busy). Ask if there's any more you can do to make it easier to accept.<br />
<br /></div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-43796747653286358092014-11-27T10:07:00.001+13:002014-11-29T10:11:51.149+13:00Writing applications for Ubuntu Phone<div dir="ltr" style="text-align: left;" trbidi="on">
<div>
I've just released my fifth application for the Ubuntu phone and I thought I'd do a write up of my experiences developing for Ubuntu Phone. In summary, it's been pretty positive!</div>
<div>
<br /></div>
<div>
The good:</div>
<div>
<ul style="text-align: left;">
<li><a href="http://developer.ubuntu.com/start/ubuntu-sdk/installing-the-sdk/">Installing the SDK</a> is as easy as installing any application in Ubuntu.</li>
<li>Writing applications is fast. You can throw together something fairly nice in a few hours.</li>
<li><a href="http://developer.ubuntu.com/publish/apps/packaging-click-apps/">Click packages</a> are so easy to build! It makes .deb packages feel like something from the 1990s. Which is appropriate, because they are from the 1990s.</li>
<li>The deployment process is incredibly fast. You create a click package from the SDK, upload it to the store in a web form and it lands on my (or anyone else's) phone in under a minute normally. A freaking minute! That's amazing!</li>
</ul>
</div>
<div>
The bad / ugly:</div>
<ul>
<li>The Ubuntu SDK (aka Qt Creator) still reinforces why I don't like IDEs. While it's better than older IDEs it's still overly complicated and cluttered with buttons. I only use it to dogfood the process and the command line tools aren't great for building and deploying applications (yet).</li>
<li>QML is... OK. It has all the technology of a modern toolkit (e.g. transitions, it's declarative, you can develop using a dynamic language) which is good. But it feels like it was put together in a rush. It's often not clear what the best way is to solve a problem and some components seem to be missing useful functionality (e.g. containers).</li>
<li>Javascript is great for small applications but quickly becomes unwieldy for large ones. The default other option is to use C++ which is just an enormous step backwards into complexity. I haven't yet tried <a href="https://github.com/go-qml/qml">Go QML</a> but hopefully that will be a better combination.</li>
<li>The Ubuntu store interface is very basic. There's no way to list apps by ranking, you can't see new applications, there's no web interface. I'm sure it will get better soon but it's currently hard to find what's available (which is a big part of why I'm writing this blog post).</li>
</ul>
<div>
Here's what I've made; all these applications are released under the GPL 3 license and available on Launchpad. You can get the source for any of them by typing "bzr branch lp:euchre" from an Ubuntu machine.</div>
<div>
<br /></div>
<h4 style="clear: both; text-align: center;">
<a href="https://launchpad.net/euchre">Euchre</a></h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-kiWKpbLB8Fg/VHYzmD8wwAI/AAAAAAAAKqA/ZnMdr_1fFmY/s1600/screenshot-euchre.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-kiWKpbLB8Fg/VHYzmD8wwAI/AAAAAAAAKqA/ZnMdr_1fFmY/s1600/screenshot-euchre.png" height="320" width="192" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div>
My first Ubuntu phone application. It's a <a href="http://en.wikipedia.org/wiki/Euchre">classic four player trick taking card game</a> with a basic AI. I learnt a lot about animation in QML developing this. It's all written in Javascript which is really pushing the limits of maintainability for an application like this (1833 lines of QML). While it is the oldest it is also the least downloaded of my applications I think because Euchre is a bit of a niche game and I don't have any in game help.</div>
<div>
<br /></div>
<h4 style="text-align: center;">
<a href="https://launchpad.net/animal-farm">Animal Farm</a></h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-cnl3_QCJ_60/VHYzneBIBAI/AAAAAAAAKqI/7ybmF0KTMj4/s1600/screenshot-animal-farm.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-cnl3_QCJ_60/VHYzneBIBAI/AAAAAAAAKqI/7ybmF0KTMj4/s1600/screenshot-animal-farm.png" height="320" width="192" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
The inspiration for this was my daughter enjoying applications like this on Android. You touch the animals and they shake and meow / baa etc. It's trivially small (157 lines of QML).</div>
<div>
<br /></div>
<h4 style="text-align: center;">
<a href="https://launchpad.net/dotty">Dotty</a></h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-JQAmqUyQF28/VHYzoeO4CaI/AAAAAAAAKqM/t54UvO4uyeo/s1600/screenshot-dotty.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-JQAmqUyQF28/VHYzoeO4CaI/AAAAAAAAKqM/t54UvO4uyeo/s1600/screenshot-dotty.png" height="320" width="192" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Dotty is a clone of the very successful <a href="https://play.google.com/store/apps/details?id=com.nerdyoctopus.gamedots">iOS / Android game Dots</a>. I thought I'd see if copying a popular game would transfer into success in Ubuntu and it has. This is my most popular game with 362 users currently compared to 160 for Animal Farm which is the next most popular. I learnt how to do dynamic components (i.e. the lines and the dots falling down) with this. A good size at 605 lines of QML.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<h4 style="text-align: center;">
<a href="https://launchpad.net/five-letters">Five Letters</a></h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-FotN96rwCUc/VHYzpo1vPLI/AAAAAAAAKqU/6_b9vXLOcBk/s1600/screenshot-five-letters.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-FotN96rwCUc/VHYzpo1vPLI/AAAAAAAAKqU/6_b9vXLOcBk/s1600/screenshot-five-letters.png" height="320" width="192" /></a></div>
<div>
<br /></div>
<div>
Like Dotty I was looking for the type of games that are already popular on existing platforms. Word games are quite successful and I was thinking of games like <a href="https://play.google.com/store/apps/details?id=com.blueoxtech.sevenlittlewords">7 little words</a> when designing this. The "making words from five letters" is a common newspaper game. I spent a lot of time trimming the dictionary of possible games to remove anything offensive or obscure so it should be reasonably possible to solve all the puzzles (there's about 1300 of them). 406 lines of QML.</div>
<div>
<br /></div>
<h4 style="text-align: center;">
<a href="https://launchpad.net/pairs-game">Pairs</a></h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-W28Yq6kncvE/VHYzqDEl67I/AAAAAAAAKqc/nB8uqF5HPg0/s1600/screenshot-pairs.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-W28Yq6kncvE/VHYzqDEl67I/AAAAAAAAKqc/nB8uqF5HPg0/s1600/screenshot-pairs.png" height="320" width="192" /></a></div>
<div>
<br /></div>
<div>
My newest game! Released last night. Like Animal Farm I was thinking of something my children might like to play. You turn over the cards two at a time and try and find the matching colours. The colours I've used actually make it quite difficult and fun to play as an adult. 409 lines of QML.</div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-76217021559753994652014-09-22T21:58:00.000+12:002016-05-11T21:34:59.442+12:00Using EGL with GTK+<div dir="ltr" style="text-align: left;" trbidi="on">
I recently needed to port some code from GTK+ OpenGL code from GLX to EGL and I couldn't find any examples of how to do this. So to seed the Internet here is what I found out.<br />
<br />
This is the simplest example I could make to show how to do this. In real life you probably want to do a lot more error checking. This will only work with X11; for other systems you will need to use equivalent methods in gdk/gdkwayland.h etc. For anything modern you should probably use OpenGL ES instead of OpenGL - to do this you'll need to change the attributes to eglChooseConfig and use EGL_OPENGL_ES_API in eglBindAPI.<br />
<br />
Compile with:<br />
<span style="font-family: "courier new" , "courier" , monospace;">gcc -g -Wall egl.c -o egl `pkg-config --cflags --libs gtk+-3.0 gdk-x11-3.0` -lEGL -lGL</span><br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: .1em .1em .1em .8em; border: solid gray; overflow: auto; padding: .2em .6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #557799;">#include <gtk/gtk.h></span>
<span style="color: #557799;">#include <gdk/gdkx.h></span>
<span style="color: #557799;">#include <EGL/egl.h></span>
<span style="color: #557799;">#include <GL/gl.h></span>
<span style="color: #008800; font-weight: bold;">static</span> EGLDisplay <span style="color: #333333;">*</span>egl_display;
<span style="color: #008800; font-weight: bold;">static</span> EGLSurface <span style="color: #333333;">*</span>egl_surface;
<span style="color: #008800; font-weight: bold;">static</span> EGLContext <span style="color: #333333;">*</span>egl_context;
<span style="color: #008800; font-weight: bold;">static</span> <span style="color: #333399; font-weight: bold;">void</span> <span style="color: #0066bb; font-weight: bold;">realize_cb</span> (GtkWidget <span style="color: #333333;">*</span>widget)
{
EGLConfig egl_config;
EGLint n_config;
EGLint attributes[] <span style="color: #333333;">=</span> { EGL_RENDERABLE_TYPE, EGL_OPENGL_BIT,
EGL_NONE };
egl_display <span style="color: #333333;">=</span> eglGetDisplay ((EGLNativeDisplayType) gdk_x11_display_get_xdisplay (gtk_widget_get_display (widget)));
eglInitialize (egl_display, <span style="color: #007020;">NULL</span>, <span style="color: #007020;">NULL</span>);
eglChooseConfig (egl_display, attributes, <span style="color: #333333;">&</span>egl_config, <span style="color: #0000dd; font-weight: bold;">1</span>, <span style="color: #333333;">&</span>n_config);
eglBindAPI (EGL_OPENGL_API);
egl_surface <span style="color: #333333;">=</span> eglCreateWindowSurface (egl_display, egl_config, gdk_x11_window_get_xid (gtk_widget_get_window (widget)), <span style="color: #007020;">NULL</span>);
egl_context <span style="color: #333333;">=</span> eglCreateContext (egl_display, egl_config, EGL_NO_CONTEXT, <span style="color: #007020;">NULL</span>);
}
<span style="color: #008800; font-weight: bold;">static</span> gboolean <span style="color: #0066bb; font-weight: bold;">draw_cb</span> (GtkWidget <span style="color: #333333;">*</span>widget)
{
eglMakeCurrent (egl_display, egl_surface, egl_surface, egl_context);
glViewport (<span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">0</span>, gtk_widget_get_allocated_width (widget), gtk_widget_get_allocated_height (widget));
glClearColor (<span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">1</span>);
glClear (GL_COLOR_BUFFER_BIT <span style="color: #333333;">|</span> GL_DEPTH_BUFFER_BIT);
glMatrixMode (GL_PROJECTION);
glLoadIdentity ();
glOrtho (<span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">100</span>, <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">100</span>, <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">1</span>);
glBegin (GL_TRIANGLES);
glColor3f (<span style="color: #0000dd; font-weight: bold;">1</span>, <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">0</span>);
glVertex2f (<span style="color: #0000dd; font-weight: bold;">50</span>, <span style="color: #0000dd; font-weight: bold;">10</span>);
glColor3f (<span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">1</span>, <span style="color: #0000dd; font-weight: bold;">0</span>);
glVertex2f (<span style="color: #0000dd; font-weight: bold;">90</span>, <span style="color: #0000dd; font-weight: bold;">90</span>);
glColor3f (<span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">1</span>);
glVertex2f (<span style="color: #0000dd; font-weight: bold;">10</span>, <span style="color: #0000dd; font-weight: bold;">90</span>);
glEnd ();
eglSwapBuffers (egl_display, egl_surface);
<span style="color: #008800; font-weight: bold;">return</span> TRUE;
}
<span style="color: #333399; font-weight: bold;">int</span> <span style="color: #0066bb; font-weight: bold;">main</span> (<span style="color: #333399; font-weight: bold;">int</span> argc, <span style="color: #333399; font-weight: bold;">char</span> <span style="color: #333333;">**</span>argv)
{
GtkWidget <span style="color: #333333;">*</span>w;
gtk_init (<span style="color: #333333;">&</span>argc, <span style="color: #333333;">&</span>argv);
w <span style="color: #333333;">=</span> gtk_window_new (GTK_WINDOW_TOPLEVEL);
gtk_widget_set_double_buffered (GTK_WIDGET (w), FALSE);
g_signal_connect (G_OBJECT (w), <span style="background-color: #fff0f0;">"realize"</span>, G_CALLBACK (realize_cb), <span style="color: #007020;">NULL</span>);
g_signal_connect (G_OBJECT (w), <span style="background-color: #fff0f0;">"draw"</span>, G_CALLBACK (draw_cb), <span style="color: #007020;">NULL</span>);
gtk_widget_show (w);
gtk_main ();
<span style="color: #008800; font-weight: bold;">return</span> <span style="color: #0000dd; font-weight: bold;">0</span>;
}
</pre>
</div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-40990389499604472582014-06-19T10:09:00.002+12:002016-05-11T21:35:31.250+12:00GTK+ applications in Unity 8 (Mir)<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: left;">
<a href="http://blogs.gnome.org/desrt/">Ryan Lortie</a> and I have been tinkering away with making getting GTK+ applications to run in Unity 8 and as you can see below it works!</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/fCgt1MCG-30?feature=player_embedded' frameborder='0'></iframe></div>
<br />
This shows me running the <a href="http://www.olli-ries.com/doing-more-with-your-unity8-preview-session-in-ubuntu-14-04/">Unity 8 preview session</a>. Simple Scan shows up as an option and can be launched and perform a scan.<br />
<br />
This is only a first start, and there's still lots of work to be done. In particular:<br />
<br />
<ul style="text-align: left;">
<li>Applications need to set X-Ubuntu-Touch=true in their .desktop files to show in Unity 8.</li>
<li>Application icons from the gnome theme do not show (<a href="https://bugs.launchpad.net/bugs/1331309">bug</a>).</li>
<li>GTK+ applications don't go fullscreen (<a href="https://bugs.launchpad.net/bugs/1324824">bug</a>).</li>
<li>No cursors changes (<a href="https://bugs.launchpad.net/bugs/1206780">bug</a>).</li>
<li>We only support single window applications because we can't place/focus the subwindows yet (<a href="https://bugs.launchpad.net/bugs/1324101">bug</a>). We're currently faking menus and tooltips by drawing them onto the same surface.</li>
</ul>
<br />
If you are using Ubuntu 14.10 you can install the packages for this from a PPA:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">$ sudo apt-add-repository ppa:ubuntu-desktop/gtk-mir</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">$ sudo apt-get update</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">$ sudo apt-get upgrade</span><br />
<div style="text-align: left;">
<span style="font-family: inherit;"><br /></span></div>
<div style="text-align: left;">
<span style="font-family: inherit;">The PPA contains a version of GTK+ with Mir support, fixes for libraries that assume you are running in X and a few select applications patched so they show in Unity 8.</span></div>
<br />
The Mir backend currently on the <a href="https://git.gnome.org/browse/gtk+/tree/?h=wip/mir">wip/mir branch</a> in the GTK+ git repository. We will keep developing it there until it is complete enough to propose into GTK+ master. We have <a href="https://git.gnome.org/browse/jhbuild/commit/?id=658bb4fb89ead10b08eb2240e91cf1996c99dbfc">updated jhbuild</a> to support Mir so we can easily build and test this backend going forward.<br />
<br /></div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-47217156665968235632014-05-09T12:07:00.000+12:002014-05-09T12:07:46.921+12:00errors.ubuntu.com<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: left;">
One of the most useful tools I use is <a href="http://errors.ubuntu.com/">errors.ubuntu.com</a>. This shows statistics for the crash reports that are sent from users machines. Before I would trawl through <a href="https://bugs.launchpad.net/ubuntu">Launchpad</a> trying to work out which crash reports were more significant. Now we have numbers to see what's important.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Here is a graph showing the crash reports received for the last six releases over the last year:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-cRg3LZ7dgN0/U2wV7dtqYzI/AAAAAAAAJ54/KbIyD-nv9XA/s1600/Crash+reports.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-cRg3LZ7dgN0/U2wV7dtqYzI/AAAAAAAAJ54/KbIyD-nv9XA/s1600/Crash+reports.png" height="167" width="400" /></a></div>
Some interesting things from the graph:<br />
<br />
<ul style="text-align: left;">
<li>We get more crash reports on weekdays than weekends (see the ripple in the 12.04 and 12.10 lines which otherwise seem quite stable).</li>
<li>Apparently people stop using Ubuntu development releases around Christmas (huge dip in the middle of the Ubuntu 14.04 line). Stable releases are unaffected.</li>
<li>You can see a step in crash reports for the 14.04 beta release (March). Looks like there was an outage then too as every release has a dip in reports before.</li>
<li>It looks like as soon as 14.04 was released (April) there's been a rapid migration from 13.10 users to 14.04 so there is now probably less than half the number of 13.10 users there were before.</li>
<li>If you look closely you can also see a slight decrease in crash reports from 12.04 after the 14.04 release, so people are migrating LTS to LTS.</li>
<li>I guess the 12.10 users love it because they don't seem to have started migrating at all.</li>
<li>We get a huge number of crash reports from release days and these very smoothly drop off over approximately three months. I guess this is due to the bugs being fixed and the users slowly updating.</li>
<li>Sorry, I don't get the vertical axis any more than you do other than to say "bigger means more crash reports" (<a href="https://bugs.launchpad.net/bugs/1237499">bug</a>). The X axis also show months from 2013/2014 (<a href="https://bugs.launchpad.net/bugs/1276178">bug</a>).</li>
<li>Not sure why the left hand side is so high - have we really reduced crash reports that much?</li>
</ul>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-23751029972114647892014-03-23T13:45:00.001+13:002014-03-23T13:48:07.802+13:00Why the display server doesn't matter<div dir="ltr" style="text-align: left;" trbidi="on">
Display servers are the component in the display stack that seems to hog a lot of the limelight. I think this is a bit of a mistake, as it's actually probably the least important component, at least to a user.<br />
<br />
In the modern display stack there are five main components:<br />
<ul style="text-align: left;">
<li>Hardware</li>
<li>Driver</li>
<li>Display Server / Shell</li>
<li>Toolkit / Platform API</li>
<li>Applications</li>
</ul>
The hardware we have no control over. We just get to pick which hardware to buy. The driver we have more control over - drivers range from completely closed source to fully open source. There's a tug of war between the hardware manufacturers who are used to being closed (like their hardware) and the open source community which wants to be able to modify / fix the drivers.<br />
<br />
For (too) many years we've lived with the X display server in the open source world. But now we are moving into next generation display servers (as Apple and Microsoft did many years ago). At the moment there are two new classes of contender for X replacement, Mir and a set of Wayland based compositors (e.g. weston, mutter-wayland etc).<br />
<br />
Applications use toolkits and platform APIs to access graphical functionality. There are plenty of toolkits out there (e.g. GTK+, Qt) and existing libraries are growing more broad, consistent and stable to be considered as a complete platform API (which is great for developers).<br />
<br />
If you read the Internet you would think the most important part in this new world is the display server. But actually it's just a detail that doesn't matter that much.<br />
<ul style="text-align: left;">
<li>Applications access the display server via a toolkit. All the successful toolkits support multiple backends because there's more than one OS out there today. In general you can take a GTK+ application and run it in Windows and everything just works.</li>
<li>The hardware and drivers are becoming more and more generic. Video cards used to have very specialised functionality and OpenGL used to provide only a fixed function function. Now video cards are basically massively parallel processors (see OpenCL) and OpenGL is a means of passing shaders and buffer contents.</li>
</ul>
<div>
The result of this is the display server doesn't matter much to applications because we have pretty good toolkits that already hide all this information from us. And it doesn't matter much to drivers as they're providing much the same operations to anything that uses them (i.e. buffer management and passing shaders around).</div>
<div>
<br /></div>
<div>
So what does matter now?</div>
<ul style="text-align: left;">
<li>It does matter that we have open drivers. Because there will be different things exercising them we need to be able to fix drivers when display server B hits a bug but A doesn't. We saw this working with Mir on Android drivers. Since these drivers are only normally used by SurfaceFlinger there are odd bugs if you do things differently. Filing a bug report is no substitute to being able to read and fix the driver yourself.</li>
<li>The shell matters a lot more. We're moving on from the WIMP paradigm. We have multiple form factors now. The shell expresses what an application can do and different shells are likely to vary in what they allow.</li>
</ul>
<div>
I hope I've given some insight into the complex world of display stacks and shown we have plenty of room for innovation in the middle without causing major problems to the bits that matter to users. </div>
</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com0tag:blogger.com,1999:blog-20155916.post-9459431501585230902013-03-12T11:33:00.001+13:002013-03-12T11:33:42.697+13:00Ubuntu GNOME<div dir="ltr" style="text-align: left;" trbidi="on">
Thanks to the hard work of Jeremy Bicha and others Ubuntu GNOME is <a href="https://lists.ubuntu.com/archives/ubuntu-gnome/2013-March/000035.html">now an official Ubuntu flavour</a>. Flavours get some <a href="https://wiki.ubuntu.com/RecognizedFlavors">infrastructure and support benefits </a>such as ISO creation that make it easier to release and support.</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com2tag:blogger.com,1999:blog-20155916.post-33078018870069840732013-03-12T09:47:00.000+13:002013-03-12T09:47:56.667+13:00More information on Mir<div dir="ltr" style="text-align: left;" trbidi="on">
With the recent announcement of <a href="http://launchpad.net/mir">Mir</a> there's been some concern about what this means for Ubuntu and the wider Linux ecosystem. Christopher Halse Rogers who is on the Mir team has written some excellent posts covering some of the major questions: <a href="https://plus.google.com/u/0/113883146362955330174/posts/PXc93m8nKwk">why Mir and not Wayland/Weston</a>, <a href="https://plus.google.com/u/0/113883146362955330174/posts/P4GFie3VoD8">what does this mean for other desktops on Ubuntu</a> and <a href="https://plus.google.com/u/0/113883146362955330174/posts/QwMqCgC7c9G">what does this mean for Linux graphics drivers</a>.<br />
<br />
Well worth the read.</div>
Robert Ancellhttp://www.blogger.com/profile/06377999550703204187noreply@blogger.com4