Switching from WordPress to Nikola

Goodbye WordPress!

For a long while my personal blog has been running WordPress. Every so often I've looked at other options but never really been motivated to change it, because everything worked, and it was not too much effort to manage.

Then I got 'hacked'. :(

I host my blog on a Bitfolk VPS. I had no idea my server had been compromised until I got a notification on Boxing Day from the lovely Bitfolk people. They informed me that there was a deluge of spam originating from my machine, so it was likely compromised. Their standard procedure is to shutdown the network connection, which they did.

At this point I had access to a console to diagnose and debug what had happened. My VPS had multiple copies of WordPress installed, for various different sites. It looks like I had an old theme or plugin on one of them, which the attackers used to splat their evil doings on my VPS filesystem.

Being the Christmas holidays I didn't really want to spend the family time doing lots of phorensics or system admin. I had full backups of the machine, so I requested that Bitfolk just nuke the machine from orbit and I'd start fresh.

Bitfolk have a really handy self-service provisioning tool for just these eventualities. All I needed to do was ssh to the console provided and follow the instructions on the wiki, after the network connection was re-enabled, of course.

However, during the use of the self-serve installer we unconvered a bug and a billing inconsistency. Andy at Bitfolk spent some time on Boxing Day to fix both the bug and the billing glitch, and by midnight that night I'd had a bank-transfer refund! He also debugged some DNS issues for me too. That's some above-and-beyond level of service right there!

Hello Nikola!

Once I'd got a clean Ubuntu 16.04 install done, I had a not-so-long think about what I wanted to do for hosting my blog going forward. I went for Nikola - a static website generator. I'd been looking at Nikola on and off since talking about it over a beer with Martin in Heidelberg

Beer in Heidelberg

As I'd considered this before, I was already a little prepared. Nikola supports importing data from an existing WordPress install. I'd already exported out my WordPress posts some weeks ago, so importing that dump into Nikola was easy, even though my server was offline.

The things that sold me on Nikola were pretty straightforward.

Being static HTML files on my server, I didn't have to worry about php files being compromised, so I could take off my sysadmin hat for a bit, as I wouldn't have to do WordPress maintenance all the time.

Nikola allows me to edit offline easily too. So I can just open my text editor of choice start bashing away some markdown (other formats are supported). Here you can see what it looks like when I'm writing a blog post in todays favourite editor, Atom. With the markdown preview on the right, I can easily see what my post is going to look like as I type. I imagine I could do this with WordPress too, sure.

Writing this post

Once posts are written I can easily preview the entire site locally before I publish. So I get two opportunities to spot errors, once in Atom while editing and previewing, and again when serving the content locally. It works well for me!

Nikola Workflow

Nikola is configured easily by editing conf.py. In there you'll find documentation in the form many comments to supplement the online Nikola Handbook. I set a few things like the theme, disqus comments account name, and configuration of the Bitfolk VPS remote server where I'm going to host it. With ssh keys all setup, I configured Nikola to deploy using rsync over ssh.

When I want to write a new blog post, here's what I do.

cd popey.com/site
nikola new_post -t "Switching from WordPress to Nikola" -f markdown

I then edit the post at my leisure locally in Atom, and enable preview there with CTRL+SHIFT+M.

Once I'm happy with the post I'll build the site:-

nikola build

I can then start nikola serving the pages up on my laptop with:-

nikola serve

This starts a webserver on port 8000 on my local machine, so I can check the content in various browsers, and on mobile devices should I want to.

Obviously I can loop through those few steps over and again, to get my post right. Finally once I'm ready to publish I just issue:-

nikola deploy

This sends the content to the remote host over rsync/ssh and it's live!

Conclusion

Nikola is great! The documentation is comprehensive, and the maintainers are active. I made a mistake in my config and immediately got a comment from the upstream author to let me know what to do to fix it!

I'm only using the bare bones features of Nikola, but it works perfectly for me. Easy to post & maintain and simple to deploy and debug.

Have you migrated away from WordPress? What did you use? Let me know in the comments below.

My Ubuntu 16.04 GNOME Setup

My Ubuntu 16.04 GNOME Setup

This is a post for friends who saw my desktop screenshot and anyone else who likes Unity and is looking at alternatives. A big thanks to Stuart Langridge and Joey Sneddon whose linked posts inspired some of this.

The recent news that upcoming versions of Ubuntu will use GNOME as the default desktop rather than Unity, made me take another look at the GNOME desktop

If you're not interested in my opinion but just want to know what I did, then jump to "Migration from Unity to GNOME" below.

Why Unity?

I'm quite a Unity fan - yes, we exist! I've used it on my laptops and desktops my daily desktop pretty much since it came out, long before I worked at Canonical. I've tried a few other desktop environments, usually for no more than a week or so before getting frustrated and running back to Unity.

Here's what my typical desktop looks like on Ubuntu 16.04

Unity as I use it

At this point I'm sure there are a few people reading this and wondering why I like Unity, incredulous that anyone would. I get this from time to time. Some people seem to bizzarely think "I don't like Unity, therefore nobody does." which is ludicrous, but very obviously happening.

Anecdotally, I still see way more Unity screenshots than other desktops in random non-Linux videos on YouTube, on stranger's laptops on trains & on "millions of dollars" worth of laptops sold by Dell, System76 etc. I've also been told in person by people who like it, but don't like speaking up for fear of unwanted confrontation. ¯\_(ツ)_/¯

But it's a vocal minority of Linux users who tell me what desktop I (and everyone else) shouldn't use. Screw them, it's my PC, I'll run what I like. :)

However, that said, Unity is "dead", apparently, despite it having a few years of support left on the 16.04 LTS release. So I thought I'd take a fresh look at GNOME to see if I can switch to it easily and keep the parts of the Linux desktop I like, change the things I don't and tolerate the things I can't.

For me, it's not one single feature that made me come back to Unity time and time again, but a variety of reasons. Here's a non-exhaustive list of features I enjoy:-

  • Dash - Single button + search to launch apps and find files
  • HUD - Single button + search to find application features in menus
  • Launcher - Quick access via keyboard (or mouse) to top 10+ apps I use, always on screen
  • Window controls - Top left is their rightful place
  • Menus - In the titlebar or top bar (global)
  • App & Window switch behaviour via Alt+Tab & Alt+(key-above-tab)
  • App Spread - Super+S and Super+W to see all windows, or all windows of an app
  • Focus follows mouse - Initially global menu broke this but it was fixed

Much of this comes down to "is really well managed with just a keyboard" which is amusing given how many people tell me Unity (before Unity 8) is awful because it's designed for touch screens.

The things I think could be improved in Unity comprise a pretty short list, and if I thought really hard, I might expand this. If I did they'd probably only be papercut nit-picks rather than significant issues. So, I would have liked these things to have been fixed at some point, but that probably won't happen now :(

  • Memory footprint - It would be nice if the RAM usage of Unity was lower.
  • CPU/GPU overhead - Sometimes it can take a second or two to launch the dash, which should be near-instant all the time
  • Incompleteness - There were interesting designs & updates which never really got finished in Unity7
  • Cross distro support - It would have been nice to have Unity on other distros than just Ubuntu

So let's say a fond farewell to my primary desktop for over 6 years and make the switch.

Migration from Unity to GNOME

With that said, to move from Unity to GNOME on my ThinkPad T450 running Ubuntu 16.04 LTS I did the following:-

Install GNOME

I decided to go with the GNOME version shipping in the archive. People have suggested I try PPAs, but for my primary desktop I'd rather keep using the archive builds, unless there's some really compelling reason to upgrade.

So I backed up my laptop - well, I didn't - my laptop is backed up automatically every 6 hours, so I just figured if anything went belly-up I'd go back to the earlier backup. I then installed GNOME using this command:-

sudo apt install ubuntu-gnome-desktop^

Logout from Unity, choose GNOME at the login screen and we're in.

Default GNOME Desktop

Default GNOME Desktop

First impresssions

These are the things that jump out at me that I don't like and how they're fixed. One thing that's pretty neat about GNOME Shell is the ability to modify it via extensions. For most of the things I didn't like, there was an extension to change the behaviour.

Some are just plain extensions installed via GNOME Extensions, but some needed extra fiddling with Tweak Tool.

Activites hot corner

I find this too easily triggered, so I used No TopLeft Hot Corner. Later, I also discovered the Hide Activtes Button which helps even more by moving the window controls to the very top left, without the "Activities" in the way. I can still use the Super key to find things, with Activities hidden.

No Launcher

GNOME hides the launcher until you press Activites or the Super key. I fixed that with Dash to Dock.

In Tweak Tool, Dash to Dock settings -> Position and size -> tick "Panel mode: extend to the screen edge". I set "Intelligent Autohide" off, because I never liked that feature in Unity, although it had some vocal fans. Also I set the pixel size to 32px. In the Launchers tab I set "Move the applications button at the beginning of the dock".

Legacy indicators

Apparently developers are terrible people and haven't updated their indicators to some new spec, so they get relegated to the "Lower Left Corner of Shame". This is dumb. I used TopIcons Plus to put them where $DEITY intended, and where my eyes are already looking, the top right corner.

Volume control

In Unity I'm used to adjusting the master volume with the mouse wheel while the sound indicator is clicked. I fixed this with Better Volume Indicator

Giant titlebars

GNOME always feels to me like it's designed to waste vertical space with titlebars so I added Pixel Saver.

Missing Rubbish Bin

I like having the Trash / Rubbish Bin / Recycle Bin / Basket on screen. In Unity it's at the bottom of the launcher. I couldn't find an extension which did this so I used trash extension to move it to the top panel indicator area.

Slow animations

Some things felt a bit sluggish to me, so it was recommend that I install the Impatience extension, which seems to have helped my perception, if nothing else.

Remaining niggles

Things I haven't figured out yet. If you have any suggestions, do let me know in the comments below.

  • How to hide the clock completely
    • I frequently record screencasts of my laptop and the time jumping around in the recording can be distracting. So I just hide the clock. I don't see an easy way to do that yet.
  • Make accelerator keys work in alt+space window menu
    • For many years I have used the accelerators in the window controls menu accessed via Alt+space to do things like maximize the window. Alt+Space,x is welded in my muscle memory. I don't understand why they were disabled in GNOME Shell (they work in other desktops).
  • Alt-Tab behaviour is broken (by design (IMHO))
    • All windows of an application come to front when Alt+Tabbed to, even if I only want one window. I have to dance around with Alt+Tab & Alt+Grave.

Reader Suggestions

In the comments below, the following addtional extensions have been suggested.

Greg suggested the Alt Tab List First Window Extension which on initial play seems to fix the Alt-Tab issue listed above! Many thanks Greg!

Alif mentioned Status Area Horizontal Spacing which is great for compressing the gaps out of the indicator area in the top right, to use the space more efficiently. Thanks Alif!

Conclusion

So this is now my desktop, and I'm quite pleased with it! Massive thanks to the GNOME team, the Ubuntu GNOME flavour team, and all the extension authors who made this possible.

My new Ubuntu GNOME Desktop

My new Ubuntu GNOME Desktop

Initially I was a bit frustrated by the default behaviour of GNOME Shell. I've been pleasantly surprised by the extent and functionality of extensions available. Without them, there's no way I'd use this on a daily basis, as it's just too irritating. I'm sure somebody loves the default behaviour though, and that's great :)

I realise I'm running an 'old' version of GNOME Shell (3.18) coming directly from the Ubuntu 16.04 (LTS) archive. It may be there's additional features or fixes that are going to improve things further. I won't be upgrading to 16.10, 17.04 or 17.10 however, and likely won't use a GNOME PPA for my primary desktop. I'll stick with this until 18.04 (the next Ubuntu LTS) has baked for a while. I don't want to upgrade to 18.04 and find extensions break and put me backwards again.

I've had this setup for a few days now, and I'm pretty pleased with how it went. Did you try this? Any other changes you made? Let me know in a comment below! Thanks. :D

Dell XPS 13 9360 Review

Dell XPS 13 9360 Review

On the 'Tasty Different Cow' (don't ask) episode of the Ubuntu Podcast - we reviewed the latest Dell XPS 13 9360 Laptop shipping with Ubuntu.

Dell kindly sent us the review unit for a couple of weeks, and while we talked all about it on the podcast, I thought I'd jot some notes down here in case I missed anything or it's not clear in the audio version.

Turns out I made more notes than I thought! Scroll to the very bottom to read my conclusion.

Hardware

First up, what did I get? Dell have a ton of different laptops available, shipping with Ubuntu out of the box, this is the one I got. I didn't get to choose it, they just sent me this unit, which I suspect has done the rounds to a few proper journalists before I got my grubby mits on it.

Specs

  • i5-7200U 2.5GHz
  • 8GB RAM
  • 256GB SSD
  • Atheros QCA6174A 802.11ac wireless
  • 1080p matte screen
  • SD slot
  • Combined headset/mic port
  • 2xUSB 3 (one each side)
  • 1xUSB-C peripheral and charging port
  • US keyboard layout
  • 60Wh design battery - model RNP726B

Mine also shipped with a very dinky 45W charger which terminates in a barrel connector. The power socket on this Dell is on the left side at the back, next to the USB C port. I didn't try charging via USB C because I don't have a high enough wattage charger - my OnePlus Dash charger wasn't sufficient. I also don't have any USB C peripherals, so couldn't test that port at all.

The laptop feels premium, very sturdy to hold, like it would stand up to some abuse, but I think a prior journalist has tested that a bit too far. My review unit was a bit bent, so rocked on the desk when typing. I can't imagine brand new factory models have this issue though, so it didn't colour my review.

Software

The Dell shipped with a relatively up to date install of Ubuntu 16.04 LTS. It didn't have the latest HWE stack installed, but that's not a big problem.

The install wasn't an out of the box experience, but had a password stuck on the box, which again, I assume is not normal ;) but for the journalists using this review unit.

Differences from stock Ubuntu

Once I logged in, there were a few things I noticed which differ from the standard Ubuntu 16.04 install I'm used to on my ThinkPad T450. This may be old news to those of you who have tried a Dell XPS 'Sputnik' laptop in the past, but it was new to me, a die-hard ThinkPad fanboy :)

There's a Dell PPA enabled by default, and some of the packages installed on the system came from there. I expect this was to support the hardware or known issues Dell customers had. Good to see, so long as it's maintained and doesn't break upgrades, in my opinion.

The Windows 'Super' key doesn't work by default. This was a bit of a shock to me, as I use the Super key on Unity all the time to launch applications and find files. Having to move my hand from the keyboard to the trackpad or mouse and move it all the way over to the top left is super inefficient and breaks the keyboard-centric nature of Unity. Thankfully this mis-feature can be disabled, Andrew Hayzen wrote some notes about his laptop (a similar model) detailing how.

The base software shipped on the laptop is different from a stock install too. With both Google Chrome and Chromium installed and listed in the launcher. Firefox isn't installed at all on this image. I don't know if the motivation is user-centric or financial, but I had no problem with this change, as I've used Google Chrome as my default browser for some time now. Others may prefer Firefox, and of course that can easily be installed.

The combined headphone/headset port was interesting to me because while I have the same on my ThinkPad, the behaviour was different. Plugging headphones into the port presented a popup dialog asking the user what type of device was attached. A handy dialog the first time I saw it, but it got a bit annoying every single time I plugged my headphones in.

Dell Recovery

One of the applications shipped from the Dell PPA is the "Dell Recovery" tool. It can make recovery USB sticks, and reboot into a recovery tool which resets the laptop back to factory defaults. This is a most welcome addition to the Ubuntu install im my mind. I've often wanted to nuke a laptop back to initial install and start fresh. I'm sure many other people have too. Having the tool built in rather than having to download an ISO and put it on a USB stick is a bonus. The downside is that the recovery media eats a chunk of your precious laptop disk space. In my mind, this is worth having.

I tested the recovery tool a couple of times, initially to see if the version of Ubuntu I had been shipped installed was the same as the out of the box one from Dell. It was. The tool worked perfectly each time, and got me back to factory install in a few minutes, with little interaction from me, just a confirmation and password prompt.

The ability to make a recovery USB stick - tailored for Dell devices - which has the PPA enabled, and the customisations mentioned previously, is also a welcome bonus. In fact my good friend and colleague Martin Wimpress made use of this on my review laptop to make a USB stick to reset his Dell back to factory defaults. More manufacturers of Linux machines should do this kind of customisation, it's fantastic.

Daily uses

Overall the laptop 'feels' really fast, especially when building software or other intense things. The fan did spin up a bit on heavy load, which was pretty audible, but I didn't hear the famous 'coil whine' that others have reported with previous models.

I had the 1080p matte screen version. I don't think I'd go for a hidpi, touch or reflective version. 1080p seems the right resolution for the 13" screen.

The keyboard has only a short travel, but that didn't present a problem when I was typing on it. I did however find myself feeling cramped using the laptop on a desk. The tiny bezel makes the 13" model look and feel pretty tiny and a bit cramped.

As mentioned, the laptop I got has a US layout, which also has the modifier keys in a strange (to me) layout. The order from bottom left is Ctrl, Fn, Win, Alt. I'm more used to having Fn in the far left, I'm sure I'd get used to this, but it threw me initially, having Ctrl farther away for my left thumb than I'm used to.

As with most laptops of the day, the Fuction keys (F1-F12) double up as media device control keys. The Fn key in the bottom left corner of the keyboard is both a 'shift' (hold-and-press) key and a toggle (tap to change mode) key. The problem here is there's no indication to tell you which mode you're in before you press a function key. The ThinkPad lines have a little light on the key itself so you know if you're in "function key" or "device control" mode, but the Dell doesn't, which is a little odd and takes getting used to.

The tiny bezel does however make my ThinkPad look like it was made in 1984. The Dell looks like a modern, premium device, not something fabricated in an eighties Soviet nuclear bunker.

The downside of having nearly no bezel is that camera location. Having been on plenty of hangouts with friends and colleagues who own this device, I now know what the insides of their nostrils look like. Not a great look.

I noticed some flicker in Google Chrome, which I've also seen on my ThinkPad, so I suspect it's either a software issue with Chrome, Unity or Compiz, or perhaps a GPU driver bug. I never got to the bottom of it on my ThinkPad, and haven't seen it on non-Intel machines. ¯\_(ツ)_/¯

Initially the Ubuntu battery gauge says 9h battery life on a full charge after boot up. I did a bit of 'work' on it and the fan spun up a bit, whereupon the battery gauge went down to 4.5h left. Once the workload was comlpete, the fan eventually span down and estimated battery life shot back back up to 7 hours. So 9h is "doing nothing" battery life, 4 hours could be considered "quite busy" battery life. Real life I found, somewhere in between, doing the usual workload of typing docs, browsing, a bit of audio and the odd game. Clearly this will vary greatly on your workload. I can't see how anyone can get more than about 7-8 hours under real world conditions though.

Media

I use Spotify for my music these days, and am not any kind of audiophile. I am usually happy listen to music on my OnePlus3T headphones, or a bluetooth speaker. So whatever I say about audio quality has to be taken with a pinch of salt.

I initially listened to 3 'test' tracks on the laptop, Sparks - "This Town Aint Big Enough", XTC - "Making Plans for Nigel" and Jean-Michel Jarre - "Equinoxe Pt 5", as they're all songs I'm likely to listen to in my regular playlist.

Unsurprisingly the speakers are exactly what you'd expect from non-Apple laptop units. Basically "okay", but not "great" by any stretch. They're alright for background music, watching a talky YouTube video or listening to the radio, but not for rocking out with. I had to dial down the volume a bit for most tracks as they're too loud in mid range. As expected, when using headphones, it was a different story. No problem at all, but nothing I'd rave about.

Steam / Gaming

I installed steam from the Ubuntu repo with a simple apt install steam and launched it from the dash. It did the usual update dance and all worked fine, no problem. I installed a bunch of games including Mini Metro, Goat Simulator and Talos Principle.

Mini Metro ran perfectly, unsurprisingly as it's not a massively demanding game. You should buy it though, it's great fun if you're even remotely a train nerd, or like the aesthetic of London Undergroud maps.

Goat Simulator defaulted to 720p (on a 1080p panel) and was just about playable, but at 20fps, with the fan blasting out, I don't think I'd recommend it. Perhaps dialling down some detail or reducing the resolution even further might have helped, but I gave up after a few minutes.

Talos Principle however was excellent. On first launch I got around 30fps at 1080p which for a pretty puzzle game was fine. I played this quite a bit, and had no issue with the actual game itself. However at one point I tried to take a screenshot and the laptop froze for a minute or so. I alt-tabbed out and found a GPU hang in dmesg. So I guess that's an Intel video driver or mesa issue. I only had it happen once though.

I did try updating to the latest HWE stack and latest Mesa drivers which made no discernable difference to any of the games I tried, nor the flickering in Chrome.

I did a firmware update via GNOME Software. "XPS 13 9360 System Update 66306" which says "Updating the system firmware improves performance" and (in red) "Device cannot be used during update". If I click it there is "No update description available" which doesn't inspire confiidence. However, if I click "Restart and install" it flickers and then I get the usual Ubuntu shut down dialog, so I clicked "Restart". After reboot the Dell logo appeared and then it proceeded to apply the update successfully then rebooted back into Ubuntu. So well done Dell & GNOME Software developers for making firmware updates easy on Linux!

Benchmarking

I used phoronix test suite to run a batch of tests. Here's the results. Unfortunately I did this on the base 16.04 install, and not with the HWE stack, perhaps someone else who has updated to the latest HWE / Mesa can re-run them and compare. Leave a comment below if you have.

Questions

We asked some listeners of Ubuntu Podcast in our Telegram Chat if they wanted me to test anything, and here's what they said:-

Andrew Hayzen, who has the fully loaded version offered his notes up http://www.ahayzen.com/docs/installs/devices/dell_9360/

John Lenton asked, "does it sleep or suspend? What does powertop say it eats when on battery, noodling around? (run a powertop --calibrate first, but note that takes a while and you can't use it)"

It suspends. After calibartion, 6.6w with just terminal open doing nothing much, down to 5.9 if I lower the brightness nearly all the way.

Sturmflut asked " does the external display come back after suspend?"

I have no usb c to display cable - so can't test as it has no VGA or HDMI. Andrew has one though and says it does!

Joe Ressington asked "does everything work in trisquel?"

Unfortunately I couldn't get Trisquel to boot at all, so couldn't test this.

Andre Bacao asked a 'battery' of questions, "Battery time? Time to charge? How is the charger? Small? Bulky? Full load noise? Does it heats up by being plugged?"

Battery life was betwen 4 and 9 hours depending on load. I didn't get a chance to time the charge, but it didn't take long. It was quite warm but not unpleasantly hot.

André Bação then asked "Charger watts and power cord lenght? Suitable or lacking in range?"

It's a small non-bulky 45W model. There was a pretty long cable on it too, so I didn't feel I was stretching it to get to a mains outlet.

Joe Ressington also asked "Oh straight connector or right angle? Seems minor but means the difference between longevity and a quickly broken power jack"

It's a straight barrel connector with a light on the connector itself so you know it's plugged into the mains under the desk.

Finally, André Bação asked "Keyboard feel? Does the keys touch the screen when closed? Hate when the screen has keys marks"

I certainly found it different to type on than my T450. The lack of travel was certainly notiable initially, but I soon got used to it. I couldn't see any marks on the screen from the keys.

Thanks for the questions!

Conclusion

Overall I'd have to say my experience with this laptop has made me re-think my opinion of Dell laptops. I've been a ThinkPad (X220 then T450) user for some years, and had a Apple MacBook Pro before that. When I bought all of those laptops, I'd looked at Dell and relegated them to 3rd or 4th place in my shopping list. This laptop changed that. This has pushed the Dell XPS to the top of my list despite it not having a TrackPoint, which I've usually used as an excuse to stay on the ThinkPad train. The only thing I'd probably consider is getting a physically larger display such as the 15" version.

Snapcraft Docs Day

Announcing Snapcraft Docs Day

Snap is a simple archive format for big things.

Snapcraft is a delightful tool for automatically building and publishing software for any Linux system or device. Our documentation and tutorials are great for getting started with snapcraft. We can always improve these though, so this Friday, will be our first Snapcraft Docs Day.

  • When: Friday, 31st March 2017, all day
  • Where: #snapcraft on Rocket Chat
  • Who: Developers & documentation experts of all levels

Why we're doing this

The goal is to ensure our documentation and tutorials are useful and accurate. We’re keen to get people testing our documentation, to make sure it’s clear, understandable and comprehensive. If we’re missing anything, or there are mistakes then file those issues, or better yet, fix them yourself.

If you’ve got something you want to snap, this is also a great day to get started. We’ve personally used these tools all day every day for a couple of years now, but perhaps we’re missing something you need. Now is a great time to test the tools and let us know.

Get involved

If you’re interested in contributing to the projects but don’t know where, here’s a great place to start. Snapcraft and Snapd are both free software, and hosted on GitHub. Snapcraft is written in Python, and Snapd is a Go-based project. The teams behind these projects are super friendly, and keen to help you contribute.

Some examples of things you might want to try:-

Hang out with us in #snapcraft on Rocket Chat during the day. Many of the snapd and snapcraft developers are there to answer your questions and help you. See you there!

Code and bug trackers

Here's a handy reference of the projects mentioned with their repos and bug trackers:-

Project Source Issue Tracker
Snapd Snapd on GitHub Snapd bugs on Launchpad
Snapcraft Snapcraft on GitHub Snapcraft bugs on Launchpad
Snapcraft Docs Snappy-docs on GitHub Snappy-docs issues on GitHub
Tutorials Tutorials on GitHub Tutorials issues on GitHub

Tuesday is Snappy Playpen Day

snappy

For some weeks now on the Ubuntu Community Team we've been setting aside Tuesdays as Taco "Snappy Playpen" Tuesdays.

Join us on Tuesday 6th September 2016 in #snappy on freenode IRC or on Gitter.

The general goal of the Snappy Playpen is simple:-

  • Create snaps for a broad range of interesting, popular, or fun applications, games and utilities for desktop, IoT and Server
  • Exercise & improve Snapcraft and snapd on multiple Linux distros
  • Answer questions from Snappy users and developers

This week we're focusing on the following, but welcome all contributions, as always:-

  • Fixing issues with existing playpen items
  • Work through backlog of pull requests
  • Create snaps for our Core Apps for the desktop, focused on Dekko, Terminal, Doc Viewer, File Manager and the Libertine GUI
  • Answer questions in questions
  • Help developers in #snappy and Gitter

We use GitHub and CodeReviewHub to manage the development of our snaps.

See you there! :D

Testing Ubuntu Apps As A Service

tl;dr. Stuart Langridge and I made an simple, easy to use, experimental app tester called Marvin, for Ubuntu Click Packages, which emails you screenshots and logs of your app while running on a real device you may not own.

Screenshot of email from Marvin

I frequently get asked by new developers in the community to help test their apps on Ubuntu Phone. Typically they don't want extensive testing of all features, just a simple "does it start and what does it look like on the device you have?".

Often they don't have a physical device when developing on the desktop with our SDK, but want an on-device sanity check before they upload to the store. Sometimes they have one device such as a phone, but want to see what their app looks like on a different one, perhaps a tablet.

I've been happy to help developers test their apps on various devices, but this doesn't scale well, is time consuming and relies on me being online and having a phone which I'm happy to install random click packages on.

Meanwhile, at OggCamp I gave a short talk about our recent security incident on Ubuntu phone. During the Q&A and in the bar afterwards a couple of people suggested that we should have some system which enables automated testing of devices. They were coming at it from the security point of view, suggesting heavy instrumentation to find these kinds of issues before they hit the store.

While we (Canonical) already have tools which review apps before they go in the store, we currently don't actually install and execute the apps on devices, and have no plan to implement such a service (that I know of).

I'm aware that other platforms have implemented automated systems for testing and instrumenting apps and wondered how hard it would be to setup something really basic to cover at least one of the two use cases above. So I took to Telegram to brainstorm with my good friend Stuart Langridge.

Snippet of conversation with Stuart

We thrashed out what was needed for a 'minimum viable product' and some nice-to-have future enhancements. Pretty soon after, with a bit of python and some hacked-together shell scripts, 'Marvin' was born. I then approached Daniel McGuire who kindly provided some CSS to make it look prettier.

Screenshot of upload page

A developer can upload a click package to the site, and specify their email address & one or more of the available devices. Some time later they'll get an email showing a few screenshots of the app or scope running on a device and pertinent logs extracted after it ran. While the developer waits, the website shows the current status as 'pending' (you're in a queue), 'claimed' (by a device) and 'finished' (check your inbox).

This fulfills the simplest of use cases, making sure the app starts, and extracting the log if it didn't. Clearly there's plenty more it could potentially do, but this was our first target met.

Under the covers, there's a device attached to a computer which checks periodically for uploaded clicks and processes them in sequence. In between each run the phone is cleaned up, so each test is done on a blank device. Currently it tests traditional apps/games and scopes, webapps are rejected, but may be supported later.

The reason we reject webapps is because currently the devices have no network access at all - no wifi or cellular data. So running webapps would just result in this:-

Screenshot of webapp with no network access

It's experimental so not completely robust, being a prototype hacked together over a couple of weekends/evenings, but it works (for the most part). There's no guarantees of availability of the service or indeed the devices. It could go offline at any time. Did I mention it's experimental?

Significantly, I've disabled network access completely on the device, with no SIM inside, so any app which requires external network access is going to have a bad day. Locally installed apps however, will work fine.

We currently don't do any interaction with the uploaded applications, but simply run them and wait a few seconds (to give it time to quiesce) then take a screenshot. The image at the top of this post shows what a typical email from Marvin looks like.

It contains:-

  • click-review.txt - The output from running the click review tools
    • Note: Apps which fail the click review process (the same one run by the click store) will not be installed or tested.
  • install.txt - Output from the commands used to install the click on the device - good for debugging install failures
  • Screenshot-0 - What the "home" app scope looks like with the click installed - useful for showing the icon and description
  • Screenshot-1 - What happens immediately after starting the app, showing the splash screen
  • Screenshot-2 - The app after 5 seconds
  • Screenshot-3 - The app after 10 seconds
    • Note: We attempt to de-duplicate the screenshots so you may not get all four if any are identical
  • application-log.txt - The actual output (stdout) from the application, pulled from ~/.cache/upstart
  • dmesg.txt - Any kernel logging generated from the app during the app run
  • device-version.txt - The output of 'system-image-cli --info' run on the device, so the developer knows what OTA level, channel and device it ran on

There's clearly a ton of other things that could be added to the mail, or extra items which could be instrumented or monitored, and features we could add. Off the top of my head we could potentially add:-

  • Scripted touch/gestures
  • Networking
  • VPN endpoints (so the phone looks like it's in a particular region)
  • Orientation changes
  • Faked GPS location
  • Video/screencast recording during runtime
  • Input from microphone / camera(s)
  • Specify which release / channel to flash on the device prior to testing

Clearly all of these need some careful thought and planning, especially those enabling network access from the device.

We're interested in feedback from developers who might use Marvin, and suggestions for improvements we might make. There are a limited number of devices in the pool, and not all supported devices are currently available. In the future we may have more devices connected to Marvin as they become available.

So go and test your apps at marvin.popey.com!

Troubleshooting as a Choose Your Own Adventure

READ MAP

We have a lot of documentation and help in the Ubuntu project, and much of it is quite hostile to new users. We have IRC channels, mailing lists, dense & out of date wiki pages, lengthy and hard to consume forum posts & lengthy out of date tutorial videos. We also have some more modern tools such as AskUbuntu and Discourse.

Most are good for asking one specific question, but most aren't well suited to guiding a user through a specific problem diagnosis. If you know what questions to ask, then a search engine might find part of the problem, or hopefully part of the solution.

However one aspect we don't cover very well is guided self-support. This is very apparent in distribution upgrades. People often give up completely when an upgrade breaks down, rather than work through the problem as they would with anything else. There can be many reasons for this of course, but from what I've seen in the community, fixing a broken upgraded system is hard, and made harder when you only have a black screen, or tty to work with when you're not an expert.

GET ROPE AND SWORD

I'm thinking very specifically about a target audience of non-technical users. Someone who uses Ubuntu but wants to feel empowered to fix a problem themselves, and not have to call their wife or daughter or other 'expert' to fix it. I want someone to find a guide which gets them out of the upgrade issue. Obviously I'd like us to fix the problem which cause the issues, but I want to start here, because frankly it's easier for me.

WAIT

While I can hear some of my co-workers shouting "But popey, Snappy Core fixes the issue of broken updates!", no it doesn't, not today, and not while there are people running the traditional Debian based desktop - which I suspect will be for a good few years yet.

WAIT

I can also hear those of you saying "I never do upgrades, I always clean install", and I'm happy for you, but we have an upgrade system, people should be confident that it works, and when it doesn't it should be fixable without going nuclear. Just the same as "Buy a new TV" isn't usually the solution to "My TV Remote stopped working".

EAT LUNCH

In my mind a user might be more inclined to fix their system if there's an easy to use 'expert system' which they can walk through to get them working again. I wondered about this some time back and considered the idea of a Choose Your Own Adventure style of troubleshooting issues, rather than just punting people to IRC when their system breaks. We have been helping people with broken upgrades for years, surely we can amass the technical knowledge into a self-service system to help future users.

One assumption I am making with this is that the person going through this is able to access the Internet (and thus this guide) via another machine or their phone. This seems semi-reasonable as we often get people in IRC or on other support resources asking for help with a broken upgrade. So the guide should work well on any browser and ideally on mobile too.

There are of course unfortunate people for which they have only one computer and no smartphone so have no way to access this resource easily. This system doesn't cater for them well.

GET BARREL

I made a little prototype using Twine which is an "an open-source tool for telling interactive, nonlinear stories." - typically text adventure or "Interactive Fiction" games, but can be used for other things too. I chose Twine because it's open source, easy to use, cross platform and simple. You can even create Twine 'stories' directly in your browser. Neat!

The output generated by Twine is HTML and can be customised and styled with a stylesheet. I did one simple bit of styling in that I used the Ubuntu font :).

THROW BARREL THROUGH TRAP DOOR

Here's what the map of the 'world' looks like inside Twine (after I carefully moved the blocks around so none criss-cross):-

Twine Screenshot

Editing the pages within the 'story' is super easy, linking to other pages is as simple as typing the name of a new page in square brackets:-

Editing a page

You can try out the very early prototype I made at http://popey.com/~alan/twines/UbuntuUpgrades.html. Obviously this isn't finished, won't actually help anyone fix their system, and is hosted somewhere obscure, all "TODO" items.

OPEN CURTAIN

There's a few known issues with the above prototype twine:-

  • Browser back button exits the story, it should go back a step
  • Twine back button is hard to see, some CSS will fix that
  • Many of the pages are quite wordy, or technical, probably could be simplified

SAY TO BARD "SHOOT THE DRAGON"

I wanted to share what I'd done to see if it seemed like a good idea, and whether people might be interested in helping create and curate content for it. The idea is to make a prototype and get some content rather than make the thing look especially pretty right now. The way I see it there's some important steps to do:-

  • Confirm this isn't a stupid idea (this blog post)
  • Figure out the best way to distribute this and make it accessible
  • Find people interested in helping
  • Identify a bunch of specific breakages that happen in Ubuntu upgrades
  • Craft diagnostic steps to identify one breakage scenario from another
  • Come up with robust solutions for each scenario
  • Test the scenarios
  • Publish and publicise the work

Things I haven't yet considered, but probably should:-

  • Translations
  • Accessibility
  • Co-ordinating work
  • How far back I'll roll my eyes at people telling me rolling distributions are better

PUT TREASURE IN CHEST

Comments, suggestions or flames are welcome here or in my inbox.

DevRelCon 2015 Trip Report

Huh, this turned out to be longer than I expected. Don't feel obliged to read it, it's more notes for myself, and to remind me of why I liked the event.

Event

On Wednesday I went to DevRelCon in London. DevRelCon is "a one day single track conference for technical evangelists, developer advocates and anyone interested in developer relations" setup by Matthew Revell. I don't think there's a lot of difference between my role (defined as Community Manager) at Canonical and Developer Relations so figured it would probably have appropriate content for my role. Boy was I right!

DevRelCon was easily the single most valuable short conference I've ever attended. The speakers were knowledgeable, friendly and accessible, and easy to understand. I took a ton of notes, and will distil some of them down here, but will almost certainly keep referring back to them over the coming months as I look to implement some of the suggestions I heard.

Venue

The event took place at The Trampery Old Street, in Shoreditch, the trendy/hipster part of London. We had access to a bright and airy 'ballroom' and were served with regular drinks, snacks and a light lunch. Free WiFi was also available, which worked well, but I didn't use it much as we had little time away from the talks.

Structure

The day consisted of a mix of long (40 minute) talks, some shorter (20 min) ones, and a few 'lightning' talks. Having a mix of durations worked well I think. We started a little late, but Matthew massaged the timetable to claw back some time, and as it was a single track day there was no real issue if things didn't run to time, as you weren't likely to run off to another talk, and miss something.

All the talks were great, but I took considerably more notes in some than others, so this is represented below in that I haven't listed every talk.

Morning Talks

Rob Spectre - Twilio - Scaling Developer Evangelism.

This started off well as Rob plugged in his laptop and we were greeted with an Ubuntu desktop! He started off detailing some interesting stats to focus our minds on who we're evangelising to. Starting with the 18.2m developers worldwide, given ~3Bn smartphone users, and ~4Bn Internet users that means ~0.08% have the capability to write code. There's a 6% year on year increase in developers, mostly in developing nations, the ratio is less in the western world. So for example India could overtake every other countries' developer count by ~2017.

Rob talked at length about the structure of Developer Evangelists, Developer Educators and the Community Team at Twilio. The talk continued to outline how valuable developers are, how at Twilio their Developer Evangelists are the 'Red Carpet' to their community. I was struck by how very differently we (Open Source projects) and Ubuntu specifically treat contributors to the project.

There was also a section on running developer events, and Rob spent some time talking about strategies for successful events, and how those can feed back to improve your product. He also talked a little about measurement, which was also going to be covered in later talks that day.

Another useful anecdote Rob detailed was regarding conversion of talks into blog posts. While a talk at an event can catalyse the 20-100 people in the room, converting that into a detailed tutorial blog post can bring in hundreds or thousands more.

The final slide in Rob's talk was "Would you recommend this talk?" with a phone number attendees could send a score to. I thought this was a particularly cunning strategy. There was also talk of using the external IP address of the venue WiFi as one factor to determine the effectiveness / conversion rate of attendees.

Cristiano Betta - Braintree - Tooling your way to a great devrel team

Cristiano started off talking about BattleHack which I'd not heard of. These are in person events where teams of developers get 24 hours to work on a project fuelled by coffee, cake and Red Bull to be in with a chance of winning a cash prize and an amusing axe.

He then went on to talk about a personal project to manage event sign-ups. This replaces tools like Eventbrite and MailChimp and enables Cristiano to get a better handle on the success of his events.

Laura Cowen - IBM - Building a developer community in an enterprise world

Laura started off giving some history of the products and groups inside IBM who are responsible for WAS, the public facing developer sites and the struggles she's had updating them

The interesting parts for me came when Laura was detailing the pain she had getting developer time to update documentation and engage with users and communities outside their own four walls. Laura also talked about the difficulty when interfacing developers and marketing, their differing goals and some strategies for coping.

I recognised for example the frustration in people wanting to publish everything on a developer site, whether it's appropriate to the target audience or not. Sometimes we (in Ubuntu) fail to deeply consider the target audience before we publish articles, guides or documentation. I think we can do better here. Pushing back on content creators, and finding the right place for a published article is worth it, if the target audience is to be defended.

Lightning Talks

Shaunak Kashyap - elastic - Getting the measure of DevRel

In this short talk Shaunak gave some interesting snippets on how elastic measure community engagement. I found a couple interesting which I felt we might use in Ubuntu. Measuring "time to first response" for questions and issues by looking for responses from someone other than the first poster. While I don't think they were actively using this data yet, getting an initial base line would be useful.

Shaunak also detailed one factor in measuring meet-up effectiveness. Typically elastic have 3-4 meet-ups a week, globally. For each meet-up group they measured "time since last meetup". For those where there was a long delta between one meetup and the next they would consider actions. This could be contacting the group to see if there's issues, offering assistance, swag & 'meet up in a box' kits, and finally disbanding the group if there wasn't sufficient critical mass.

I took away a few good ideas from this talk, especially given recent conversations in Ubuntu about sparking up more meet-ups.

Phil Leggetter - Pusher - ROI on DevRel

Phil kicked off his short talk by talking about the ROI on DevRel by explaining Acquisition vs Activation. Where Acquisition of new developers might be them signing up for an account or downloading a product/sdk/library. Activation would be the conversion which might be measured differently per product. So perhaps "purchased paid API key" or "submitted app with N downloads".

Phil then moved on to talk a bit about how they can measure the effectiveness of online tutorials or blog articles by correlating sign ups with traffic coming from those online articles. There was some more discussion on this later on including the effectiveness of giving away vouchers/codes to incentivise downloads, with some disagreement on the results of doing so.

Afternoon Talks

Brandon West - SendGrid - Burnout

I've been to many talks and discussions about burnout in developer communities over the years. This talk from Brandon one was easily the most useful, factual and actionable one. I also enjoyed Brandon's attempts to inject Britishness into his talk which lightened the mood on a potentially very dark topic.

Brendon kicked off with a bit of a 'woe is me' #firstworldproblems introduction to his own current life issues. The usual things that affect a lot of people, but all happening at once, becoming overwhelming. We then moved on to defining burnout clearly, and what types of people are likely to suffer (clue: anyone) and some strategies for recognizing and preventing burnout.

A few key assertions / take-aways:-

"Burnout & depression are pathalogically indistinguishable" "Burnout and work engagement are not exclusive or correlatable" "Those most likely to burnout believe they are least at risk" "Learn a skill on holiday - the holiday will be more rewarding"

Tim Fogarty - Major League Hacking - Hackathons as a part of your DevRel strategy

Another great talk which built upon what Cristiano talked about earlier in the day - hackathons. Tim introduced different types of hackathons and which in his experience were more popular with developers and why.

Tim started by breaking down the types of hackathon - 'hacking', 'corporate' and 'civic' with the second being least popular as it's seen as free labour by developers, and so they're distrustful. He went on to reasons why people might run hackathons including evangelism, gathering (+ve and -ve) feedback, recruiting and mindshare (marketing).

He then moved on to strategies for making an impact, measuring the effect, sponsoring and how to craft the perfect demo to kick off the event.

Having never been to an in-person hackathon I found this another fascinating talk and will be following up with Tim Later.

Jessica Rose - Stop talking about diversity and just do it

Well. This was enlightening. This talk was excellent, and covered two main topics. First the focus was on getting a more diverse set of people running / attending / talking at your event. Some strategies were discussed and Jessica highlighted where many people go terribly wrong, assumptions people make and excuses people give.

The second part was a conversation about the ways in which an event can cater for as many people as possible. Here's a highlight of some of the ways we discussed, but this obviously doesn't cover everything:-

  • Attendees and speakers should be able to get in under their own power
  • Meal choices should be available - possibly beyond vegetarian/vegan
  • Code of Conduct
  • Sign language for talks
  • Well lit and safe feeling route from venue to accomodation
  • Space for breastfeeding / pumping, with snacks / drinks nearby
  • Non boozy spaces
  • Prayer room
  • After party with low noise level - and covered by Code of Conduct
  • Childcare
  • Professional chapparones (for under 18's)
  • Diversity tickets & travel grants
  • Scale inclusivity to budget (be realistic about what you can achieve)

Lots to think about!

Joe Nash - Braintree - Engaging Students

Joe kicked off his fast-paced talk with an introduction to things which influenced how he got where he is, including "Twilio Heroes". The talk was focussed on the UK University system, how to engage with students and some tips for running events which engage effectively with both CS and non-CS students.

James Milner - ersi UK - So you want to run a meet-up

James talked about his personal experience running GeoDev Meet-Ups. I found this information quite valuable as the subject is under discussion in Ubuntu. James gave some great tips for running good meet-ups, and had a number of things he's clearly learned the hard way. I hope to put some of his tips into action in the UK.

Dawn Foster - Lessons about community from science fiction.

This was a great uplifting talk to end the day. Dawn drew inspiration from her prolific science fiction reading to come up with some tips for people running community projects. I'll give you a flavour with a few of them. Each was accompanied by an appropriate picture.

Picture: Star Trek Red Shirt Lesson: "Participate and contribute in a way that people will notice and value your work"

Picture: Doctor Who TARDIS Lesson: "Communities look different from inside then when viewing as an outsider"

Picture: Enders Game Lesson: "Age is often unknown, encourage young people to contribute"

Dawn is a thoughtful, entertaining and engaging speaker. I'd certainly like to see more of her talks.

After Party

We all left the venue after the last talk and headed to a nearby trendy bar for a pint then headed home, pretty exhausted. A great event, I look forward to the next one.

Easily port mobile HTML5 games to Ubuntu Phone

Article also available in Spanish at http://thinkonbytes.blogspot.co.uk/2015/07/migrar-facilmente-juegos-moviles-en.html thanks to Marcos Costales.

I really like playing games on my phone & tablet and wanted some more games to play on Ubuntu. With a little work it turns out it's really pretty easy to 'port' games over to Ubuntu phone. I put the word 'port' in quotes simply because in some cases it's not a tremendous amount of effort, so calling it a 'port' might make people think it's more work than it is.

Update: A few people have asked why someone would want to even do this, and why not just bookmark a game in the browser. Sorry if that's not clear. With this method the game is entirely cached offline on the customer phone. Having fully offline games is desirable in many situations including when travelling or in a location with spotty Internet access. Not all games are fully offline of course, this method wouldn't help with a large on-line multi-player game like Clash of Clans for example. It would be great for many other titles though. This method also makes use of application confinement on Ubuntu so the app/game cannot access anything outside of the game data directory.

I worked with sturmflut from the Ubuntu Insiders on this over a few evenings and weekends. He wrote it up in his post Panda Madness.

We had some fun porting a few games and I wanted to share what we did so others can do the same. We created a simple template on github which can be used as a starting point, but I wanted to explain the process and the issues I had, so others can port apps/games.

If you have any questions feel free to leave me a comment, or if you'd rather talk privately you can get in contact in other ways.

Proof of concept

To prove that we could easily port existing games, we licensed a couple of games from Code Canyon. This is a marketplace where developers can license their games either for other developers to learn from, build upon or redistribute as-is. I started with a little game called Don't Crash which is an HTML5 game written using Construct 2. I could have licensed other games, and other marketplaces are also available, but this seemed like a good low-cost way for me to test out this process.

Screenshot from 2015-07-28 13-06-19

Side note: Construct 2 by Scirra is a popular, powerful, point-and-click Windows-only tool for developing cross-platform HTML5 apps and games. It's used by a lot of indie game developers to create games for desktop browsers and mobile devices alike. In development is Construct 3 which aims to be backwards compatible, and available on Linux too.

Before I licensed Don't Crash I checked it worked satisfactorily on Ubuntu phone using the live preview feature on Code Canyon. I was happy it worked, so I paid and received a download containing the 'source' Construct 2 files.

device-2015-07-28-130757

If you're a developer with your own game, then you can of course skip the above step, because you've already got the code to port.

Porting to Ubuntu

The absolute minimum needed to port a game is a few text files and the directory containing the game code. Sometimes a couple of tweaks are needed for things like permissions and lock rotation, but mostly it Just Works(TM).

I'm using an Ubuntu machine for all the packaging and testing, but in this instance I needed a Windows machine to export out the game runtime using Construct 2. Your requirements may vary, but for Ubuntu if you don't have one, you could install it in a VM like VMWare or VirtualBox, then add the SDK tools as detailed at developer.ubuntu.com.

This is the entire contents of the directory, with the game itself in the www/ folder.

alan@deep-thought:~/phablet/code/popey/licensed/html5_dontcrash⟫ ls -l
total 52
-rw-rw-r-- 1 alan alan   171 Jul 25 00:51 app.desktop
-rw-rw-r-- 1 alan alan   167 Jun  9 17:19 app.json
-rw-rw-r-- 1 alan alan 32826 May 19 19:01 icon.png
-rw-rw-r-- 1 alan alan   366 Jul 25 00:51 manifest.json
drwxrwxr-x 4 alan alan  4096 Jul 24 23:55 www

Creating the metadata

Manifest

This contains the basic details about your app like name, description, author, contact email and so on. Here's mine (called manifest.json) from the latest version of Don't Crash. Most of it should be fairly self-explanitory. You can simply replace each of the fields with your app details.

{
    "description":  "Don't Crash!",
    "framework":    "ubuntu-sdk-14.10-html",
    "hooks": {
        "dontcrash": {
            "apparmor": "app.json",
            "desktop":  "app.desktop"
        }
    },
    "maintainer":   "Alan Pope ",
    "name":         "dontcrash.popey",
    "title":        "Don't Crash!",
    "version":      "0.22"
}

Note: "popey" is my developer namespace in the store, you'll need to specify your namespace which you configure in your account page on the developer portal.

Screenshot from 2015-07-28 13-11-17

Security profile

Named app.json, this details what permissions my app needs in order to run:-

{
    "template": "ubuntu-webapp",
    "policy_groups": [
        "networking",
        "audio",
        "video",
        "webview"
    ],
    "policy_version": 1.2
}
Desktop file

This defines how the app is launched, what the icon filename is, and some other details:-

[Desktop Entry]
Name=Don't Crash
Comment=Avoid the other cars
Exec=webapp-container $@ www/index.html
Terminal=false
Type=Application
X-Ubuntu-Touch=true
Icon=./icon.png

Again, change the Name and Comment fields, and you're mostly done here.

Building a click package

With those files created, and an icon.png thrown in, I can now build my click package for uploading to the store. Here's that process in its entirety:-

alan@deep-thought:~/phablet/code/popey/licensed⟫ click build html5_dontcrash/
Now executing: click-review ./dontcrash.popey_0.22_all.click
./dontcrash.popey_0.22_all.click: pass
Successfully built package in './dontcrash.popey_0.22_all.click'.

Which on my laptop took about a second.

Note the "pass" is output from the click-review tool which sanity checks click packages immediately after building, to make sure there's no errors likely to cause it to be rejected from the store.

Testing on an Ubuntu device

Testing the click package on a device is pretty easy. It's just a case of copying the click package over from my Ubuntu machine via a USB cable using adb, then installing it.

adb push dontcrash.popey_0.22_all.click /tmp
adb shell
pkcon install-local --allow-untrusted /tmp/dontcrash.popey_0.22_all.click

Switch to the app scope and pull down to refresh, tap the icon and play the game.

device-2015-07-28-130907

Success! :)

device-2015-07-28-130522

Tweaking the app

At this point for some of the games I noticed some issues which I'll highlight here in case others also have them:-

Local loading of files

Construct 2 moans that "Exported games won't work until you upload them. (When running on the file:/// protocol, browsers block many features from working for security reasons." in a javascript popup and the game doesn't start. I just removed that chunk of js which does the check from the index.html and the game works fine in our browser.

Device orientation

With the most recent Over The Air (OTA) update of Ubuntu we enabled device orientation everywhere which means some games can rotate and become unplayable. We can lock games to be portrait or landscape in the desktop file (created above) by simply adding this line:-

X-Ubuntu-Supported-Orientations=portrait

Obviously changing "portrait" to "landscape" if your game is horizontally played. For Don't Crash I didn't do this because the developer had coded orientation detection in the game, and tells the player to rotate the device when it's the wrong way round.

Twitter links

Some games we ported have Twitter links in the game so players can tweet their score. Unfortunately the mobile web version of Twitter doesn't support intents so you can't have a link which contains the content "Check out my score in Don't Crash" embedded in it for example. So I just removed the Twitter links for now.

Cookies

Our browser doesn't support locally served cookies. Some games use this. For Heroine Dusk I ported from cookies to Local Storage which worked fine.

Uploading to the store

Uploading click packages to the Ubuntu store is fast and easy. Simply visit myapps.developer.ubuntu.com/dev/click-apps/, sign up/in, click "New Application" and follow the upload steps.

Screenshot from 2015-07-28 13-10-31

That's it! I look forward to seeing some more games in the store soon. Patches also welcome to the template on github.

Making a Portable Persistent Ubuntu USB Stick

I recently wanted to make a slightly modified persistent bootable USB stick running a recent version Ubuntu. I made some notes and have put them here in case they're useful to anyone else. It's a bit of a manual process which could probably be streamlined / automated. This was just what I did as a one-off, take from it what you will.

USB3 sticks in a USB3 port work best as USB2 can be a bit on the slow side, especially for IO intensive operations like package installation or compiling.

Note: A few people have pointed out the fragility and short lifespan of USB sticks. This same procedure can be used to install on a hard disk or SSD in a USB enclosure. Once the image is copied to the external storage, simply use gparted to resize it up to take all available space. The goal I had was to make an image which can be copied to USB stick to provide a persistent bootable Ubuntu SDK development environment. This could be useful for people who don't run Ubuntu as their primary OS (Yes, these people exist, I know right!?) but want to dabble in Ubuntu application development. It's also handy if you're running an App Dev School where the computers aren't yours, or run some other OS. The students could potentially take the sticks away with the full OS and all their work on. Just make the image and then copy it to multiple sticks before the class starts.

I also wanted to make it ask for locale and user details on first boot, so it could be easily configured and used in any language. This is pretty easy given the Ubuntu installer has all of that built in.

I used Ubuntu 15.04 i386 (but also tried with Ubuntu 14.04.1 LTS) and an 8GB USB stick which leaves a couple of GB over for work. Obviously a larger stick gives more space to the user. It turned out though that using an 8GB USB stick was a bit tight for SDK work. I ended up with 76MB left after creating one 15.04 armhf kit. Maybe 8GB is good for desktop and qml/html5 only development (although still a bit tight), but not for cross architecture or other binary builds. 16GB would have enough room for multiple kits and could build binaries for devices.

Some of these steps can be done in the background while you do other things. It's not a massively time consuming task if you have a decent connection and fast USB stick / hard disk, but as I mentioned, is a bit manual.

The result is a USB stick which you can boot from and work off with data saved to the stick. You can optionally enable home directory encryption during the final end-user setup if that's important to you.

Step 1 - Prep

Have an 8GB (or larger) USB 3 stick handy. I am using Kingston 8 GB USB 3.0 DataTraveler G4 Flash Drive and later Kingston Technology 16GB Data Traveler G4 USB 3.0 Flash Drive. Faster sticks are available of course, but I wanted something cheap to prototype on. Have a laptop with a USB 3 port (or ports) and supports kvm. I did all this on my Ubuntu Vivid Vervet (15.04) Thinkpad X220 laptop which has a single USB3 port. Make a directory on a local disk to store scratch image - will need 16GB or more space Install qemu-kvm and gddrescue on host Download ubuntu-15.04-desktop-i386.iso from http://releases.ubuntu.com/15.04/. (torrent link).

Step 2 - Installation of base system

Make a blank image on local disk

 dd if=/dev/zero of=./disk_image bs=1M count=7500

This should result in a file a bit under 8GB. e.g.
alan@deep-thought:/data/usb⟫ dd if=/dev/zero of=./disk_image bs=1048576 count=7500
7500+0 records in
7500+0 records out
7864320000 bytes (7.9 GB) copied, 34.468 s, 228 MB/s

Install Ubuntu into the image using kvm

sudo kvm -m 2048 -cdrom ~/Downloads/ubuntu-15.04-desktop-i386.iso -hda ./disk_image -boot d
This should boot off the ISO

QEMU_425

At the A11Y (person = keyboard) icon, hit space

QEMU_426

At the boot menu, choose language (this is just language for the installer, user will later choose which language to use)

QEMU_427

Press F3 and choose keyboard layout

QEMU_428

Press F4 and choose OEM install

QEMU_429

Pick "Install" from the menu.

QEMU_430

QEMU_431

Follow the installer prompts as normal. I configured with no swap, but use the entire disk for an ext4 volume for the root filesystem. Set a password for the oem user, which will be thrown away later, and the user will get to set their own password. Shut-down at the end

Step 3 - Install the SDK

This is the part where you make the modifications to the image (if any). I wanted to install the Ubuntu SDK.

Optionally at this point, make a backup of your cleanly installed Ubuntu 15.04.1 system cp ./disk_image ./ubuntu_15.04_install_backup

Boot the previously created install (note the additional options - these are handy) sudo kvm -m 2048 -hda ./disk_image -chardev stdio,id=mon -mon mon Once booted to the desktop, in the terminal on the host at the (qemu) prompt type this to switch the VM to the console (which is faster to do stuff than the GUI :) ):- (qemu) sendkey ctrl-alt-f1 Login to the tty with the oem user/password set in Step 2. Follow the usual guide to install the SDK and update the system:- sudo add-apt-repository ppa:ubuntu-sdk-team/ppa sudo apt-get update sudo apt-get install ubuntu-sdk sudo apt-get dist-upgrade sudo apt-get clean sudo apt-get autoremove Shut down the vm sudo shutdown -h now

Optionally at this point, make a backup of your "SDK-installed" (or modified in whatever way you choose) OEM mode Ubuntu 15.04 system cp ./disk_image ./ubuntu_15.04_install_sdk_oem_backup

Note: At this point you can boot the disk image and do further customisation - maybe adding other packages which may be of use, but I stopped here.

Step 4 - Prepare the OEM image for 'shipping'

This is the point where we flip the switch in the installed image before handing it off to another user. On first boot they will get prompted to set locale and configure a new user.

Boot the previously created install which has the SDK installed sudo kvm -m 2048 -hda ./disk_image Click the "Prepare for shipping to end user" icon on the desktop - this sets the system to be ready for the first-boot experience for a new user Shut down the system

Step 5 - Test this all worked

Make a copy of the master image for testing cp ./disk_image ./testing_oem_install Boot the test image to try it out sudo kvm -m 2048 -hda ./testing_oem_install At this point you should be prompted for the usual post-install setup tasks including language / locale / username & password. Setup as you would a normal machine Open the SDK (or whatever you installed), test it all works I tried creating a kit and do other SDK related things Shutdown when done Delete the test image rm ./testing_oem_install

Step 6 - Copy the OEM image to a USB stick for shipping / use

Now we have a 'final' image (and optionally some backups) we can copy this to a stick for use by us / someone else. We can of course make more than one by doing this step multiple times with different sticks. On my system as you can see it took ~30 mins to copy the image to the stick. Faster, more expensive sticks may be better, these were pretty cheap.

Copy the disk image to an appropriately sized USB stick sudo ddrescue -d -D --force ./disk_image /dev/sdX

e.g.

 alan@deep-thought:/data/usb⟫ time sudo ddrescue -D -d --force disk_image /dev/sdc
GNU ddrescue 1.19
Press Ctrl-C to interrupt
rescued:     7864 MB,  errsize:       0 B,  current rate:    1966 kB/s
   ipos:     7864 MB,   errors:       0,    average rate:    4884 kB/s
   opos:     7864 MB, run time:   26.83 m,  successful read:       0 s ago
Finished

real    26m51.682s
user    0m1.212s
sys     0m30.084s

Step 7 - Test & use the stick

Put the USB stick in a computer set to boot from external media. Test that you get a desktop and the usual OEM prompts you got in Step 5. If that works then you can do step 5 again for the same stick or as many sticks as you have.

Success!

Comments and suggestions welcome!