Testing Ubuntu Apps As A Service

tl;dr. Stuart Langridge and I made an simple, easy to use, experimental app tester called Marvin, for Ubuntu Click Packages, which emails you screenshots and logs of your app while running on a real device you may not own.

Screenshot of email from Marvin

I frequently get asked by new developers in the community to help test their apps on Ubuntu Phone. Typically they don’t want extensive testing of all features, just a simple “does it start and what does it look like on the device you have?”.

Often they don’t have a physical device when developing on the desktop with our SDK, but want an on-device sanity check before they upload to the store. Sometimes they have one device such as a phone, but want to see what their app looks like on a different one, perhaps a tablet.

I’ve been happy to help developers test their apps on various devices, but this doesn’t scale well, is time consuming and relies on me being online and having a phone which I’m happy to install random click packages on.

Meanwhile, at OggCamp I gave a short talk about our recent security incident on Ubuntu phone. During the Q&A and in the bar afterwards a couple of people suggested that we should have some system which enables automated testing of devices. They were coming at it from the security point of view, suggesting heavy instrumentation to find these kinds of issues before they hit the store.

While we (Canonical) already have tools which review apps before they go in the store, we currently don’t actually install and execute the apps on devices, and have no plan to implement such a service (that I know of).

I’m aware that other platforms have implemented automated systems for testing and instrumenting apps and wondered how hard it would be to setup something really basic to cover at least one of the two use cases above. So I took to Telegram to brainstorm with my good friend Stuart Langridge.

Snippet of conversation with Stuart

We thrashed out what was needed for a ‘minimum viable product’ and some nice-to-have future enhancements. Pretty soon after, with a bit of python and some hacked-together shell scripts, ‘Marvin‘ was born. I then approached Daniel McGuire who kindly provided some CSS to make it look prettier.

Screenshot of upload page

A developer can upload a click package to the site, and specify their email address & one or more of the available devices. Some time later they’ll get an email showing a few screenshots of the app or scope running on a device and pertinent logs extracted after it ran. While the developer waits, the website shows the current status as ‘pending’ (you’re in a queue), ‘claimed’ (by a device) and ‘finished’ (check your inbox).

This fulfills the simplest of use cases, making sure the app starts, and extracting the log if it didn’t. Clearly there’s plenty more it could potentially do, but this was our first target met.

Under the covers, there’s a device attached to a computer which checks periodically for uploaded clicks and processes them in sequence. In between each run the phone is cleaned up, so each test is done on a blank device. Currently it tests traditional apps/games and scopes, webapps are rejected, but may be supported later.

The reason we reject webapps is because currently the devices have no network access at all – no wifi or cellular data. So running webapps would just result in this:-

Screenshot of webapp with no network access

It’s experimental so not completely robust, being a prototype hacked together over a couple of weekends/evenings, but it works (for the most part). There’s no guarantees of availability of the service or indeed the devices. It could go offline at any time. Did I mention it’s experimental?

Significantly, I’ve disabled network access completely on the device, with no SIM inside, so any app which requires external network access is going to have a bad day. Locally installed apps however, will work fine.

We currently don’t do any interaction with the uploaded applications, but simply run them and wait a few seconds (to give it time to quiesce) then take a screenshot. The image at the top of this post shows what a typical email from Marvin looks like.

It contains:-

  • click-review.txt – The output from running the click review tools
    • Note: Apps which fail the click review process (the same one run by the click store) will not be installed or tested.
  • install.txt – Output from the commands used to install the click on the device – good for debugging install failures
  • Screenshot-0 – What the “home” app scope looks like with the click installed – useful for showing the icon and description
  • Screenshot-1 – What happens immediately after starting the app, showing the splash screen
  • Screenshot-2 – The app after 5 seconds
  • Screenshot-3 – The app after 10 seconds
    • Note: We attempt to de-duplicate the screenshots so you may not get all four if any are identical
  • application-log.txt – The actual output (stdout) from the application, pulled from ~/.cache/upstart
  • dmesg.txt – Any kernel logging generated from the app during the app run
  • device-version.txt – The output of ‘system-image-cli –info’ run on the device, so the developer knows what OTA level, channel and device it ran on

There’s clearly a ton of other things that could be added to the mail, or extra items which could be instrumented or monitored, and features we could add. Off the top of my head we could potentially add:-

  • Scripted touch/gestures
  • Networking
  • VPN endpoints (so the phone looks like it’s in a particular region)
  • Orientation changes
  • Faked GPS location
  • Video/screencast recording during runtime
  • Input from microphone / camera(s)
  • Specify which release / channel to flash on the device prior to testing

Clearly all of these need some careful thought and planning, especially those enabling network access from the device.

We’re interested in feedback from developers who might use Marvin, and suggestions for improvements we might make. There are a limited number of devices in the pool, and not all supported devices are currently available. In the future we may have more devices connected to Marvin as they become available.

So go and test your apps at marvin.popey.com!

Troubleshooting as a Choose Your Own Adventure

READ MAP

We have a lot of documentation and help in the Ubuntu project, and much of it is quite hostile to new users. We have IRC channels, mailing lists, dense & out of date wiki pages, lengthy and hard to consume forum posts & lengthy out of date tutorial videos. We also have some more modern tools such as AskUbuntu and Discourse.

Most are good for asking one specific question, but most aren’t well suited to guiding a user through a specific problem diagnosis. If you know what questions to ask, then a search engine might find part of the problem, or hopefully part of the solution.

However one aspect we don’t cover very well is guided self-support. This is very apparent in distribution upgrades. People often give up completely when an upgrade breaks down, rather than work through the problem as they would with anything else. There can be many reasons for this of course, but from what I’ve seen in the community, fixing a broken upgraded system is hard, and made harder when you only have a black screen, or tty to work with when you’re not an expert.

GET ROPE AND SWORD

I’m thinking very specifically about a target audience of non-technical users. Someone who uses Ubuntu but wants to feel empowered to fix a problem themselves, and not have to call their wife or daughter or other ‘expert’ to fix it. I want someone to find a guide which gets them out of the upgrade issue. Obviously I’d like us to fix the problem which cause the issues, but I want to start here, because frankly it’s easier for me.

WAIT

While I can hear some of my co-workers shouting “But popey, Snappy Core fixes the issue of broken updates!”, no it doesn’t, not today, and not while there are people running the traditional Debian based desktop – which I suspect will be for a good few years yet.

WAIT

I can also hear those of you saying “I never do upgrades, I always clean install”, and I’m happy for you, but we have an upgrade system, people should be confident that it works, and when it doesn’t it should be fixable without going nuclear. Just the same as “Buy a new TV” isn’t usually the solution to “My TV Remote stopped working”.

EAT LUNCH

In my mind a user might be more inclined to fix their system if there’s an easy to use ‘expert system’ which they can walk through to get them working again. I wondered about this some time back and considered the idea of a Choose Your Own Adventure style of troubleshooting issues, rather than just punting people to IRC when their system breaks. We have been helping people with broken upgrades for years, surely we can amass the technical knowledge into a self-service system to help future users.

One assumption I am making with this is that the person going through this is able to access the Internet (and thus this guide) via another machine or their phone. This seems semi-reasonable as we often get people in IRC or on other support resources asking for help with a broken upgrade. So the guide should work well on any browser and ideally on mobile too.

There are of course unfortunate people for which they have only one computer and no smartphone so have no way to access this resource easily. This system doesn’t cater for them well.

GET BARREL

I made a little prototype using Twine which is an “an open-source tool for telling interactive, nonlinear stories.” – typically text adventure or “Interactive Fiction” games, but can be used for other things too. I chose Twine because it’s open source, easy to use, cross platform and simple. You can even create Twine ‘stories’ directly in your browser. Neat!

The output generated by Twine is HTML and can be customised and styled with a stylesheet. I did one simple bit of styling in that I used the Ubuntu font :).

THROW BARREL THROUGH TRAP DOOR

Here’s what the map of the ‘world’ looks like inside Twine (after I carefully moved the blocks around so none criss-cross):-

Twine Screenshot

Editing the pages within the ‘story’ is super easy, linking to other pages is as simple as typing the name of a new page in square brackets:-

Editing a page

You can try out the very early prototype I made at http://popey.com/~alan/twines/UbuntuUpgrades.html. Obviously this isn’t finished, won’t actually help anyone fix their system, and is hosted somewhere obscure, all “TODO” items.

OPEN CURTAIN

There’s a few known issues with the above prototype twine:-

  • Browser back button exits the story, it should go back a step
  • Twine back button is hard to see, some CSS will fix that
  • Many of the pages are quite wordy, or technical, probably could be simplified

SAY TO BARD “SHOOT THE DRAGON”

I wanted to share what I’d done to see if it seemed like a good idea, and whether people might be interested in helping create and curate content for it. The idea is to make a prototype and get some content rather than make the thing look especially pretty right now. The way I see it there’s some important steps to do:-

  • Confirm this isn’t a stupid idea (this blog post)
  • Figure out the best way to distribute this and make it accessible
  • Find people interested in helping
  • Identify a bunch of specific breakages that happen in Ubuntu upgrades
  • Craft diagnostic steps to identify one breakage scenario from another
  • Come up with robust solutions for each scenario
  • Test the scenarios
  • Publish and publicise the work

Things I haven’t yet considered, but probably should:-

  • Translations
  • Accessibility
  • Co-ordinating work
  • How far back I’ll roll my eyes at people telling me rolling distributions are better

PUT TREASURE IN CHEST

Comments, suggestions or flames are welcome here or in my inbox.

DevRelCon 2015 Trip Report

Huh, this turned out to be longer than I expected. Don’t feel obliged to read it, it’s more notes for myself, and to remind me of why I liked the event.

Event

On Wednesday I went to DevRelCon in London. DevRelCon is “a one day single track conference for technical evangelists, developer advocates and anyone interested in developer relations” setup by Matthew Revell. I don’t think there’s a lot of difference between my role (defined as Community Manager) at Canonical and Developer Relations so figured it would probably have appropriate content for my role. Boy was I right!

DevRelCon was easily the single most valuable short conference I’ve ever attended. The speakers were knowledgeable, friendly and accessible, and easy to understand. I took a ton of notes, and will distil some of them down here, but will almost certainly keep referring back to them over the coming months as I look to implement some of the suggestions I heard.

Venue

The event took place at The Trampery Old Street, in Shoreditch, the trendy/hipster part of London. We had access to a bright and airy ‘ballroom’ and were served with regular drinks, snacks and a light lunch. Free WiFi was also available, which worked well, but I didn’t use it much as we had little time away from the talks.

Structure

The day consisted of a mix of long (40 minute) talks, some shorter (20 min) ones, and a few ‘lightning’ talks. Having a mix of durations worked well I think. We started a little late, but Matthew massaged the timetable to claw back some time, and as it was a single track day there was no real issue if things didn’t run to time, as you weren’t likely to run off to another talk, and miss something.

All the talks were great, but I took considerably more notes in some than others, so this is represented below in that I haven’t listed every talk.

Morning Talks

Rob SpectreTwilio – Scaling Developer Evangelism.

This started off well as Rob plugged in his laptop and we were greeted with an Ubuntu desktop! He started off detailing some interesting stats to focus our minds on who we’re evangelising to. Starting with the 18.2m developers worldwide, given ~3Bn smartphone users, and ~4Bn Internet users that means ~0.08% have the capability to write code. There’s a 6% year on year increase in developers, mostly in developing nations, the ratio is less in the western world. So for example India could overtake every other countries’ developer count by ~2017.

Rob talked at length about the structure of Developer Evangelists, Developer Educators and the Community Team at Twilio. The talk continued to outline how valuable developers are, how at Twilio their Developer Evangelists are the ‘Red Carpet’ to their community. I was struck by how very differently we (Open Source projects) and Ubuntu specifically treat contributors to the project.

There was also a section on running developer events, and Rob spent some time talking about strategies for successful events, and how those can feed back to improve your product. He also talked a little about measurement, which was also going to be covered in later talks that day.

Another useful anecdote Rob detailed was regarding conversion of talks into blog posts. While a talk at an event can catalyse the 20-100 people in the room, converting that into a detailed tutorial blog post can bring in hundreds or thousands more.

The final slide in Rob’s talk was “Would you recommend this talk?” with a phone number attendees could send a score to. I thought this was a particularly cunning strategy. There was also talk of using the external IP address of the venue WiFi as one factor to determine the effectiveness / conversion rate of attendees.

Cristiano BettaBraintree – Tooling your way to a great devrel team

Cristiano started off talking about BattleHack which I’d not heard of. These are in person events where teams of developers get 24 hours to work on a project fuelled by coffee, cake and Red Bull to be in with a chance of winning a cash prize and an amusing axe.

He then went on to talk about a personal project to manage event sign-ups. This replaces tools like Eventbrite and MailChimp and enables Cristiano to get a better handle on the success of his events.

Laura CowenIBM – Building a developer community in an enterprise world

Laura started off giving some history of the products and groups inside IBM who are responsible for WAS, the public facing developer sites and the struggles she’s had updating them

The interesting parts for me came when Laura was detailing the pain she had getting developer time to update documentation and engage with users and communities outside their own four walls. Laura also talked about the difficulty when interfacing developers and marketing, their differing goals and some strategies for coping.

I recognised for example the frustration in people wanting to publish everything on a developer site, whether it’s appropriate to the target audience or not. Sometimes we (in Ubuntu) fail to deeply consider the target audience before we publish articles, guides or documentation. I think we can do better here. Pushing back on content creators, and finding the right place for a published article is worth it, if the target audience is to be defended.

Lightning Talks

Shaunak Kashyapelastic – Getting the measure of DevRel

In this short talk Shaunak gave some interesting snippets on how elastic measure community engagement. I found a couple interesting which I felt we might use in Ubuntu. Measuring “time to first response” for questions and issues by looking for responses from someone other than the first poster. While I don’t think they were actively using this data yet, getting an initial base line would be useful.

Shaunak also detailed one factor in measuring meet-up effectiveness. Typically elastic have 3-4 meet-ups a week, globally. For each meet-up group they measured “time since last meetup”. For those where there was a long delta between one meetup and the next they would consider actions. This could be contacting the group to see if there’s issues, offering assistance, swag & ‘meet up in a box’ kits, and finally disbanding the group if there wasn’t sufficient critical mass.

I took away a few good ideas from this talk, especially given recent conversations in Ubuntu about sparking up more meet-ups.

Phil LeggetterPusher – ROI on DevRel

Phil kicked off his short talk by talking about the ROI on DevRel by explaining Acquisition vs Activation. Where Acquisition of new developers might be them signing up for an account or downloading a product/sdk/library. Activation would be the conversion which might be measured differently per product. So perhaps “purchased paid API key” or “submitted app with N downloads”.

Phil then moved on to talk a bit about how they can measure the effectiveness of online tutorials or blog articles by correlating sign ups with traffic coming from those online articles. There was some more discussion on this later on including the effectiveness of giving away vouchers/codes to incentivise downloads, with some disagreement on the results of doing so.

Afternoon Talks

Brandon WestSendGrid – Burnout

I’ve been to many talks and discussions about burnout in developer communities over the years. This talk from Brandon one was easily the most useful, factual and actionable one. I also enjoyed Brandon’s attempts to inject Britishness into his talk which lightened the mood on a potentially very dark topic.

Brendon kicked off with a bit of a ‘woe is me’ #firstworldproblems introduction to his own current life issues. The usual things that affect a lot of people, but all happening at once, becoming overwhelming. We then moved on to defining burnout clearly, and what types of people are likely to suffer (clue: anyone) and some strategies for recognizing and preventing burnout.

A few key assertions / take-aways:-

“Burnout & depression are pathalogically indistinguishable”

“Burnout and work engagement are not exclusive or correlatable”

“Those most likely to burnout believe they are least at risk”

“Learn a skill on holiday – the holiday will be more rewarding”

Tim FogartyMajor League Hacking – Hackathons as a part of your DevRel strategy

Another great talk which built upon what Cristiano talked about earlier in the day – hackathons. Tim introduced different types of hackathons and which in his experience were more popular with developers and why.

Tim started by breaking down the types of hackathon – ‘hacking’, ‘corporate’ and ‘civic’ with the second being least popular as it’s seen as free labour by developers, and so they’re distrustful. He went on to reasons why people might run hackathons including evangelism, gathering (+ve and -ve) feedback, recruiting and mindshare (marketing).

He then moved on to strategies for making an impact, measuring the effect, sponsoring and how to craft the perfect demo to kick off the event.

Having never been to an in-person hackathon I found this another fascinating talk and will be following up with Tim Later.

Jessica Rose – Stop talking about diversity and just do it

Well. This was enlightening. This talk was excellent, and covered two main topics. First the focus was on getting a more diverse set of people running / attending / talking at your event. Some strategies were discussed and Jessica highlighted where many people go terribly wrong, assumptions people make and excuses people give.

The second part was a conversation about the ways in which an event can cater for as many people as possible. Here’s a highlight of some of the ways we discussed, but this obviously doesn’t cover everything:-

  • Attendees and speakers should be able to get in under their own power
  • Meal choices should be available – possibly beyond vegetarian/vegan
  • Code of Conduct
  • Sign language for talks
  • Well lit and safe feeling route from venue to accomodation
  • Space for breastfeeding / pumping, with snacks / drinks nearby
  • Non boozy spaces
  • Prayer room
  • After party with low noise level – and covered by Code of Conduct
  • Childcare
  • Professional chapparones (for under 18’s)
  • Diversity tickets & travel grants
  • Scale inclusivity to budget (be realistic about what you can achieve)

Lots to think about!

Joe NashBraintree – Engaging Students

Joe kicked off his fast-paced talk with an introduction to things which influenced how he got where he is, including “Twilio Heroes”. The talk was focussed on the UK University system, how to engage with students and some tips for running events which engage effectively with both CS and non-CS students.

James Milnerersi UK – So you want to run a meet-up

James talked about his personal experience running GeoDev Meet-Ups. I found this information quite valuable as the subject is under discussion in Ubuntu. James gave some great tips for running good meet-ups, and had a number of things he’s clearly learned the hard way. I hope to put some of his tips into action in the UK.

Dawn FosterLessons about community from science fiction.

This was a great uplifting talk to end the day. Dawn drew inspiration from her prolific science fiction reading to come up with some tips for people running community projects. I’ll give you a flavour with a few of them. Each was accompanied by an appropriate picture.

Picture: Star Trek Red Shirt
Lesson: “Participate and contribute in a way that people will notice and value your work”

Picture: Doctor Who TARDIS
Lesson: “Communities look different from inside then when viewing as an outsider”

Picture: Enders Game
Lesson: “Age is often unknown, encourage young people to contribute”

Dawn is a thoughtful, entertaining and engaging speaker. I’d certainly like to see more of her talks.

After Party

We all left the venue after the last talk and headed to a nearby trendy bar for a pint then headed home, pretty exhausted. A great event, I look forward to the next one.

Easily port mobile HTML5 games to Ubuntu Phone

Article also available in Spanish at http://thinkonbytes.blogspot.co.uk/2015/07/migrar-facilmente-juegos-moviles-en.html thanks to Marcos Costales.

I really like playing games on my phone & tablet and wanted some more games to play on Ubuntu. With a little work it turns out it’s really pretty easy to ‘port’ games over to Ubuntu phone. I put the word ‘port’ in quotes simply because in some cases it’s not a tremendous amount of effort, so calling it a ‘port’ might make people think it’s more work than it is.

Update: A few people have asked why someone would want to even do this, and why not just bookmark a game in the browser. Sorry if that’s not clear. With this method the game is entirely cached offline on the customer phone. Having fully offline games is desirable in many situations including when travelling or in a location with spotty Internet access. Not all games are fully offline of course, this method wouldn’t help with a large on-line multi-player game like Clash of Clans for example. It would be great for many other titles though. This method also makes use of application confinement on Ubuntu so the app/game cannot access anything outside of the game data directory.

I worked with sturmflut from the Ubuntu Insiders on this over a few evenings and weekends. He wrote it up in his post Panda Madness.

We had some fun porting a few games and I wanted to share what we did so others can do the same. We created a simple template on github which can be used as a starting point, but I wanted to explain the process and the issues I had, so others can port apps/games.

If you have any questions feel free to leave me a comment, or if you’d rather talk privately you can get in contact in other ways.

Proof of concept

To prove that we could easily port existing games, we licensed a couple of games from Code Canyon. This is a marketplace where developers can license their games either for other developers to learn from, build upon or redistribute as-is. I started with a little game called Don’t Crash which is an HTML5 game written using Construct 2. I could have licensed other games, and other marketplaces are also available, but this seemed like a good low-cost way for me to test out this process.

Screenshot from 2015-07-28 13-06-19

Side note: Construct 2 by Scirra is a popular, powerful, point-and-click Windows-only tool for developing cross-platform HTML5 apps and games. It’s used by a lot of indie game developers to create games for desktop browsers and mobile devices alike. In development is Construct 3 which aims to be backwards compatible, and available on Linux too.

Before I licensed Don’t Crash I checked it worked satisfactorily on Ubuntu phone using the live preview feature on Code Canyon. I was happy it worked, so I paid and received a download containing the ‘source’ Construct 2 files.

device-2015-07-28-130757

If you’re a developer with your own game, then you can of course skip the above step, because you’ve already got the code to port.

Porting to Ubuntu

The absolute minimum needed to port a game is a few text files and the directory containing the game code. Sometimes a couple of tweaks are needed for things like permissions and lock rotation, but mostly it Just Works(TM).

I’m using an Ubuntu machine for all the packaging and testing, but in this instance I needed a Windows machine to export out the game runtime using Construct 2. Your requirements may vary, but for Ubuntu if you don’t have one, you could install it in a VM like VMWare or VirtualBox, then add the SDK tools as detailed at developer.ubuntu.com.

This is the entire contents of the directory, with the game itself in the www/ folder.

alan@deep-thought:~/phablet/code/popey/licensed/html5_dontcrash⟫ ls -l
total 52
-rw-rw-r-- 1 alan alan   171 Jul 25 00:51 app.desktop
-rw-rw-r-- 1 alan alan   167 Jun  9 17:19 app.json
-rw-rw-r-- 1 alan alan 32826 May 19 19:01 icon.png
-rw-rw-r-- 1 alan alan   366 Jul 25 00:51 manifest.json
drwxrwxr-x 4 alan alan  4096 Jul 24 23:55 www

Creating the metadata

Manifest

This contains the basic details about your app like name, description, author, contact email and so on. Here’s mine (called manifest.json) from the latest version of Don’t Crash. Most of it should be fairly self-explanitory. You can simply replace each of the fields with your app details.

{
    "description":  "Don't Crash!",
    "framework":    "ubuntu-sdk-14.10-html",
    "hooks": {
        "dontcrash": {
            "apparmor": "app.json",
            "desktop":  "app.desktop"
        }
    },
    "maintainer":   "Alan Pope ",
    "name":         "dontcrash.popey",
    "title":        "Don't Crash!",
    "version":      "0.22"
}

Note: “popey” is my developer namespace in the store, you’ll need to specify your namespace which you configure in your account page on the developer portal.

Screenshot from 2015-07-28 13-11-17

Security profile

Named app.json, this details what permissions my app needs in order to run:-

{
    "template": "ubuntu-webapp",
    "policy_groups": [
        "networking",
        "audio",
        "video",
        "webview"
    ],
    "policy_version": 1.2
}

Desktop file

This defines how the app is launched, what the icon filename is, and some other details:-

[Desktop Entry]
Name=Don't Crash
Comment=Avoid the other cars
Exec=webapp-container $@ www/index.html
Terminal=false
Type=Application
X-Ubuntu-Touch=true
Icon=./icon.png

Again, change the Name and Comment fields, and you’re mostly done here.

Building a click package

With those files created, and an icon.png thrown in, I can now build my click package for uploading to the store. Here’s that process in its entirety:-

alan@deep-thought:~/phablet/code/popey/licensed⟫ click build html5_dontcrash/
Now executing: click-review ./dontcrash.popey_0.22_all.click
./dontcrash.popey_0.22_all.click: pass
Successfully built package in './dontcrash.popey_0.22_all.click'.

Which on my laptop took about a second.

Note the “pass” is output from the click-review tool which sanity checks click packages immediately after building, to make sure there’s no errors likely to cause it to be rejected from the store.

Testing on an Ubuntu device

Testing the click package on a device is pretty easy. It’s just a case of copying the click package over from my Ubuntu machine via a USB cable using adb, then installing it.

adb push dontcrash.popey_0.22_all.click /tmp
adb shell
pkcon install-local --allow-untrusted /tmp/dontcrash.popey_0.22_all.click

Switch to the app scope and pull down to refresh, tap the icon and play the game.

device-2015-07-28-130907

Success! :)

device-2015-07-28-130522

Tweaking the app

At this point for some of the games I noticed some issues which I’ll highlight here in case others also have them:-

Local loading of files

Construct 2 moans that “Exported games won’t work until you upload them. (When running on the file:/// protocol, browsers block many features from working for security reasons.” in a javascript popup and the game doesn’t start. I just removed that chunk of js which does the check from the index.html and the game works fine in our browser.

Device orientation

With the most recent Over The Air (OTA) update of Ubuntu we enabled device orientation everywhere which means some games can rotate and become unplayable. We can lock games to be portrait or landscape in the desktop file (created above) by simply adding this line:-

X-Ubuntu-Supported-Orientations=portrait

Obviously changing “portrait” to “landscape” if your game is horizontally played. For Don’t Crash I didn’t do this because the developer had coded orientation detection in the game, and tells the player to rotate the device when it’s the wrong way round.

Twitter links

Some games we ported have Twitter links in the game so players can tweet their score. Unfortunately the mobile web version of Twitter doesn’t support intents so you can’t have a link which contains the content “Check out my score in Don’t Crash” embedded in it for example. So I just removed the Twitter links for now.

Cookies

Our browser doesn’t support locally served cookies. Some games use this. For Heroine Dusk I ported from cookies to Local Storage which worked fine.

Uploading to the store

Uploading click packages to the Ubuntu store is fast and easy. Simply visit myapps.developer.ubuntu.com/dev/click-apps/, sign up/in, click “New Application” and follow the upload steps.

Screenshot from 2015-07-28 13-10-31

That’s it! I look forward to seeing some more games in the store soon. Patches also welcome to the template on github.

Making a Portable Persistent Ubuntu USB Stick

I recently wanted to make a slightly modified persistent bootable USB stick running a recent version Ubuntu. I made some notes and have put them here in case they’re useful to anyone else. It’s a bit of a manual process which could probably be streamlined / automated. This was just what I did as a one-off, take from it what you will.

USB3 sticks in a USB3 port work best as USB2 can be a bit on the slow side, especially for IO intensive operations like package installation or compiling.

Note: A few people have pointed out the fragility and short lifespan of USB sticks. This same procedure can be used to install on a hard disk or SSD in a USB enclosure. Once the image is copied to the external storage, simply use gparted to resize it up to take all available space.

The goal I had was to make an image which can be copied to USB stick to provide a persistent bootable Ubuntu SDK development environment. This could be useful for people who don’t run Ubuntu as their primary OS (Yes, these people exist, I know right!?) but want to dabble in Ubuntu application development. It’s also handy if you’re running an App Dev School where the computers aren’t yours, or run some other OS. The students could potentially take the sticks away with the full OS and all their work on. Just make the image and then copy it to multiple sticks before the class starts.

I also wanted to make it ask for locale and user details on first boot, so it could be easily configured and used in any language. This is pretty easy given the Ubuntu installer has all of that built in.

I used Ubuntu 15.04 i386 (but also tried with Ubuntu 14.04.1 LTS) and an 8GB USB stick which leaves a couple of GB over for work. Obviously a larger stick gives more space to the user. It turned out though that using an 8GB USB stick was a bit tight for SDK work. I ended up with 76MB left after creating one 15.04 armhf kit. Maybe 8GB is good for desktop and qml/html5 only development (although still a bit tight), but not for cross architecture or other binary builds. 16GB would have enough room for multiple kits and could build binaries for devices.

Some of these steps can be done in the background while you do other things. It’s not a massively time consuming task if you have a decent connection and fast USB stick / hard disk, but as I mentioned, is a bit manual.

The result is a USB stick which you can boot from and work off with data saved to the stick. You can optionally enable home directory encryption during the final end-user setup if that’s important to you.

Step 1 – Prep

Have an 8GB (or larger) USB 3 stick handy. I am using Kingston 8 GB USB 3.0 DataTraveler G4 Flash Drive and later Kingston Technology 16GB Data Traveler G4 USB 3.0 Flash Drive. Faster sticks are available of course, but I wanted something cheap to prototype on.
Have a laptop with a USB 3 port (or ports) and supports kvm. I did all this on my Ubuntu Vivid Vervet (15.04) Thinkpad X220 laptop which has a single USB3 port.
Make a directory on a local disk to store scratch image – will need 16GB or more space
Install qemu-kvm and gddrescue on host
Download ubuntu-15.04-desktop-i386.iso from http://releases.ubuntu.com/15.04/. (torrent link).

Step 2 – Installation of base system

Make a blank image on local disk
dd if=/dev/zero of=./disk_image bs=1M count=7500
This should result in a file a bit under 8GB.
e.g.

alan@deep-thought:/data/usb⟫ dd if=/dev/zero of=./disk_image bs=1048576 count=7500
7500+0 records in
7500+0 records out
7864320000 bytes (7.9 GB) copied, 34.468 s, 228 MB/s

Install Ubuntu into the image using kvm
sudo kvm -m 2048 -cdrom ~/Downloads/ubuntu-15.04-desktop-i386.iso -hda ./disk_image -boot d
This should boot off the ISO

QEMU_425

At the A11Y (person = keyboard) icon, hit space

QEMU_426

At the boot menu, choose language (this is just language for the installer, user will later choose which language to use)

QEMU_427

Press F3 and choose keyboard layout

QEMU_428

Press F4 and choose OEM install

QEMU_429

Pick “Install” from the menu.

QEMU_430

QEMU_431

Follow the installer prompts as normal. I configured with no swap, but use the entire disk for an ext4 volume for the root filesystem.
Set a password for the oem user, which will be thrown away later, and the user will get to set their own password.
Shut-down at the end

Step 3 – Install the SDK

This is the part where you make the modifications to the image (if any). I wanted to install the Ubuntu SDK.

Optionally at this point, make a backup of your cleanly installed Ubuntu 15.04.1 system
cp ./disk_image ./ubuntu_15.04_install_backup

Boot the previously created install (note the additional options – these are handy)
sudo kvm -m 2048 -hda ./disk_image -chardev stdio,id=mon -mon mon
Once booted to the desktop, in the terminal on the host at the (qemu) prompt type this to switch the VM to the console (which is faster to do stuff than the GUI :) ):-
(qemu) sendkey ctrl-alt-f1
Login to the tty with the oem user/password set in Step 2.
Follow the usual guide to install the SDK and update the system:-
sudo add-apt-repository ppa:ubuntu-sdk-team/ppa
sudo apt-get update
sudo apt-get install ubuntu-sdk
sudo apt-get dist-upgrade
sudo apt-get clean
sudo apt-get autoremove

Shut down the vm
sudo shutdown -h now

Optionally at this point, make a backup of your “SDK-installed” (or modified in whatever way you choose) OEM mode Ubuntu 15.04 system
cp ./disk_image ./ubuntu_15.04_install_sdk_oem_backup

Note: At this point you can boot the disk image and do further customisation – maybe adding other packages which may be of use, but I stopped here.

Step 4 – Prepare the OEM image for ‘shipping’

This is the point where we flip the switch in the installed image before handing it off to another user. On first boot they will get prompted to set locale and configure a new user.

Boot the previously created install which has the SDK installed
sudo kvm -m 2048 -hda ./disk_image
Click the “Prepare for shipping to end user” icon on the desktop – this sets the system to be ready for the first-boot experience for a new user
Shut down the system

Step 5 – Test this all worked

Make a copy of the master image for testing
cp ./disk_image ./testing_oem_install
Boot the test image to try it out
sudo kvm -m 2048 -hda ./testing_oem_install
At this point you should be prompted for the usual post-install setup tasks including language / locale / username & password. Setup as you would a normal machine
Open the SDK (or whatever you installed), test it all works
I tried creating a kit and do other SDK related things
Shutdown when done
Delete the test image
rm ./testing_oem_install

Step 6 – Copy the OEM image to a USB stick for shipping / use

Now we have a ‘final’ image (and optionally some backups) we can copy this to a stick for use by us / someone else. We can of course make more than one by doing this step multiple times with different sticks. On my system as you can see it took ~30 mins to copy the image to the stick. Faster, more expensive sticks may be better, these were pretty cheap.

Copy the disk image to an appropriately sized USB stick
sudo ddrescue -d -D --force ./disk_image /dev/sdX

e.g.

alan@deep-thought:/data/usb⟫ time sudo ddrescue -D -d --force disk_image /dev/sdc
GNU ddrescue 1.19
Press Ctrl-C to interrupt
rescued: 7864 MB, errsize: 0 B, current rate: 1966 kB/s
ipos: 7864 MB, errors: 0, average rate: 4884 kB/s
opos: 7864 MB, run time: 26.83 m, successful read: 0 s ago
Finished

real 26m51.682s
user 0m1.212s
sys 0m30.084s

Step 7 – Test & use the stick

Put the USB stick in a computer set to boot from external media.
Test that you get a desktop and the usual OEM prompts you got in Step 5.
If that works then you can do step 5 again for the same stick or as many sticks as you have.

Success!

Comments and suggestions welcome!