New Adventures in Software by Dan Dyer

It’s five years ago this week that I first released Uncommons Maths as a standalone project spun-off from an early version of the Watchmaker Framework. Uncommons Maths is a Java library that provides, among other things, various classes for working with random numbers, probability distributions and combinatorics. There hasn’t been a release in over two years so today I’ve published a maintenance release, version 1.2.3, on GitHub.

There are two main changes in this version. Firstly, thanks to a contribution from Dave LeBlanc, the Uncommons Maths JAR file is now a valid OSGi bundle. Secondly, I found and fixed a pretty fundamental bug in the convertBytesToLong method of BinaryUtils. The fact this bug went undetected for so long suggests that nobody is actually using this method. It also reinforces that full test coverage is worthless if you don’t select your test inputs carefully.

Finally, the documentation has been updated to make it clear that none of the RNG implementations support the seeding mechanism inherited from the java.util.Random base class. The reason for this is that the setSeed method only takes a single long argument. This means it’s not possible to provide more than 64 bits of entropy, which is insufficient for most of the Uncommons Maths RNGs. In practice calling this method on an RNG instance has no effect. Ideally the method would be over-ridden to throw UnsupportedOperationException but to do so is not possible because setSeed is invoked internally by the java.util.Random constructors. If it threw an exception it would not be possible to create an instance of an object that inherits from java.util.Random.

Twelve months on from my first visit to the UK edition of Droidcon, I was back in Islington this week for this year’s version. The format was the same: barcamp on day 1, conference proper on day 2. I’m not sure the distinction between the two days is all that worthwhile. After the initial hour of pitching and organising, the first day proceeds along similar lines to the second, with four tracks of presentations. I’d vote for just having two normal conference days with the programme for both days published in advance, which appears to be how the Dutch Droidcon works. If there is a need to democratise the programme, this could be done online beforehand.

Anyway, while it’s all still fresh in the memory, here are some of the things I thought worth noting. These may or may not be of interest to other Android developers. If you want to watch some of the talks, keep an eye on the Skills Matter website as I suspect the videos will start appearing on there shortly.

Proguard author Eric Lafortune talked about obfuscation and optimisation of Android apps. I’ve been using his Java obfuscator for many years, since well before it became an integral part of the Android tool set, so it was good to speak to him and finally put a face to the name. Eric’s start-up has another Proguard-based obfuscator/optimiser, called DexGuard, with extra Android-specific features, which may be of interest to app developers who need stronger protection and are willing to pay for it.

Erik Hellman made using OpenCV for computer vision on Android look reasonably painless. I’d previously discarded a good app idea that required this kind of functionality as too complicated and too processor-intensive. Might need to revisit that.

David Teitelbaum of Apkudo showed step-by-step how to crack an APK and inject code. He was able to cheat at one of Zynga’s word games by reverse-engineering the obfuscated code. It was an eye-opening demo and may well have caused a few app developers to seriously consider DexGuard as an extra line of defence against this kind of attack.

Jenkins contributors Christopher Orr and Jørgen Tjernø spoke about the plugins that are available for continuous integration on Android projects. If you have the server resources to dedicate to it, you can use Jenkins to automatically build and test your app with an emulator for every combination of Android version, screen size and configuration that you are interested in. A long time ago I wrote why you should use Jenkins for general Java projects, or Hudson as it was then (technically Hudson still exists as a separate Oracle-backed project but the community has moved en masse to the Jenkins fork). Christopher and Jørgen made the case for using it to improve the quality of your Android apps.

James Hugman gave an updated overview of Kirin, a hybrid approach to cross-platform app development that uses native-specific UIs and Javascript for logic. This was one of the more interesting topics from last year but I never got around to actually trying it out. It was interesting to see how the project has progressed. It now seems to be based around Node.js.

Given my interest in artificial intelligence, the session I was most interested in attending was the one called “Integrating AI into your app”. Unfortunately this was a complete waste of time and served only to prevent me from taking an earlier train home. It was nothing more than an advert for some tool for building expert systems. Neither the tool itself nor the generated systems had anything to do with Android. You expect some degree of being advertised at from the fee-paying keynote speakers who are subsidising the event but at least they have the decency not to stray too far from the main theme of the conference.

That’s the type of session I’d like to see fewer of but I’d like to see more like the preceding session by Anders Ericsson. He delivered a live-coding demo of a fairly clean approach to applying animation to custom views. The results were impressive (the approach is also outlined in this blog post) and the presentation followed on nicely from Chiu-Ki Chan‘s earlier talk on developing custom views. A couple of people pointed out that the new animation features in Android 3.0 and later can be used to achieve similar effects but this approach also works on the large number of Android devices that are still running Android 2.x. This session and David Teitelbaum’s APK cracking were probably the two best talks and are the kind of things I’d like to see if I attend next year.

A couple of months ago I purchased an O2 Mobile Broadband USB dongle so that I could continue working during an ADSL outage. Once the ADSL was back on I had no further use for it. Seeing as I also had a 3G-enabled Samsung Galaxy Tab 10.1 without a SIM, I thought it would make sense to use the remaining data allowance on that. However I was initially unable to find the right settings to make the tablet connect to O2’s network.

Having given it another shot before the unused data expires, I have eventually stumbled upon the correct settings, which I’ll document here for anyone who might need to achieve the same. Previously I was trying the device’s default O2 access point configurations and the APN settings I found on the web. The problem with these is that they are for pay-monthly and pay-as-you-go SIMs sold specifically for use in phones and tablets. The settings for the mobile broadband SIMs are different. You need to use the m-bb.o2.co.uk APN rather than payandgo.o2.co.uk or mobile.o2.co.uk. The full working settings from my Galaxy Tab are shown to the right (the obscured password is, predictably, ‘password’).

Update (30/1/2013): After the remaining data had expired, I didn’t use the 3G functionality of the tablet for a while, during which time I updated the tablet to Android 4.0.4. For whatever reason, the settings shown here no longer worked after adding more credit to the SIM. Eventually I discovered that the “APN type” field had to be set to “default” rather than “internet”. I also switched to using the “o2bb” username that the OS X connection manager software uses with the dongle.

In the absence of anything worthwhile of my own to write here at the moment, I thought I’d instead highlight a couple of interesting blog posts that I’ve discovered recently by id Software’s John Carmack.

I read Carmack’s thoughts on static analysis a few weeks ago and his reports on the effectiveness of tools for analysis of C++ code chimed with my own experiences with Java tools such as FindBugs and IntelliJ IDEA.

The first step is fully admitting that the code you write is riddled with errors. That is a bitter pill to swallow for a lot of people, but without it, most suggestions for change will be viewed with irritation or outright hostility. You have to want criticism of your code.

Automation is necessary. It is common to take a sort of smug satisfaction in reports of colossal failures of automatic systems, but for every failure of automation, the failures of humans are legion. Exhortations to “write better code” plans for more code reviews, pair programming, and so on just don’t cut it, especially in an environment with dozens of programmers under a lot of time pressure. The value in catching even the small subset of errors that are tractable to static analysis every single time is huge.

In the other article, which I only read today, Carmack espouses the virtues of pure (i.e. side-effect-free) functions. His is a pragmatic approach concerned with how to exploit purity in mainstream languages, where it is entirely optional, as opposed to advocating jumping ship to Haskell. Even when 100% purity is impractical there are still benefits in minimising impurity.

A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in. In a multithreaded environment, the lack of understanding and the resulting problems are greatly amplified, almost to the point of panic if you are paying attention. Programming in a functional style makes the state presented to your code explicit, which makes it much easier to reason about, and, in a completely pure system, makes thread race conditions impossible.

My experiences with Haskell have informed how I approach coding in other languages. In certain cases I’ve ended up taking the functional approach to such extremes that I’ve been left questioning my original choice of implementation language.

No matter what language you work in, programming in a functional style provides benefits. You should do it whenever it is convenient, and you should think hard about the decision when it isn’t convenient.

Internet rants about PHP are hardly uncommon but rarely are they as comprehensive as Eevee’s recent effort in which the world’s favourite bad programming language is dissected with a Yegge-esque disregard for brevity.

I view PHP as bash scripting for the web. Useful for getting small tasks accomplished quickly due to its ubiquity but dangerous and just about unmaintainable when taken to extremes.

Unfortunately over the last couple of years I’ve written and read far more PHP than is healthy (exposure to PHP, like radiation, should probably be monitored and subject to annual safe limits). If I had to pick just one complaint from the seemingly bottomless well of PHP’s shortcomings it would be encapsulated by this quote from Eevee’s article:

When faced with either doing something nonsensical or aborting with an error, [PHP] will do something nonsensical.

This fail-slow mentality permeates the language and its type system (or lack thereof). The runtime attempts to avoid disappointing the programmer with error messages and instead carries on regardless until things either grind to a halt, or worse, fail spectacularly in splendid silence. PHP is about as helpful as a SatNav that tells you you’ve made a wrong turn but not until 200 miles later.

Related article: Understanding PHP – A Journey into the darkness…

I was rather surprised on Thursday to be shown a newly released free app on Android Market that looked suspiciously like one of my paid apps. To me and other observers it appeared that this app was simply a modified version of Appmonger being touted by someone who claimed to have paid somebody over four thousand dollars to write it for her so that she could give it away for nothing.

I provide more details about the whole situation over at the Rectangular Software blog. The episode has left me somewhat confused both as to the motivations of this individual and the extent to which she and/or her coder have ripped off my work. Clearly (screenshots to the right to judge for yourself) this app is more than just inspired by or competing with Appmonger but after running the app and digging into the .apk I am now uncertain how much code, if any, has been copied/reused. She may actually have paid somebody else a tidy sum to write the whole thing from scratch. That sounds even more insane to me than just cracking somebody else’s code and passing it off as your own. Why go to all the effort of building an entire app without putting your own stamp on it to avoid any accusations of plagiarism or copyright infringement?

Having convinced myself that I’d quite like a new Blackberry PlayBook courtesy of RIM’s bribe-an-Android-developer program, I have just completed the process of repackaging a couple of Android apps and submitting them to Blackberry App World. This was my first experience of publishing on a Blackberry platform. Previously I had been scared off by horror stories of the pain involved.

The first step was to become a registered Blackberry App World Vendor. This involved e-mailing some documents to RIM to prove that my company actually exists (if you are selling apps as an individual you have to provide proof of your own identity). I submitted the documents on Friday and was approved by Tuesday, which is not too bad considering that they were probably processing a much larger volume of applications than usual due to the free PlayBook offer. It was not as straightforward as signing up for Android Market or the Amazon App Store but it was a lot quicker than the 3 weeks it took Apple to process my application for the iTunes App Store.

The process of acquiring a code-signing certificate involves submitting another form and waiting an hour or two for a pair of e-mails containing a RIM Development Key (RDK) and a PlayBook Debug Token (PBDT). It took me two attempts at this to get both as the first time only the RDK arrived. Once you have the necessary files you then have to use the tools provided to register with RIM and create a developer certificate. This degree of cryptographic hoop-jumping is beyond that required by Google or Amazon and more comparable to Apple’s Provisioning Profile rituals (except much slower due to the waiting around). The whole process is unnecessarily convoluted.

When you’re ready to convert your Android app you’ll want to install the PlayBook simulator so that you can test it. The simulator is provided as a VMWare machine image wrapped up in an installer that is only available as a .exe (for Windows) or .dmg (for OS X). If you develop on Windows this is fine, you can use the free VMWare Player. RIM doesn’t seem to encourage development on Linux but if you can extract the files from the installer you can run the simulator using the free VMWare Player for Linux. The situation on OS X is less satisfactory. There is no free VMWare Player so you’d have to buy VMWare Fusion (£39.99). I was unable to get VirtualBox to boot the image. The simulator also has a major limitation in that it is not possible to change the device orientation, which is less than ideal for testing.

Finally it’s time to convert your app. Firstly you need to make sure it doesn’t use any of the many Android features that are not currently supported by the Blackberry Android app player. There is a verifier tool that will check your .apk and warn you of any problems. You can then convert the .apk into a .bar file, which is a simple one-step process.

Load this .bar onto the simulator, launch your app and then feel the disappointment really kick-in. On the simulator (on my laptop at least, under both Windows and Linux) the experience is abysmal. The apps flicker very badly with artifacts from the previous activity frequently appearing on top of the current activity. On top of this, font rendering is atrocious. Getting useful screenshots for uploading to App World is just about impossible.

I tried multiple Android apps and I know others have experienced similar problems with their apps. I can only hope that it is specific to the simulator and is not a problem on the actual hardware. If not, it needs to be fixed in the final Blackberry Tablet OS 2.0 release otherwise there is absolutely no point in repackaging your Android apps since they will be unusable.

Ignoring the app’s performance on the simulator, the final step is to sign the .bar file and upload it to App World. Signing is straightforward. Submitting the app is not too much different to other app stores. Bizarrely App World restricts screenshots to a maximum of 640×640 pixels (the PlayBook’s screen is 1024×600 so you have to reduce the size). As with Android Market, you are required to upload a feature graphic, although in this case it needs to be a massive 1920×1186 pixels. If you are selling paid apps you don’t have the pricing freedom that Android Market provides. Instead you have to pick from one of the pre-defined price points, just as on iTunes.

I’d be more enthusiastic about Android apps on Blackberry devices if it weren’t for the simulator glitches. As it stands I’ll have to wait until I get my hands on the hardware to judge just how worthwhile it is for Android developers to deploy to non-Android platforms.

It’s fair to say that Blackberry maker RIM’s tablet offering, the 7-inch PlayBook, has not been a commercial success. Launched in the UK at a £399 price point last June, by October the 16GB model’s price had been slashed to £249 in the face of underwhelming demand and last week RIM cut the price again – this time to £169.

In the VAT-free US the asking price is even lower, just $199 – exactly the same as Amazon’s Android-powered Kindle Fire. The two devices are both 7-inch tablets with 1Ghz dual-core processors and 1024×600 displays but the PlayBook has twice the internal storage (16GB vs 8GB), twice the RAM (1GB vs 512MB) and both front (3Mp) and rear (5Mp) cameras (the Kindle Fire has neither).  If Amazon struggles to break-even on the Kindle Fire, preferring instead to make its money selling content, then the current selling price of the higher-spec PlayBook must represent a significant loss for RIM.

To counter the growing consensus that it couldn’t give the devices away, RIM has started doing just that. Specifically, the Canadian firm is targeting developers in a last-ditch effort to rescue its ailing tablet platform from the squeeze being applied by Apple’s iPad and the myriad Android pretenders. All attendees at Blackberry’s Devcon in Amsterdam next week will be rewarded with a shiny new PlayBook and yesterday the company announced that it would give a device to every Android developer that repackaged an existing Android app for distribution on Blackberry App World (the PlayBook is capable of running Android apps non-natively). Today that offer was extended to include any developer publishing any kind of app (native, Android or Adobe AIR) on App World before the 13th February.

This industrial scale bribing of developers represents a concerted push to revitalise the PlayBook platform with the upcoming release of version 2.0 of the device’s QNX-based operating system. Increasing the number of tablet apps available on Blackberry App World is a core part of this strategy. For Android developers it’s an opportunity to get hold of what is by most accounts a decent bit of hardware for the minimal effort of repackaging an existing app.

To test mobile versions of websites, it is useful to be able to connect to a web server on your local machine from a web browser on an Android emulator without having to expose the web server to the Internet.  You can’t use the normal loop-back IP address of 127.0.0.1 because that refers to the emulated Android device itself.  Instead you have to use 10.0.2.2 to connect to the host machine.

That’s fine if your local web server is serving a single site, but if you are using name-based virtual hosting to serve different sites depending on the host name of the request (with aliases for localhost defined in your machine’s hosts file), then you need to be making requests from the browser using the correct host name, not the IP address.

The Android emulator does not use the host machine’s hosts file for name resolution so attempting to access http://myvirtualhost in the emulator’s browser will not work.  This is because the emulated Android device has its own hosts file, so you have to update this to map the virtual host names to the local machine.

The first step is to start the AVD with an increased partition size otherwise you may get an out of memory error when you try to save the modified hosts file:

  emulator -avd MyAVD -partition-size 128

You then have to remount the system partition so that it is writeable:

  adb remount

Then copy the hosts file from the emulated device to the host machine:

  adb pull /etc/hosts

Edit the hosts file so that it includes mappings for all relevant virtual host names:

127.0.0.1        localhost
10.0.2.2         myvirtualhost1 myvirtualhost2

Then copy the updated file back to the emulated device:

  adb push hosts /etc/hosts

You should then be able to visit http://myvirtualhost1 in the emulator’s browser and see the correct site.

Appillionaires is a new book by Chris Stevens about the mobile apps gold rush. I received my copy from Amazon yesterday and within a day I’d finished it, which, if nothing else, is testament to its readability.

Stevens’ credibility on this topic comes from his first-hand experience as a successful publisher of interactive books for the iPad. Early in the book he draws on this experience to articulate the seductive appeal of a new world of possibilities for independent software developers.

Overnight we were a global sensation – newspapers and blogs all over the world were hailing us as the future of publishing. Watching the money roll in and the download statistics rise became a minor obsession. There’s something wildly captivating about viewing your successes through the microscope of modern analytic software. Second by second, hour by hour. It provides the kind of statistic fetishism where viewing download graphs by geographical region or time of day becomes an opportunity to self-indulgently wallow in your accomplishment.

Appillionaires focuses exclusively on the iTunes App Store and the iOS platform. The Android Market is mentioned only in passing and other stores not at all. This doesn’t detract from the book though as, in terms of paid app sales, the iTunes App Store is still a long way ahead of the competition and it’s not an unreasonable assumption that successful strategies there will translate to other similar platforms.

The first third of the book reviews how we got to where we are today. The Apple worship is kept to a minimum and the company’s missteps in the mobile phone market are covered along with its successes. In particular, and in contrast to recent revisionist histories, Stevens doesn’t gloss over the failure of the first iTunes phone, the Motorola Rokr. He also reminds us that third-party apps were not part of the original iPhone offering. Steve Jobs was apparently reluctant to open up the platform to outside developers and at first it was only through the reverse engineering efforts of the jailbreak community that programmers were able to target the device.

Following the history lesson, the meat of the book is the five case studies of independent developers who have earned serious amounts of money from the App Store. As you work your way through these chapters it becomes clear that the book isn’t really about mobile apps, it’s about iPhone games. This is my one major criticism of Appillionaires. Games are a huge part of the mobile app revolution but they are arguably the least interesting part. The book gives honourable mentions to a couple of novelty entertainment apps (iBeer and Pull My Finger) but there is no space devoted to productivity apps, or location-aware apps, or any of the tens of thousands of actually useful apps that are making things just a little bit more convenient for millions of people every day. If just one of the gaming case studies had been dropped in favour of something from some other part of the App Store it would have presented a more complete picture.

There are several points that recur in most or all of the gaming case studies:

• Developers of successful apps have usually released several largely unsuccessful apps before they hit on the winning formula.
• Successful apps rarely become successful straight away. Sometimes they go through several versions, refining the formula until it starts to gain traction with app buyers.
• Successful developers pay attention to small details and don’t settle for “good enough”.
• Successful iPhone games use control systems that are well adapted to the platform, with intuitive touch gestures.

The final section of the book is devoted to business considerations. Stevens discusses how independent game developers are competing with, and being acquired by, the likes of EA, how the vast majority of developers are losing money on app development, and the threat of patent trolls such as Lodsys. This section aims to disabuse the reader of the notion that the Appillionaire outcome is common or even likely. Throughout the book it is made clear that luck plays a large part in which apps become successful. Also, through the choice of examples from the frontier days of the iPhone platform, Stevens gives an impression that, aside from talent and hard work, success was also partly due to being in the right place at the right time and that that time may already have passed.

This is a book that is grounded in reality. The success stories are enticing but the app developer path is frequently portrayed as a form of gambling with many more losers than winners. The gambling analogy is apt. In most forms of gambling if you look beyond the players you’ll find that the real winner is the house. The genius of Apple’s ecosystem is that developers have been convinced to play cards in a poker room with a 30% rake.