Google Skynet

Posted in The Internet by Dan on May 16th, 2013

Google policy is to get right up to the creepy line and not cross it.

These are the words of former Google CEO Eric Schmidt, speaking to The Atlantic in October 2010.

We don’t need you to type at all. We know where you are. We know where you’ve been. We can more or less know what you’re thinking about.

This was two and a half years ago. Today Google is so far over the creepy line it can’t even see the line any more. The problem is not so much individual Google products but the way in which the vast data from all these disparate services is combined to build a very detailed profile of you. Not just what you do on Google sites but also any other site you visit that includes Google’s +1, Adsense or Analytics JavaScript (including this one). Google can read every e-mail you send and receive, it knows everything you search for online and knows pretty much every website you ever visit. It knows where you are, where you’ve been and probably with whom. Google knows more about you than your mother does and with Google+ it’s all neatly connected to your real identity.

I’ve so far avoided signing up for Google+ but it’s increasingly difficult to ignore, particularly as an Android developer, as it’s becoming more tightly integrated into everything that Google does. It’s the keystone of Google’s anti-privacy agenda.

The privacy implications are not the only issue. Most users perceive Google’s search results to represent some sort of objective truth, with links ranked only by their relevance to the search query, but in fact each user is served personalised results. If you and I both search for information on some contentious political topic, Google won’t necessarily give us the same response; it will show us each what it thinks we want to see based on its profiling of us.

In his Google I/O Q&A yesterday, Google CEO Larry Page dismissed concerns over the implications of this profiling:

[Audience member]: Most of my opinion, I can trace back to a Google search. As search becomes more and more personalized, and predictive, I worry that it informs my world view and rules out the possibility of some other serendipitous discovery. Any comment on that?

[Page]: People have a lot of concern about that – I’m totally not worried about that at all.

Personally, I don’t like Google’s all-encompassing vision. It’s not the only company that employs such methods but I can easily ignore the likes of Facebook and Bing as I don’t use them. Google is everywhere.

Spurred more by the closure of Google Reader than anything else, a couple of months ago I began to consider alternatives to relying on the benevolence of one omniscient company for the services I use every day.

The first thing to go was GMail. E-mail is important enough to be worth paying for so I signed-up with FastMail (now owned by Opera), which offers an ad-free service from $4.95 per year. It has a refined web interface, IMAP access, and a simple way to import your existing messages from GMail.

I eventually replaced Google Reader with The Old Reader and for search I’m now using DuckDuckGo, which promises not to track its users or filter search results. DuckDuckGo doesn’t quite match Google in terms of the freshness or depth of its results but it’s fast and has some useful features. On DuckDuckGo’s recommendation I also installed Ghostery for Opera to block Google and others’ attempts to track me on third-party sites. I’ve removed the Google +1 buttons from this blog and will be looking for more privacy-friendly alternatives to Google Analytics and Adsense (both of which would be useless anyway if everybody is driven to use the likes of Ghostery).

I’m not giving up on Google entirely, I just prefer to keep it at arm’s length. I’ll still be developing for Android (for which there were many very welcome developer announcements at I/O yesterday) and no doubt I’ll continue to use some of its other services.

Android ListView – Fixing Missing/Blank Dividers

Posted in Android, Java by Dan on January 24th, 2013

A post with code in it. Because it’s been a long time since the last one.

If you work with ListViews in Android, as all Android developers will do at some point, you may notice that if you set your list items to be non-selectable the dividers that are drawn between each cell disappear. In the Holo Light theme you would normally get a thin light grey dividing line between cells. Depending on how you’ve implemented your adapter, you may find that these dividers become white/transparent when the cells are not selectable. According to Android framework engineer Romain Guy, this is the intended behaviour.

As Haythem Souissi points out in this Stack Overflow answer, you can work around this by ensuring that the areAllItemsEnabled method returns true, even though all items are not enabled (maybe none of them are). The isEnabled method will take care of actually disabling the cells and the dividers will be drawn between each of them.

All very straightforward so far but it all goes wrong again when you try to add a non-selectable header view to the list. The way that ListView deals with headers and footers is that it creates its own adapter to wrap/decorate yours and insert the header/footer views in the appropriate places. This is fine but it’s not delegating the areAllItemsEnabled method, so our above fix no longer works.

Fortunately, and unusually for Android, everything we need to resolve this issue is part of the public API. The adapter decorator class is android.widget.HeaderViewListAdapter. We just need to create our own instance, wrapping our own adapter, and override areAllItemsEnabled as above. There’s a slight complication in that we have to wrap the header view in an instance of ListView.FixedViewInfo and this is a non-static inner class of ListView, but we can reach into our bag of obscure Java tricks to create an instance from outside the enclosing class.

// I assume you know how get a reference to the ListView and create your own adapter.
ListView listView = (ListView) view.findViewById(android.R.id.list);
CustomAdapter adapter = new CustomAdapter(context, listOfItems);
 
// You can create any view you like for the header.
TextView listHeader = (TextView) inflater.inflate(R.layout.list_header, null);
listHeader.setText("My Header Text");
 
// This is how you create an instance of the non-static nested class, providing
// it with a reference to an instance of the containing class (ListView).
ListView.FixedViewInfo headerInfo = listView.new FixedViewInfo();
 
headerInfo.view = listHeader;
headerInfo.isSelectable = false;
 
// HeaderViewListAdapter insists on concrete ArrayLists.
ArrayList headers = new ArrayList(1);
headerInfoList.add(headerInfo);
ArrayList footers = new ArrayList(0);
 
HeaderViewListAdapter wrapper = new HeaderViewListAdapter(headers, footers, adapter)
{
    @Override
    public boolean areAllItemsEnabled()
    {
        return true;
    }
};
listView.setAdapter(wrapper);

Google’s Lag Problem – Or Why Android App Development is 22 Months Behind iOS

Posted in Android, iOS by Dan on January 20th, 2013

On 12th October 2011 Apple unveiled iOS 5.0 to the world. 7 days later Google released Android 4.0 (Ice Cream Sandwich).

Apple rarely announces how many devices are running each version of its mobile operating system but, according to third-party stats[1], today only a few percent of iPhone/iPad/iPod Touch owners are still running an earlier version of iOS.

In contrast, Google is much more up-front about how many people are running each version but it doesn’t compare favourably. Just 39.3% of users have access to the 15-month-old Android OS or one of its later revisions.

Of course, there are some mitigating factors. Android is a much more open ecosystem with dozens of manufacturers needing to test and approve updates for hundreds or even thousands of devices. Apple on the other hand retains a strict control over its (smaller) mobile empire and is able to push out updates directly to end users. But whatever the reasons, and despite Google’s recent efforts to rein in manufacturers and network operators, the fact remains that, unlike their iOS counterparts, Android app developers cannot target a recent version of the operating system and hope to have their app runnable by the majority of device owners.

Google has tried to address this issue with its support library, which brings some of the more recent API additions to earlier Android versions but this is only a partial solution that adds complexity and does nothing to resolve the vast visual differences between Android 4.x and its predecessors.

Today if you want to reach over 90% of iOS users you must support iOS 5.1, which was released in March 2012, and later versions. To reach the same proportion of Android users you would have to target Android 2.2 (Froyo), which dates back to May 2010.

The many comparisons of the relative merits of the latest iOS versus the latest Android version are largely irrelevant to Android app developers who are working from a baseline that is almost two years older than that of their iOS counterparts. An iOS developer can build an app that requires the absolute latest version of the operating system confident that the user base will soon be there (iOS 6 has reached 78.5% penetration in four months). Android developers will always need to be more conservative.

1. These stats may not be fully representative of all iOS users but they provide useful ballpark figures.

Uncommons Maths 1.2.3

Posted in Java by Dan on November 22nd, 2012

It’s five years ago this week that I first released Uncommons Maths as a standalone project spun-off from an early version of the Watchmaker Framework. Uncommons Maths is a Java library that provides, among other things, various classes for working with random numbers, probability distributions and combinatorics. There hasn’t been a release in over two years so today I’ve published a maintenance release, version 1.2.3, on GitHub.

There are two main changes in this version. Firstly, thanks to a contribution from Dave LeBlanc, the Uncommons Maths JAR file is now a valid OSGi bundle. Secondly, I found and fixed a pretty fundamental bug in the convertBytesToLong method of BinaryUtils. The fact this bug went undetected for so long suggests that nobody is actually using this method. It also reinforces that full test coverage is worthless if you don’t select your test inputs carefully.

Finally, the documentation has been updated to make it clear that none of the RNG implementations support the seeding mechanism inherited from the java.util.Random base class. The reason for this is that the setSeed method only takes a single long argument. This means it’s not possible to provide more than 64 bits of entropy, which is insufficient for most of the Uncommons Maths RNGs. In practice calling this method on an RNG instance has no effect. Ideally the method would be over-ridden to throw UnsupportedOperationException but to do so is not possible because setSeed is invoked internally by the java.util.Random constructors. If it threw an exception it would not be possible to create an instance of an object that inherits from java.util.Random.

Thoughts on Droidcon London 2012

Posted in Android by Dan on October 27th, 2012

Twelve months on from my first visit to the UK edition of Droidcon, I was back in Islington this week for this year’s version. The format was the same: barcamp on day 1, conference proper on day 2. I’m not sure the distinction between the two days is all that worthwhile. After the initial hour of pitching and organising, the first day proceeds along similar lines to the second, with four tracks of presentations. I’d vote for just having two normal conference days with the programme for both days published in advance, which appears to be how the Dutch Droidcon works. If there is a need to democratise the programme, this could be done online beforehand.

Anyway, while it’s all still fresh in the memory, here are some of the things I thought worth noting. These may or may not be of interest to other Android developers. If you want to watch some of the talks, keep an eye on the Skills Matter website as I suspect the videos will start appearing on there shortly.

Proguard author Eric Lafortune talked about obfuscation and optimisation of Android apps. I’ve been using his Java obfuscator for many years, since well before it became an integral part of the Android tool set, so it was good to speak to him and finally put a face to the name. Eric’s start-up has another Proguard-based obfuscator/optimiser, called DexGuard, with extra Android-specific features, which may be of interest to app developers who need stronger protection and are willing to pay for it.

Erik Hellman made using OpenCV for computer vision on Android look reasonably painless. I’d previously discarded a good app idea that required this kind of functionality as too complicated and too processor-intensive. Might need to revisit that.

David Teitelbaum of Apkudo showed step-by-step how to crack an APK and inject code. He was able to cheat at one of Zynga’s word games by reverse-engineering the obfuscated code. It was an eye-opening demo and may well have caused a few app developers to seriously consider DexGuard as an extra line of defence against this kind of attack.

Jenkins contributors Christopher Orr and Jørgen Tjernø spoke about the plugins that are available for continuous integration on Android projects. If you have the server resources to dedicate to it, you can use Jenkins to automatically build and test your app with an emulator for every combination of Android version, screen size and configuration that you are interested in. A long time ago I wrote why you should use Jenkins for general Java projects, or Hudson as it was then (technically Hudson still exists as a separate Oracle-backed project but the community has moved en masse to the Jenkins fork). Christopher and Jørgen made the case for using it to improve the quality of your Android apps.

James Hugman gave an updated overview of Kirin, a hybrid approach to cross-platform app development that uses native-specific UIs and Javascript for logic. This was one of the more interesting topics from last year but I never got around to actually trying it out. It was interesting to see how the project has progressed. It now seems to be based around Node.js.

Given my interest in artificial intelligence, the session I was most interested in attending was the one called “Integrating AI into your app”. Unfortunately this was a complete waste of time and served only to prevent me from taking an earlier train home. It was nothing more than an advert for some tool for building expert systems. Neither the tool itself nor the generated systems had anything to do with Android. You expect some degree of being advertised at from the fee-paying keynote speakers who are subsidising the event but at least they have the decency not to stray too far from the main theme of the conference.

That’s the type of session I’d like to see fewer of but I’d like to see more like the preceding session by Anders Ericsson. He delivered a live-coding demo of a fairly clean approach to applying animation to custom views. The results were impressive (the approach is also outlined in this blog post) and the presentation followed on nicely from Chiu-Ki Chan‘s earlier talk on developing custom views. A couple of people pointed out that the new animation features in Android 3.0 and later can be used to achieve similar effects but this approach also works on the large number of Android devices that are still running Android 2.x. This session and David Teitelbaum’s APK cracking were probably the two best talks and are the kind of things I’d like to see if I attend next year.

Using an O2 Mobile Broadband Dongle SIM in an Android Tablet

Posted in Android by Dan on October 24th, 2012

O2 mobile broadband APN settingsA couple of months ago I purchased an O2 Mobile Broadband USB dongle so that I could continue working during an ADSL outage. Once the ADSL was back on I had no further use for it. Seeing as I also had a 3G-enabled Samsung Galaxy Tab 10.1 without a SIM, I thought it would make sense to use the remaining data allowance on that. However I was initially unable to find the right settings to make the tablet connect to O2’s network.

Having given it another shot before the unused data expires, I have eventually stumbled upon the correct settings, which I’ll document here for anyone who might need to achieve the same. Previously I was trying the device’s default O2 access point configurations and the APN settings I found on the web. The problem with these is that they are for pay-monthly and pay-as-you-go SIMs sold specifically for use in phones and tablets. The settings for the mobile broadband SIMs are different. You need to use the m-bb.o2.co.uk APN rather than payandgo.o2.co.uk or mobile.o2.co.uk. The full working settings from my Galaxy Tab are shown to the right (the obscured password is, predictably, ‘password’).

Update (30/1/2013): After the remaining data had expired, I didn’t use the 3G functionality of the tablet for a while, during which time I updated the tablet to Android 4.0.4. For whatever reason, the settings shown here no longer worked after adding more credit to the SIM. Eventually I discovered that the “APN type” field had to be set to “default” rather than “internet”. I also switched to using the “o2bb” username that the OS X connection manager software uses with the dongle.

Carmack on Static Analysis and Functional Purity

Posted in Haskell, Software Development by Dan on October 16th, 2012

In the absence of anything worthwhile of my own to write here at the moment, I thought I’d instead highlight a couple of interesting blog posts that I’ve discovered recently by id Software’s John Carmack.

I read Carmack’s thoughts on static analysis a few weeks ago and his reports on the effectiveness of tools for analysis of C++ code chimed with my own experiences with Java tools such as FindBugs and IntelliJ IDEA.

The first step is fully admitting that the code you write is riddled with errors. That is a bitter pill to swallow for a lot of people, but without it, most suggestions for change will be viewed with irritation or outright hostility. You have to want criticism of your code.

Automation is necessary. It is common to take a sort of smug satisfaction in reports of colossal failures of automatic systems, but for every failure of automation, the failures of humans are legion. Exhortations to “write better code” plans for more code reviews, pair programming, and so on just don’t cut it, especially in an environment with dozens of programmers under a lot of time pressure. The value in catching even the small subset of errors that are tractable to static analysis every single time is huge.

In the other article, which I only read today, Carmack espouses the virtues of pure (i.e. side-effect-free) functions. His is a pragmatic approach concerned with how to exploit purity in mainstream languages, where it is entirely optional, as opposed to advocating jumping ship to Haskell. Even when 100% purity is impractical there are still benefits in minimising impurity.

A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in. In a multithreaded environment, the lack of understanding and the resulting problems are greatly amplified, almost to the point of panic if you are paying attention. Programming in a functional style makes the state presented to your code explicit, which makes it much easier to reason about, and, in a completely pure system, makes thread race conditions impossible.

My experiences with Haskell have informed how I approach coding in other languages. In certain cases I’ve ended up taking the functional approach to such extremes that I’ve been left questioning my original choice of implementation language.

No matter what language you work in, programming in a functional style provides benefits. You should do it whenever it is convenient, and you should think hard about the decision when it isn’t convenient.

We Can Rebuild Twitter – We Have the Technology

Posted in The Internet by Dan on June 30th, 2012

Why do we need Twitter? I don’t mean in the sense of my previous question from 2008 (i.e. what’s the point?), I mean why is it necessary to have a single company controlling “microblogging”? It’s a question that hadn’t occurred to me until I saw Matt Gemmell’s tweet this morning linking to Brent Simmons’ suggestion for a decentralised Twitter-like service:

The discussion is prompted by Twitter tightening its control of the service by restricting what third-party clients and developers can do.

It took me some time to accept Twitter as even vaguely worthwhile. One unexpected way that I’ve found myself using Twitter is as an RSS reader replacement (these days most sites post links to their articles on Twitter), particularly since Google crippled Google Reader’s sharing in order to try to drive adoption of Google+. The fact that Twitter is effectively just doing what RSS feeds and aggregators have been doing for years (microblogging is just blogging with a higher volume of shorter messages) raises the question of why it can’t use a similar decentralised architecture?

Like RSS feeds, Twitter-like feeds could be published anywhere on the open web as long as they conform to some standard format (e.g. RSS with a 140-character limit). People who don’t have a web server to host their feed could use one of a number of competing services just as they might choose Tumblr, Blogger or WordPress to host their blog. In theory clients could fetch the feeds directly from the source although in practice, as with RSS feeds, there would be some efficiencies and other benefits from using intermediate third-party aggregator services. Searching could be handled by Google or another search engine while notifications of mentions could be handled with blog-like pingbacks.

The benefits to a decentralised system based on open standards are that it would allow different services and clients to compete on implementation and would be resilient to censorship. Service providers could experiment with business models such as different advertising strategies or paid access for a premium service. Client developers could innovate in how they present information without the risk of being blocked from accessing the network. Network admins could set-up their own sub-networks based on the same standards, such as a Yammer-like network restricted to a company’s intranet.

PHP Versus the Fail-Fast Philosophy

Posted in PHP by Dan on April 10th, 2012

Internet rants about PHP are hardly uncommon but rarely are they as comprehensive as Eevee’s recent effort in which the world’s favourite bad programming language is dissected with a Yegge-esque disregard for brevity.

I view PHP as bash scripting for the web. Useful for getting small tasks accomplished quickly due to its ubiquity but dangerous and just about unmaintainable when taken to extremes.

Unfortunately over the last couple of years I’ve written and read far more PHP than is healthy (exposure to PHP, like radiation, should probably be monitored and subject to annual safe limits). If I had to pick just one complaint from the seemingly bottomless well of PHP’s shortcomings it would be encapsulated by this quote from Eevee’s article:

When faced with either doing something nonsensical or aborting with an error, [PHP] will do something nonsensical.

This fail-slow mentality permeates the language and its type system (or lack thereof). The runtime attempts to avoid disappointing the programmer with error messages and instead carries on regardless until things either grind to a halt, or worse, fail spectacularly in splendid silence. PHP is about as helpful as a SatNav that tells you you’ve made a wrong turn but not until 200 miles later.

Related article: Understanding PHP – A Journey into the darkness…

Attack of the Clones – Android Market Plagiarism

Posted in Android by Dan on March 3rd, 2012

Appmonger vs. Crapmonger

I was rather surprised on Thursday to be shown a newly released free app on Android Market that looked suspiciously like one of my paid apps. To me and other observers it appeared that this app was simply a modified version of Appmonger being touted by someone who claimed to have paid somebody over four thousand dollars to write it for her so that she could give it away for nothing.

I provide more details about the whole situation over at the Rectangular Software blog. The episode has left me somewhat confused both as to the motivations of this individual and the extent to which she and/or her coder have ripped off my work. Clearly (screenshots to the right to judge for yourself) this app is more than just inspired by or competing with Appmonger but after running the app and digging into the .apk I am now uncertain how much code, if any, has been copied/reused. She may actually have paid somebody else a tidy sum to write the whole thing from scratch. That sounds even more insane to me than just cracking somebody else’s code and passing it off as your own. Why go to all the effort of building an entire app without putting your own stamp on it to avoid any accusations of plagiarism or copyright infringement?

« Older Posts