First Qualifying Solution Submitted for $1 Million Netflix Prize

Posted in Software Development, The Internet by Dan on June 26th, 2009

The word on the street (well Reddit actually) is that the BellKor’s Pragmatic Chaos team today submitted the first qualifying solution for the Netflix Prize.  If nobody submits a better solution within the next 30 days then they will claim the $1 million reward that has so far eluded the best efforts of thousands of programmers and researchers since the competition was launched in October 2006.

Netflix is a US-based online DVD rental service.  One of their features is that they make movie recommendations to customers based on their previous viewing history.  In order to improve their recommendations system, Netflix has been offering a million dollar reward to any individual or team that is able to develop software that increases the accuracy of these recommendations by at least 10%.

The financial rewards and intellectual challenge of the Netflix Prize have encouraged almost 50,000 individuals and teams to attempt to solve the problem using a vast array of different AI and data-mining techniques.

The BellKor team have overcome such obstacles as the Napolean Dynamite problem and will no doubt have the champagne on ice while they nervously wait to see if  anybody else is able to surpass their results within the next month.

Opera Unite Divides Opinion

Posted in The Internet by Dan on June 17th, 2009

Opera Software would have you believe that yesterday they reinvented the web.  The launch of their new Opera Unite service has received a decent amount of publicity. By now you’ve probably heard all about it, but if not you can read the details here.

The 10 second summary is that version 10 of Opera’s web browser contains a web server that allows users to serve web content directly from their desktop machines or laptops. However, this description doesn’t really capture the potential of the platform.

Some commentators have dismissed the announcement with a “so what?”. Opera Unite content is only going to be available while the user’s computer is switched on and running Opera and will be constrained by their available upload bandwidth (which often isn’t much thanks to the ‘A’ in ADSL). That doesn’t really cut it when compared to low-cost web hosting packages capable of serving thousands of users, but then the comparison isn’t particularly helpful.

I don’t need Opera Unite to host my personal website from my desktop. I can install and configure Apache, tweak my firewall/router settings and find a solution to dynamic IP address issues. The point is that with Opera Unite, you don’t have to do any of that.  Opera have completely eliminated all of that hassle and in doing so have made web serving accessible to even non-technical users.  But that’s only half of the story. Serving your personal home page via Opera Unite is still sub-optimal. If you want (semi-)permanent web hosting, pay for some cheap PHP hosting or get a account.

If somebody gives you an Opera Unite URL, you shouldn’t expect that resource to be still around tomorrow or next year like you would with a link to Wikipedia. The real value in Opera Unite is in ad hoc sharing and transient collaboration. Things that were possible but bothersome previously are now trivial because you don’t have to worry about server configuration and networking issues.

For example, say I wanted to invite every reader of this blog to join a chat session. I could try to find out which IM clients you all use and try to arrange something via MSN Messenger, Skype or Google Talk. Or I could install and configure my own IRC server. Or I could try to find a third-party server to host the chat room. With Opera Unite I can simply open up my lounge and give you all the URL (regardless of which browser you happen to be using). It just takes a few clicks. The service is transient.  When we’re done, I kick you all out.

In our chat session I might decide to share some photos or other files with you.  I could send them via e-mail or upload them to an FTP server or a service like Flickr, but again it’s simpler with Unite. I just enable the appropriate service and share the URL. You can browse my shared directory and grab what you want directly from my machine. The link probably won’t work tomorrow, but you won’t need it tomorrow. Temporary is fine when it’s this easy.

The other service that I’m already finding useful is the media player, which enables me to remotely play my home MP3 collection from the office. The Unite platform is based on open standards, so it will be interesting to see what other ideas for services people come up with.

From Antipathy to Ambivalence – The Great Twitter Experient, Day 14 (The End)

Posted in The Internet by Dan on February 25th, 2009

The two weeks are up.  The Twitter experiment is complete.  Did I find the dolphin, or am I still waiting for the magic?

If nothing else, I’ve achieved a greater understanding of the dynamics of Twitter, but I don’t think that it has yet had a significant impact on the way that I communicate or the way that I consume information.  I have to admit that the experience was less awful than I thought it might be.

If you use Twitter, get a client

The first thing that I discovered is that the web interface to Twitter is just not usable.  It’s a decidedly less confusing and more dynamic experience using one of the many desktop clients.  I’m currently on TweetDeck having previously tried Twhirl, though neither is the one Twitter client to rule them all.  The static nature of the website and the way that it displays conversation fragments out of context was one significant reason why I didn’t see much value in the service.

Content is king?

I don’t feel that my initial complaints about the general unimportance of most tweets, and the plain pointlessness of many, were unfounded.  And when somebody starts tweeting every minute about the film that they are currently watching (a film that I am not watching), it becomes incredibly irritating.  They say that on the web Content is king.  Well, since the advent of Twitter, Content has abdicated and crown prince Banality is now running the show.

However, I did find that I had a higher than expected tolerance for tedium, mostly because I could easily consume/ignore most of the tweets that I received without suffering much distraction.  And even if the tweets were mostly superfluous, they did occasionally raise a chuckle, such as Wez’s new software development methodology.

They have real people on Twitter now?

I was not expecting to be able to interact with people I knew in the real world on Twitter, because two weeks ago none of them were on Twitter, but Wez and David have signed up since I started.  David is already well ahead of me in terms of followers, and all without the help of a series of self-promoting blog posts.  He is the next microblogging celebrity.  Meanwhile, Wez is using Twitter to stimulate the global economy.

But what do you actually use it for?

From the start of this experiment I’ve struggled with what to tweet.  I wanted to stay on-topic.  I thought maybe I could use Twitter to complement this blog.  I don’t think that this approach is particularly easy or even that useful.  It’s easier just to go down the stream-of-conciousness route and write anything that comes to mind.

In this regard, I wouldn’t be surprised if, for many, Twitter is effectively a write-only medium where everybody’s contributions are welcomed and few are valued – a kind of voluntary collective delusion.

Moreso than with blogging, you are subscribing to people rather than topics.  If you want to use Twitter, you have to accept that even if you pick compatible people to follow, a lot of what they write will not be of interest to you, particularly when it concerns the trivialities of daily life.

One use case for Twitter that does makes sense to me is within a development team for posting status updates that can easily be consumed and responded to by other team members.  Yammer offers a Twitter-like service well suited to this niche (access can be restricted to selected people only).

In conclusion…

If I could change one thing about Twitter it would be the 140 character limit.  Countless times I’ve sat there trying to figure out how to remove 13 characters from a message without altering its meaning. This involves creatively removing punctuation and finding abbreviations or shorter synonyms.  A limit is good for keeping messages concise and to the point, but a few more characters would go a long way to improving the quality of the content.

I found the global eavesdropping aspect kind of interesting, although it’s a bit hit-and-miss as there is no way to filter by quality.

Twitter is mostly harmless but, as far as I can see,  does not deliver on much of the hype that surrounds it.  I’m still indifferent to Stephen Fry getting stuck in a lift and wary of news organisations such as the BBC treating it as a reliable journalistic source.  Twitter is not the harbinger of a communications revolution, it’s an occasionally relevant diversion.

I suppose that the big question is will I continue to use it now that the two weeks are up?  I don’t know.  Probably to some extent.  It’s there, I have an account, it won’t take much effort to continue.  On the other hand, if Twitter disappeared tomorrow I wouldn’t miss it.  I might not even notice for a while.

Global Eavesdropping – The Great Twitter Experiment, Day 13

Posted in The Internet by Dan on February 24th, 2009

It’s been over a week since my previous post on my Twitter experiences.  In the meantime I’ve only been using it sporadically (so much for being addictive). Time flies and I’ve almost reached the end of the two-week period that I had assigned to this little experiment.

Ask a rhetorical question on Twitter and somebody will answer it within a couple of minutes. What surprised me when I pondered the usefulness of OSGi is that the person who responded was not one of my followers but somebody who I had not interacted with before. This highlighted an aspect of Twitter that I had not previously paid much attention to. As well as following individuals, you can track particular search terms. So anybody monitoring Twitter for discussions about OSGi would have been alerted by my tweet.

This is particularly interesting to me because my number one complaint about Twitter has been the utter pointlessness of most of the content (including that which I have been contributing). Focusing on this aspect, you don’t have to follow anybody. You could use Twitter in a similar way to Google Alerts, surfing the zeitgeist ready to pounce on any discussions that include your favoured keywords. This way you participate only in on-topic conversations and avoid the what-I-just-ate messages. Unfortunately, there’s still no quality control.  No Digg, Reddit or DZone equivalent to promote the good content and ignore the bad.

On a related note, you can get an overview of what topics are presently occupying the thoughts of the planet’s bored masses via TwitScoop, which provides a real-time word cloud and top 10 topics.

140 characters should be enough for anybody. Really?

One thing that I’ve held back until this penultimate post is something that I knew would irritate me right from the start: the 140 character limit. I don’t care what anybody says, it’s not enough. I understand the benefit of having a limit; it forces people to keep their entries concise. However, 140 characters is too restrictive.  Sometimes not even enough for a fully-formed sentence.  Twitter doesn’t need to be restricted by SMS limits.  The future of Twitter does not lie with SMS. Everybody who is using it from a mobile phone has Internet access. A limit of 250 or 300 characters would be altogether more civilised.

Tweeted URLs: A regression in web usability

When you start adding links to your tweets it leaves less room to provide context.  The URLs are necessarily shrunk and obfuscated to save room, making each link a leap into the unknown. It would be very nice to be able to attach the link to particular words in the message (you know, like we’ve been doing since 1992). Twitter takes you back to the days before hypertext.

Eating Navel Fluff – The Great Twitter Experiment, Day 5

Posted in The Internet by Dan on February 16th, 2009

Time for another status report from Twitterland (is that the correct name, or is it “twitosphere” or “information super bridleway” or something?).  Here are some more disjointed thoughts (is Twitter affecting my ability to combine multiple sentences into coherent prose?).

Since I started this little adventure I’ve been paying more attention to the media coverage of Twitter. Andrew Orlowski at the Register lays into Twitter and the media’s (the BBC’s) obsession with it:

Writing about Twitter is the journalistic equivalent of eating the fluff from your navel.

…The rest of the world, however, completely ignores it. But with the journalists’ attention fixed firmly on each others’ navels, they don’t seem to realise what a fringe activity Twitter is.

…Twitter is a bit of a charity-case itself: both technically and financially, it’s a lost cause. That’s plain to everybody, it seems, except the journalists who use it and who can’t stop Twittering about Twitter.

Orlowski’s piece references “Twestival”. I tracked down this disturbing footage of the London event via the BBC (who else?).  It’s a few dozen geeks with iPhones standing around ignoring each other while sending tweets to other people who aren’t there. These people need to get out less.

After 5 days, the initial novelty has worn off.   Maybe I should approach tweeting in a different way? Perhaps every message should be a haiku?

One problem with Twitter content, regardless of who you are following, is that each obfuscated URL is a leap of faith and the messages are usually too truncated to provide any meaningful context.

I haven’t posted many Twitter messages in the last two days.  I’m still trying to figure out how best to use Twitter.  What should I tweet?  Should I stay on-topic (software development and tangentally related things) or should I branch out to anything that comes to mind?  I’m thinking this would be counter-productive as people who are following me mostly came via this blog and are unlikely to share my other interests.  Without some kind of focus it becomes Facebook without the functionality.  I’m also proud of the fact that I’m 5 days in and I have yet to tweet the details of anything that I have ingested.

I also need to find more people to follow.  I’ve been through Jurgen’s list and picked out a few more people that might be of interest.

Talking to Strangers – The Great Twitter Experiment, Day 2

Posted in The Internet by Dan on February 13th, 2009

Previously on New Adventures in Software: Dan signs-up for Twitter to try to figure out the attraction of self-stalking.

I’m starting to get a feel for Twitter now. Not so much in terms of finding it useful, or even particularly interesting, but I have a better appreciation of what it is.

So what is Twitter?

Twitter inhabits a space somewhere between a feed reader and an instant messaging client. If you use it via the Twitter website it shows its more reader-like tendencies. It aggregates messages in a very static way.  You just go there and see what’s been said since you last checked it. If you use one of the myriad clients (today’s client is Twhirl), the experience is closer to using MSN Messenger or similar.  Messages pop up in the corner of your screen, not in real-time but via polling.

I rarely have IM programs running because I find them to be incredibly damaging to productivity. Conversations spawning at random, all requiring attention, distract you from whatever you were trying to do on the computer. Twitter is not quite as distracting. Messages arrive but you don’t have to respond, whereas on IM it would be rude to ignore them.  Then again, on Twitter, the messages usually aren’t directed at you. You are just tapping into other people’s stream-of-conciousness thoughts. I wouldn’t be surprised if that is the real attraction of Twitter, though people might not like to admit it. It’s nice to get these little reassurances that you are not alone in the world, right now there are other people out there doing “stuff”. You select a set of people that you like, or admire, or find interesting and you tune into their stuff. The nature of that stuff is unimportant, just so long as you are in the loop.

Of course, there is also a conversational aspect to Twitter. The people you are following might not have addressed their messages to you, they may not even know that you exist, but you can jump in at any point and respond to something that they have said. This is the “global conciousness” aspect that Twitter fans highlight. Yes, you can ignore what your parents and teachers told you and talk to strangers. It’s not a proper conversation though. It lacks the depth of e-mail and it lacks the immediacy of instant messaging.

You also get to see the conversations that don’t involve you. This is incredibly irritating when somebody you are following replies to somebody you are not following. It’s like when you are on a train and the person next to you is talking on their mobile phone and you are forced to listen to one half of a conversation. You just get some out-of-context sentence. You can try to track down the original message, but that’s just one more distraction. It would be better if the Twitter clients defaulted to not showing you messages that were addressed to other people.

Today’s Highlights

I now have 13 followers, and I am following 19 people.  It seems that there are a few people who signed up for Twitter in response to my original post.  This is no longer an experiment, it’s a movement. They too are trying to see what all the fuss is about.

I also have to abandon the “nobody in the real world uses Twitter” argument now since David has signed up and I can confirm that he is a real person that I know. Go on, you can follow him too. Maybe he’ll say something.

I’m not going to write a blog post for every single day of this 2-week experiment. I will report back again in a few days.

Finding the Dolphin – The Great Twitter Experiment, Day 0

Posted in The Internet by Dan on February 11th, 2009

I’ve said it before, I don’t get Twitter.  For me, the hysteria generated by the flat-lined signal-to-noise ratio of this limited medium is deeply confusing. The last time I felt like this was back in the early 90s as I stared cross-eyed and frustrated at one of those Magic Eye pictures, trying to find the dolphin. How come everybody else can see something in it while to me it’s just nonsense?

Since I last wrote on the subject of Twitter, the hype has increased still further. Somebody used TwitPic to post a picture of the AirBus that landed in the Hudson River in New York. As a result, this “citizen journalist” earned himself 15 minutes of international fame, including featuring on the BBC’s television news at least twice. I somehow suspect that had he chosen to upload his snap to Flickr instead, the BBC would not have been nearly as interested. Somebody at BBC News is a Twitter lover. The corporation’s online coverage of the recent attacks in Mumbai prominently featured information sourced from Twitter, whether accurate or not.

If further evidence were needed that with Twitter the medium really is more important than the message, it arrived last week. What was this seismic event that many are heralding as the tipping point for the microblogging revolution?  Stephen Fry got stuck in a lift. No, really, that’s it. Had General Melchett used his Blackberry to phone a journalist instead of to tweet, he could not have bribed them to write about this unfortunate but utterly banal occurrence.

I am not alone in my dismissal of Twitter as an irrelevance. Terence Blacker writing in the Independent aptly described it as “self-stalking” and summed it up as follows:

Twitter may have novelty value but it is more than mere surface silliness. It is anti-thought, the deadening white noise of modern life with all its pointless business. As for the dotty idea that short computer messages are full of wit, insight or observation – that is, to quote the master twitter himself, “arse, poo and widdle”.

I discovered the Independent article via Graham Linehan’s blog. Linehan (co-writer of the peerless Father Ted) has a different take on the value of Twitter:

…the manifold possibilities of Twitter are enough to make you giddy. This is a new world, people! We are officially in the future, not with jetpacks, but with something much cooler – the hive mind. Ignore those grumpy luddites in the broadsheets and elsewhere, who don’t understand it, can’t be bothered to learn how it works and are frightened at the prospect that people are entertaining themselves in a way that doesn’t involve accepted media forms.

By now, I think that I’ve firmly established that I don’t really see the point of Twitter, but I don’t want to be dismissed as a “luddite” who “can’t be bothered to learn how it works”. So it’s time to take the red pill and see for myself how deep this rabbit hole goes. Today I’m embarking on an experiment: a two-week trial to see whether there is any substance to the Twitter hype. I’m deeply sceptical but also approaching it with an open mind. Will I experience a higher level of conciousness or will I endure and proliferate a fortnight of pointless anti-thought?

I’ve just signed up for a Twitter account. If you’re already a Twitter user, I need some followers (sounds like I’m starting my own religion here). I also need interesting people to follow. Please also send me tips on how to get the most out of Twitter. If you are one of the many people who responded to my previous post agreeing that Twitter was pointless, I will report back here at regular intervals in the next couple of weeks and let you know how it’s going on the other side.

Stand back, I’m going in…

Fixing WP-Syntax

Posted in The Internet by Dan on January 26th, 2009

WordPress is great. In some ways. Sometimes. It has all these plugins that add all kinds of neat features. It even has a really easy-to-use auto-update feature for plugins. So when I saw that a new version of Ryan McGeary’s very useful WP-Syntax plugin was available, I let it do the upgrade for me.

Unfortunately, this made a mess of my previous post (thanks to David for pointing this out to me – I wasn’t actually diligent enough to check for myself that the upgrade hadn’t broken anything). All the < and > characters in my Java code had been replaced by HTML entities (&gt; and &lt;). Rolling back to WP-Syntax 0.9.1 didn’t fix the problem, which was odd. Then a few neurons flickered into life and I had a vague recollection of having solved this problem previously. It turns out that I had modified the PHP source for the WP-Syntax plugin (as described here). I simply had to make the same change for the new version.  I’m ignorant as to why WP-Syntax does not include this modification by default. Maybe it breaks something else, but it works for me.

Counting to 10 breaks the Web

Posted in JavaScript, Software Development, The Internet by Dan on December 19th, 2008

Opera Software (makers of arguably the finest web browser known to man) recently uncovered a latent Y2k-esque bug aflicting many major websites.  In pioneering the revolutionary concept of a two-digit version number, Opera 10 (currently in alpha) has shone a light on the sins of dozens of shoddy JavaScript hackers.

I can’t really understand how this happened.  Even less gifted “webmasters” should be able to grasp the concept of 10 without taking their shoes and socks off.

Twitter: I still don’t get it

Posted in The Internet by Dan on December 4th, 2008

I’m certainly not the first person to ask this question, but what exactly is the point of Twitter (I mean aside from being the poster child for scalability problems)?

Most Twitter feeds consist of messages like “Cheese sandwich for lunch”, “Just watched the latest episode of [some TV show]” and “Put more RAM in my PC, much faster now”.  Who on earth wants to read this stuff?  Yet people on Twitter are each following hundreds of other people, getting bite-sized updates about the most mundane details of the lives of huge numbers of strangers.

Perhaps, like reading blogs, the secret is to filter out the rubbish?  There are an awful lot of blogs and, it has to be said, a lot of awful blogs, but eventually you find the ones you like.  Maybe I just need to find the right people to follow on Twitter?  Fortunately, Jurgen Appelo has produced another one of his lists.  This time it’s the Top 50 Twitterers to Follow for Developers.  There are several instantly recognisable names on the list; many people whose blogs I read.  So I took a look at most of their Twitter feeds and no, these people can’t make it interesting either.  There are confusing fragments of conversations with other people that you don’t know, half-formed thoughts, obfuscated hyperlinks and yet more mundane life details that nobody needs to know.

Can it really be that the emperor is naked?  Has the medium become more important than the message?  I’d really like to know what the appeal of it is.  Is it the constant reassurance that there are hundreds of people out there who have lives as equally dull as your own?

Online it’s everywhere.  Even the BBC has started using it.  BBC News’ coverage of the recent events in Mumbai included contributions sourced from Twitter (with no indication of the validity of these contributors or guarantees as to the veracity of their information).

Then again, is Twitter really that popular?  Nobody that I know in the real world uses Twitter.  They all use Facebook, some have blogs but not one person uses Twitter, not even the software developers.  Where does the buzz come from?  Why are there dozens of imitations trying to repeat Twitter’s “success”?

So this is an invitation.  Please tell me (in the comments below) why I should care about Twitter.  What am I missing?  I’m even prepared to give it a trial, if somebody can convince me that it is worthwhile.  I’ve no problems sharing the trivial events of my daily existence (I’m thinking of maybe having soup for lunch today).  It’s the other bit that confuses me.  How can I extract value from other people’s Twittering?

« Older Posts