Over the last few days I’ve been playing with the beta of Stack Overflow. In case you are unaware, Stack Overflow is a joint venture between Jeff Atwood of Coding Horror and Joel Spolsky of Joel on Software fame. It’s basically a question and answers site for software developers. A mixture of Experts Exchange, Proggit and Wikipedia. The site is scheduled to come out of beta on Monday when it will open its doors to everyone.
From initial impressions I think it’s fair to say that the site will be a success, initially at least. Being A-list bloggers (and now podcasters too), Jeff and Joel have been able to generate a lot of exposure for their project.
Like Jeff’s blog, the minimalist site design is clean and bold and so far the whole system is very responsive (we’ll see if that’s still the case when the traffic spikes on Monday). The beta audience are already posting thousands of questions, almost all of which generate extremely prompt answers (of varying quality).
However, I think the site suffers a little from the ambitions of trying to be too many different things; is it a programming forum, or is it a Wiki?. There are a lot of different ideas in the implementation that interact via a quite complicated set of rules that have evolved over the course of the beta.
Reputation & Badges
Stack Overflow has two mechanisms for measuring a user’s standing within the community. Firstly, each user has a reputation score. This starts at 1 and increases as you make positive contributions (posting questions and answers that get voted up). As you reach various milestones you get more privileges within the community, such as being able to vote on answers or tag other people’s questions.
Your reputation can be diminished if you get voted down or reported for abuse, but it can’t go below 1 and on the whole it’s heavily biased in favour of upward movement.
The second incentive for users to contribute is the ability to collect “badges”. This works exactly like the Cub Scouts. Some badges are easy to achieve (just fill in your profile or post your first question), and others are much harder to obtain (get 100 up votes for one of your answers).
Voting
Voting is one area of the site that I think could do with an overhaul. It’s unbalanced and not transparent enough. If your answer gets voted up you gain 10 points of reputation. But if your answer gets voted down you only lose 2 points. So if you post something that sounds plausible to the uninformed masses but is actually wrong, you could get 5 up votes and 6 down votes for a net score of -1 yet still gain a 38-point reputation boost. An up vote should have equal weight to a down vote, just like on DZone or Slashdot. It also might be better to show both the number of up votes and the number of down votes (as on DZone) rather than just the net total. This would make it easier to identify controversial content (something with 10 down votes and 12 up votes is not quite the same as something with no down votes and 2 up votes).
Another problem with the voting is that down votes penalise the voter as well as the user whose answer is being voted on. So if you post something wrong like “Java passes objects by reference”, I can either ignore it or lose 1 point of reputation for giving you the down vote that you deserve (even then it will take five of us to fully cancel out the one up vote that you got from someone who didn’t know better).
When I queried the justification for penalising down-voters I was told that it was to combat attempts to game the system. Apparently, earlier in the beta, users were posting answers to questions and then voting down everybody else’s answers so that their answer would appear at the top. The idea was that by making users pay to vote down this behaviour would be discouraged. A better solution to this problem would have been to remove the conflict of interest by not allowing users to answer and vote on the same question (which is how Slashdot’s mod points work), rather than punishing all down votes across the whole site.
The net effect of this voting system is that everybody’s reputation increases pretty quickly. Beyond the minimum score required to get full privileges the numbers can become meaningless. To avoid having to rename the site Integer Overflow there are a couple of artifical limits that restrict the number of votes you can cast and the number of reputation points you can earn each day.
Other Thoughts
Aside from my reservations about the voting, my impressions of Stackoverflow are mostly positive. The fact that it has already attracted hundreds of enthusiastic participants suggests that it has genuinely found a niche. However, I do feel that it is probably more elaborate than it needs to be (I don’t really get the need for the Wiki functionality).
Further Reading
Denton Gentry and Sara Chipps have also written about their impressions of Stack Overflow. Or, on a less positive note, you could try Crap Overflow.