New Adventures in Software by Dan Dyer

I’ve moved from Blogger to my own site so that I can have more control (via WordPress). I’ve managed to import all of the old articles but seem to have lost the comments in the process. I’ve also updated my JavaBlogs entry, which has had the unfortunate side-effect of spamming the front page with old articles. Sorry about that…

Version 0.3.0 of the Watchmaker Framework for Evolutionary Computation is now available for download. The release includes several refinements to the API, one major bug fix (for Roulette Wheel Selection), and a new example program based on the first exercise in An Introduction to Genetic Algorithms.

This release can be considered the first stable release of the framework. There ought to be no major bugs and it is unlikely that any changes will be made to the API that will break backwards compatibility prior to the 1.0 release.

There are more features to be added (including support for GP and distributed fitness evaluations) before the framework is ready for its 1.0 release.

As always, I’m keen to receive suggestions for improvements and new features.

• Changelog
• API Documentation
• Travelling Salesman Example Applet

I discovered this by accident when replacing some String keys with an enum (in an attempt to reduce the potential for bugs in some code I had written). The change resulted in a big performance hit in a routine that repeatedly queries the HashMap.

In Java 5.0, the implementation of java.lang.Enum.hashCode() is much slower than it needs to be. Furthermore, hashCode() is final, so it can’t even be replaced by something trivial, such as simply returning the ordinal.

The performance problem should be fixed in Java 6.0 but, if you are stuck on Java 5.0, it’s worth keeping in mind when using enum constants to key a map in performance-sensitive code.

With plenty of good, free alternatives, such as Continuum and CruiseControl, you’d need a compelling reason to spend money on per-user licensing in order to use JetBrains’ new TeamCity product.

JetBrains’ legendary reputation for productivity-boosting, easy-to-use tools has taken a bit of dent recently (certainly within our office) thanks to the rather buggy 6.0 releases of their flagship product, IntelliJ IDEA. Regardless, they have managed to attract a legion of devoted followers eager to pay $500 each to avoid having to use Eclipse.

But can they repeat the trick for Continuous Integration? Continuum does the job with minimal fuss and is admirably simple to configure, so why pay more?

TeamCity has several nice-to-have plus points that put it ahead of the free competition, but not to the extent of $399 per user (or even the current offer price of $199 per user). Briefly, these plus points are:

1. Support for .Net projects, as well as Java, in the same product (nice if you need it).
2. Server-side code coverage analysis (you could get the same results by running EMMA from the Ant build.xml)
3. Server-side static code analysis using IDEA inspections (nice but relies on using IDEA for development – Checkstyle and FindBugs could do something similar from Ant).
4. Flashy AJAX web console (perhaps too flashy)
5. Pre-tested commits. Sends your changes to the CI server for building before committing to version control. Your changes are only checked-in if the build succeeds and all tests pass.

So if like me you got a free TeamCity licence with your IDEA 6.0 purchase, there is enough in it to make it worthwhile. However, unless your entire team uses IDEA and have free TeamCity licences, you may find it a little restrictive compared to the free alternatives. You can enable guest access to allow anybody to view your builds and download artifacts, but only licensed users can control builds and receive notifications when builds fail.

This review could end here with a lukewarm thumbs-up and a recommendation to use the product if you got it for free but not to spend any money on it. But to do so would be to ignore the crucial architectural difference between TeamCity and its free competitors. The TeamCity website alludes to this key advantage without fully explaining the implications.

Continuum, CruiseControl and the like share a common approach. The web application running inside the servlet container checks out the code from version control and builds it locally (on the web server machine). In TeamCity, the web application is merely the coordinator, building happens elsewhere. TeamCity has a grid of Build Agents to which the work is delegated. You may choose to install the Build Agent on the Team City server, and/or you may decide to have one or more other machines that do the building.

The advantages of this approach were not initially obvious to me, beyond the ability to share large workloads across several nodes. The important point is that the build grid does not have to be homogenous. Build agents can be installed on different platforms with different resources available to them.

To understand the possibilities that this provides, allow me to introduce you to my current project. The server is a J2EE application but has a mission-critical native component written in C and invoked via JNI. The target platform is Solaris 10 on Sparc hardware, but developer desktops are x86 Windows XP machines. When running on Windows, the native component is replaced by a Java stub (the Ant script detects the platform and skips building the native component). In addition, we have two native client applications, one for Win32 and one for Mac OS X.

In TeamCity you can have multiple configurations per project. Each configuration can express preferences about the build agent used to execute it. In the above example we can have a server configuration that will only be built by a Solaris build agent and two client configurations, one for Windows and one for the Mac. Assuming we have at least one build agent available to satisfy each configuration, this is where the magic happens. A change is checked into Subversion. TeamCity detects that the repository has been updated and springs into life. A Solaris machine is instructed to check out the project and run the server build. Simultaneously, a Mac is prompted to build the Mac client and a Windows box gets to work on the Windows client. After x minutes, assuming there are no errors, the built and tested server and clients are available to download from the TeamCity web console. Without any human intervention we’ve built and tested across three different platforms.

But it doesn’t end there. How do we develop the Solaris-specific parts of the server on Windows desktops? There are two options. The first is to make the changes on Windows, check them into Subversion, logon to the Solaris machine, check out and test the updates and then go back to Windows to fix the bugs. This has the potential to break other people’s builds if they do an update before we have finished.

The alternative is to shorten the write-compile-debug cycle by doing the actual development on the Solaris machine using a remote session. The drawback of this approach is that using a heavyweight IDE in an X-session on the remote server may not be feasible, particularly if there are other users vying for resources.

Fortunately, TeamCity’s build grid gives us another option. Using the TeamCity IDEA plugin (plugins for Eclipse, NetBeans and Visual Studio are apparently coming soon), we can develop on Windows and submit a remote build to the grid. Without anything being committed to Subversion, a Solaris agent will do the build and report the results. When we have the final version ready to check-in, we can do a pre-tested commit as described earlier. This will make sure that the build is successful on the target platform before committing to version control.

TeamCity has its rough edges (see Hani’s review of version 1.0), but it has raised the bar for Continuous Integration functionality. If you’re going to pay money for it though, wait for version 1.2 as there are still a few bugs that might cause frustration.

Just installed a Microsoft wireless keyboard and mouse combo on my Mac Mini. In true Microsoft style you have to reboot to complete the install. What it doesn’t mention is that if you don’t remove the CD-ROM the machine will never boot again. Since, unlike most other home computers, the Mac Mini has no mechanical mechanism for ejecting a CD in emergency situations (short of dismantling the machine with a screwdriver). It looked like I was in a bit of a fix.

Fortunately, I was able to access the web on another machine and discovered this vital piece of information: if you hold down a mouse button while the Mac boots, it will eject the disc in its drive. This may be common knowledge among hard-core Mac people but it was new to me. So my soon-to-be-discarded Logitech mouse was able to come to the rescue and everybody lived happily ever after.

The Travelling Salesman problem is a good demonstration of where evolutionary algorithms (EAs) can be useful for finding good solutions to difficult problems. This applet allows you to compare evolutionary and brute force approaches to the Travelling Salesman problem for up to 15 cities. You can tweak the parameters of the evolution to see how it affects results. The applet uses the Watchmaker Framework to implement the EA.

In a quarter of a second we can find a solution to a problem that would take three days to brute force (WARNING: Don’t try to brute force a result for 15 cities unless you have a seriously powerful machine). The EA solution is not guaranteed to be optimal (from observations it seems to be optimal about 90% of the time for this example and is “near”-optimal in the other 10% of cases).

Version 0.2 of the Watchmaker Framework for Evolutionary Algorithms is now available for download from the project website (Document & Files section).

This version includes several minor tweaks plus support for concurrent fitness evaluations to take advantage of multi-core and multi-processor machines. It also includes improved example programs (more on that later).

API Documentation

To accompany the first release of the Watchmaker Framework, I’ve posted the first two parts of a tutorial that shows how to use it:

• Part 1 – A Brief Introduction to Evolutionary Algorithms
• Part 2 – Implementing Evolutionary Algorithms in Java with the Watchmaker Framework

Part 3 will be up sometime soon(ish) and will demonstrate how to solve a real problem with Watchaker and EAs.

As usual, constructive comments, corrections and suggestions are all welcome.

Version 0.1 of the Watchmaker Framework for Evolutionary Algorithms is now available for download from the project page on java.net (where you will also find details of implemented and planned features). The framework requires Java 5.0 and a familiarity with generics.

Feel free to download and play. You can leave comments and suggestions here or on the project site.

The Watchmaker Framework for Evolutionary Algorithms brings the power of evolution to Java with a simple and flexible API. This is still very much work in progress but, if you are interested, there is working code and examples available from the Subversion repository. Suggestions for improvements are encouraged.