New Adventures in Software by Dan Dyer

Bill the Lizard (if that is his real name) wrote an interesting post revisiting the perennial debate of whether a formal Computer Science education is worthwhile for programmers or not. Bill makes several good points in the post and the comments. I’m paraphrasing here but he basically accuses self-taught programmers who dismiss a university education of arguing from a position of ignorance. If you haven’t experienced it for yourself, how do you know it wouldn’t have been beneficial?

“Education: that which reveals to the wise, and conceals from the stupid, the vast limits of their knowledge.” – Mark Twain

There are comments, both in the Reddit discussion that Bill references and in response to his own article, that suggest that a CS degree is actually an indicator of a poor programmer. As CS graduate myself, I cannot accept this hypothesis. I’ll accept that whether or not a person has a degree is not a reliable indicator of programming aptitude but I would be stunned if there was not at least some positive correlation between formal education and performance. There will always be exceptions that buck the trend. I’ve worked with some excellent developers who have not been to university and I’ve worked with people who have the degree but don’t know how to use it.

Self-learning is a vital skill for a programmer. Even if you’ve got your degree, you can’t stop learning there if you are going to continue to be useful. I do believe that it is possible to learn through self study anything that you could learn at university, but the problem with a home-brew education is that the teacher is as ignorant as the student. You tend to learn whatever you need to know to solve the current problem. It’s a piecemeal approach. Over time you’ll learn lots but the danger is that, because you don’t know what you don’t know, there may be blindspots in your knowledge. A good university course will be structured to teach what you need to know, even if it seems irrelevant at the time. It will provide a broader education, introducing concepts and programming paradigms that may not seem particularly useful but which all contribute to building a deeper understanding of the field.

The vital word in the preceding paragraph is the one emphasised: “good”. All of this debate ignores the crucial fact that degrees are not all equal. There are good degrees, bad degrees and many points in between. This fact is perhaps under-appreciated by those who have not been through the university process. If we could factor in the quality of the degree it would make for a more reliable indicator of developer aptitude.

Hiring Graduates

If you are responsible for hiring programmers you should familiarise yourself with which universities rate best for computer science education (this list may be quite different from the overall list of top universities). Something else to consider is the content of the course. The clue is normally in the name. If it’s not called just “Computer Science” or “Software Engineering” beware. It may be watered down by some other subject or it may be called something like “Management Information Systems”, which might suggest that more time was spent writing essays than writing code.

Q: What do you call somebody who graduates bottom of their class at medical school?

A: Doctor.

Perhaps the biggest mistake when considering the value of a degree as an indicator of programmer aptitude is treating it as a binary proposition: any degree = good, no degree = bad. This is simplistic and wrong. Getting a degree is easy as long as you can last the distance. Here in the UK, many universities will award a third class honours degree for an overall grade of 40%.  In fact, you can get a pass degree (no honours) with just 35%. Think about that for a while before calling them in for an interview. Over the course of 3 or 4 years, almost two thirds of everything that person did they got wrong and they have the certificate to prove it.

For senior developers degrees are mostly irrelevant since they will have copious real world experience on which they can be judged but being able to judge the worth of a degree is useful when hiring junior developers. All else being equal, somebody who graduated top of their Computer Science class at MIT will be a better bet than somebody who has a third class degree in HTML Programming from the South Staines Internet University.

It’s been very nearly a year since the last release (0.4.3) of the Watchmaker Framework for Evolutionary Computation so, before 2008 disappears completely, it’s time for a new version that includes some of the stuff that I’ve been working on intermittently during this time.

Backwards-(In)compatibility

The primary purpose of the 0.5.0 release is to break backwards-compatibility and upset users.  If you’re already using 0.4.3, there are two main changes that you need to be aware of:

Firstly, the StandaloneEvolutionEngine class has been renamed to ConcurrentEvolutionEngine. The old name did not reflect what distinguished this EvolutionEngine implementation from other implementations (some of which are yet to be added but will be in future releases). One such alternative is the new SequentialEvolutionEngine, which performs all work on the request thread and, as such, is useful in managed environments that do not permit control over threading.

Secondly, all methods and constructors that take probabilities as parameters now use the new Probability type rather than Java doubles. There was too much duplication of range-checking code and code that generated events with a given probability. All of this has now been encapsulated in one place.

Genetic Programming

The Watchmaker download includes source code for several example applications. New in this release is the tree-based genetic programming (GP) example. Genetic Programming is a type of evolutionary algorithm that involves evolving computer programs. The Watchmaker Framework is a general-purpose evolutionary computation framework.  It does not include any GP-specific classes but there is nothing preventing you from applying it to the task of evolving program trees. This example is a Watchmaker-powered Java implementation of the first GP problem presented in Toby Segaran’s book, Programming Collective Intelligence. From a set of example inputs and outputs, it generates a program that encapsulates the formula for converting the inputs to the outputs.

Useful Links

• Project home page
• Full change log
• Download latest version
• API documentation
• Example applets: Travelling Salesman, Biomorphs, Sudoku
• Feedback: Issue tracker

If you want to generate HTML pages from Haskell, there are a few different approaches available. The Haskell.org wiki lists several web-related libraries.

Text.Html

The standard Haskell libraries include the Text.Html module. This is a combinator library that allows you to generate HTML pages by combining Haskell functions. Each function is responsible for generating an HTML fragment and these fragments are combined to construct a complete page. Haskell’s static type-checking ensures that the output is well-formed (though not necessarily valid).

By arranging your code appropriately, you can achieve a satisfactory separation of presentation and logic, but I’m not comfortable with this approach. The combinator idea works excellently for implementing parsers with Parsec but, in my opinion, is not such a good fit for generating web pages. It reminds me a little bit of the dark age of Java web development when pages were generated by servlets concatenating Strings of HTML tags. The combinator approach is more elegant than the unchecked concatenation of Strings in Java, but it’s still an attempt to internalise HTML. I’d rather write my HTML in HTML than in Haskell (or Java), particularly for complex pages with CSS and JavaScript involved.

WASH/HTML

WASH/HTML is another implementation of the combinator approach. It guarantees both well-formedness and validity of its HTML 4.0.1 output. WASH stands for Web Authoring System Haskell. WASH provides a complete Haskell web stack, of which WASH/HTML is a part. With WASH/HTML, a simple Hello World page is generated using the following code:

build_document (head (title (text "Hello World!"))
               ## body (h1 (text "Hello World!")))

Haskell Server Pages

Haskell Server Pages (HSP) brings the ASP/JSP/PHP model to Haskell.  In other words, it mixes presentation mark-up with computations. There are a some minor differences in the Haskell implementation of this idea compared to the imperative variants. Perhaps most interestingly, all XML fragments are actually Haskell expressions, which allows the type-checker to enforce well-formedness. Using HSP, an HTML page can be expressed as a Haskell function as follows (where helloWorld is another function that generates the text content for the page):

page = <html>
         <head><title>Hello World!</title></head>
         <body><p><% helloWorld %></p></body>
       </html>

This is a lot closer to pure HTML than the combinator libraries. HSP also supports the concept of XML pages and hybrid pages, so instead we can create an actual HTML document that includes Haskell code fragments:

<% import System.Time %>
<html>
  <head><title>XML page</title></head>
  <body>
    <h1>Hello World!</h1>
    <p>Page requested at <% getSystemTime %></p>
  </body>
</html>

HSP has roughly the same high-level benefits and drawbacks as JSP and PHP. These technologies give you a lot of power but they can also lead you astray. It takes discipline to maintain a clean separation between logic and presentation. Since HSP permits (requires) side-effects, there’s nothing to stop you from doing truly ugly things such as embedding database update logic in the code that generates your HTML pages.

HStringTemplate

HStringTemplate is a Haskell port of Terence Parr’s StringTemplate engine (originally for Java but also available for C# and Python). StringTemplate is a templating engine along the lines of Apache Velocity but is more restrictive in what processing it permits templates to perform. Specifically, it strictly enforces separation between model and view. It is not possible for a template to cause side-effects elsewhere (no database updates, etc.). StringTemplate can be used for generating any kind of text output, not just XML/HTML. In fact, one of its principal uses is as a code generator for ANTLR.

I have chosen to use HStringTemplate, in preference to the alternatives described above, for my current Haskell project.

A StringTemplate template for an HTML page looks something like this (the StringTemplate website provides more details of the kinds of expressions that can be used):

<html>
  <head><title>Hello</title></head>
  <body><p>Hello $name$</p></body>
</html>

To convert this into an HTML page, the template must be parsed, a value assigned to the “name” attribute, and the HTML rendered to a file. The Haskell code to do this might look like this (see the Haddock documentation for descriptions of the various functions):

import Text.StringTemplate
 
main = do templateText <- readFile "templateFile"
          template = newSTMP templateText
          writeFile "outputFile" html
          where html = toString $ setAttribute "name" "Dan" template

This is only a very simple example. The StringTemplate template language provides facilities for working with lists and maps, and for recursion and basic conditionals. HStringTemplate defines the type class ToSElem for mapping custom data types to types that can be inserted into a template.

I have found that HStringTemplate works very well. It allows me to keep my HTML source separate from my Haskell code. The only criticism I have is that the conditionals are not very expressive. An “if” statement in a template can only check whether a particular attribute is set or unset, or whether a boolean is true or false. The particular problem that I had was that I wanted to apply different formatting depending on whether a value was positive or negative. There was no straightforward way to do this. Instead I had to insert a separate pre-evaluated boolean attribute, isNegative, into the template and check that. This appears to be a (deliberate) limitation of all StringTemplate variants, not just the Haskell version.