ReportNG 0.9.0 – HTML and XML Reports for TestNG

Posted in Java by Dan on January 24th, 2008

I’ve just released version 0.9.0 of ReportNG, my reporting plug-in for TestNG. This version includes several improvements over the previous release. Most importantly, it addresses the biggest problem with the last version – the lack of information about configuration failures. Previously you had to increase the TestNG verbosity and dig around in the console output. Now config failures are displayed prominently above the test results.

There is also more information about skipped tests now too. Dependencies on groups and methods are listed for tests that have been skipped so that you can figure out why they were skipped.

Finally, aside from a few other cosmetic improvements to the HTML, I have introduce an experimental JUnit XML reporter. I had some problems getting the output that I wanted from the one bundled with TestNG. Mine generates one XML file per test class, which is closer to what you get from JUnit. Now you have the option of using whichever is most suitable for you (the default TestNG reporter or this one). The JUnit XML reporter should be considered incomplete (it doesn’t include detailed test case information yet), but it is useful enough for Hudson to generate summary reports.

If you have any feedback, either post a comment below or use the issue tracker.

JavaNG: A Better Java – What would you do?

Posted in Java by Dan on January 4th, 2008

Forget the shackles of backwards-compatibility. If you could make changes to Java to make it a better language, what would you do? I don’t mean drastic surgery so that you end up with Lisp on the JVM, but what could be done to make Java a better platform-independent, object-oriented language?

Bruce Eckel has some interesting thoughts about the evolution of Java and Sun’s commitment to backwards-compatibility. I have to agree that backwards compatibility will eventually kill Java if new features are continuously compromised by nasty hacks (e.g. erasure) to avoid breaking legacy code. At some point, the guardians of Java must choose new features or backwards-compatibility. To continue to demand both is not sustainable.

With this in mind, I’ve made a list of things that I personally would like to improve in Java, assuming that backwards-compatibility is not a primary concern. I’m interested to hear others’ suggestions, so leave a comment at the end if you have better ideas.

If it weren’t for Sun’s marketing department, we could have called it Java 2. Java++ is just wrong, so I’ll follow Cedric’s naming scheme and call it JavaNG – the next generation OO language for the JVM.

Maybe it’s just me, but this really bugs me…

Firstly, and it may seem trivial to most, but I would scratch an itch that has been bothering me for over a decade. The Cloneable interface does not define the clone() method. An object must define a public clone method in order to be cloneable, but the Cloneable interface does not require this. Therefore it is perfectly legal to implement Cloneable yet not be cloneable. Of course, any class that does this is obviously broken, yet the Cloneable interface apparently cannot be changed since such a class would no longer compile. In my opinion, Sun would be doing developers a favour by making the compiler complain about this. FFS Sun, you broke everybody’s code by adding the assert keyword in 1.4, why would fixing Cloneable be such a big deal?

Is this an object-oriented language or not?

Right, now on to more substantial concerns. Primitives have to go. Everything should be an object. It would simplify learning the language and primitives don’t mix with generics without auto-boxing (and auto-boxing is horrific). No primitives – no auto-boxing – no problem.

I believe that primitives were originally included in the language because of performance concerns about doing everything with objects. Well virtual machines and hardware have both moved on significantly since the mid-90s. And if Ruby seriously is the main competition, then performance clearly doesn’t matter anyway.

BigDecimal.ONE.add(BigDecimal.ONE).compareTo(new BigDecimal("2")) == 0

Obviously if there are no primitives to unbox to, then the arithmetic operators have to be defined to work on the appropriate object types (java.lang.Integer, java.lang.Double, etc.). But why stop there? Arithmetic operators for BigInteger and BigDecimal are long overdue. Without first-class support for arbitrary-precision arithmetic, Java is not a sensible choice for many types of application. I mean, how hard should it be to assert that 1 + 1 = 2?

Types are for life, not just for compilation

I want generics without erasure. Since I can accept breaking backwards compatibility where necessary, I don’t have to accept a weaselly implementation compromise and the resulting frustrations of not being able to inspect type information at runtime.

Just say no to null

For a language that supposedly doesn’t have pointers, there are an awful lot of NullPointerExceptions in Java programs. The simple truth is that developers have proven to be incapable of keeping track of which references are allowed to be null. I want the language to do this for me. More specifically, I want the type system to do this for me. The Nice programming language has the concept of option types. I want one of those. If my method declares that its parameter cannot be null, then any attempt to pass it null (or even to pass it a value of a nullable type) should result in a compiler error. Non-nullable types will become the new ‘final fields’. They’ll be used everywhere unless there is a good reason not to.

Literal bias

Java’s verbose they say. Well Java will always look a bit wordy when compared to Python with its sleek dictionary syntax. But we could have that too. Literals for maps and lists would help to make code more concise and more readable.

Tuples

According to Gilad Bracha, Java was supposed to have had tuples from the start, they just never made it into the language. It’s time to rectify that. No more writing several trivial little classes (or generic Pair and Triple classes) just to wrap together multiple return values.

The stuff that didn’t make it…

I think that list is long enough for now (though I’m sure I’ll think of more later). You’ll notice that I haven’t taken a position on closures. I’m unconvinced and undecided. I need to read the proposals properly. In the past I’ve advocated first-class functions for Java, but for now I’m deferring on that too, at least until I’ve picked a side in the closures debate. I also think that the original Java designers got it right when they left multiple inheritance and operator-overloading in C++ land.

If you think you have a better plan for JavaNG, please add a comment. I’ll post a follow-up in a few days summarising the suggestions.

Developing with SoyLatte – First Impressions

Posted in Java, Mac by Dan on December 29th, 2007

Apple recently released an updated preview of Java 6, but only for 64-bit Intel Macs running Leopard. With the company remaining stubbornly silent on its future Java plans, nobody knows whether 32-bit Macs or the Tiger operating system will ever be supported.

For those developers who don’t want to be forced to upgrade to Leopard or to discard their perfectly adequate CoreDuo Macs, there is an alternative to playing Apple’s waiting game thanks to the efforts of Landon Fuller and his SoyLatte project.

Getting Started

SoyLatte is an X11-based port of the FreeBSD patchset for Java 6. This gives you a non-Apple JDK solution right now and on your terms (no hardware or software upgrade required). The only thing that SoyLatte lacks is Apple’s polished native look-and-feel for Swing and AWT applications (there is no seamless integration with the OS X desktop). This is because it uses X11 for display purposes (in the same way that the Mac port of OpenOffice works). If you are doing non-GUI development none of this will matter to you.

Download and installation of SoyLatte is straightforward. It’s just an archive, containing the pre-built JDK binaries, that can be dropped anywhere you please (mine is under /usr/local). Change your environment variables (JAVA_HOME, PATH, etc.) as appropriate and you are ready to go for command-line development.

Developing

My reason for requiring Java 6 was to build and hack Hudson. Hudson is Kohsuke Kawaguchi‘s open source, extensible continuous integration server. Hudson will run on Java 5 but it requires Java 6 for development. To make Mac-based development on Hudson feasible I would have to get SoyLatte to play nicely with IntelliJ IDEA and Maven.

Integrating SoyLatte with IDEA was a piece of cake. It’s just a matter of adding a new JDK in the settings dialog (just point it at your installation directory). I did have to go back and manually add the SoyLatte JRE libs to the JDK classpath as these were not picked up automatically (resulting in the IDE not being able to find classes like String and ArrayList), but once that was done everything worked perfectly. Crucially, even though I was using Java 6 classes and tools, I was still running the actual IDE using Apple’s Java 5, so I didn’t have to use X11.

Running Maven from the command line presents no new problems. It will just use whichever JDK JAVA_HOME points to. Unfortunately the IDEA Maven plugin uses whatever the IDE is using and doesn’t allow this to be changed. This is in contrast to IDEA’s Ant support, which allows the JDK to be specified explicitly. The inconsistency is probably because the Ant plugin was written by JetBrains whereas the Maven plugin is a community contribution. Running IDEA itself under SoyLatte would address this issue but, for me at least, using Maven from a terminal is preferable to running IDEA under X11.

On Maven

Posted in Java by Dan on December 20th, 2007

If I ever wanted to write down the issues I have with Maven, I couldn’t do a better job than Charles Miller’s post. Maven solves problems that I didn’t have by introducing me to a whole set of new problems that I also didn’t have.

Ant is certainly not perfect (though you can learn to use it effectively) and I remain convinced that a better solution to Java build problems can be found. However, if Maven is the answer, then we are asking the wrong question.

Watchmaker Framework for Evolutionary Computation 0.4.3

Posted in Evolutionary Computation, Java by Dan on December 14th, 2007

This is mostly a maintenance release. Uncommons Maths is now a separate project so the Watchmaker Framework has been modified to use the official version of that library. There are a few other minor tweaks (a couple of classes have been moved around, but nothing in the core framework).

Version 0.4.3 also introduces an experimental EvolutionMonitor component. This a Swing view that gives you some insight into the current state of the population while your evolutionary algorithm is running. In this first version all it does is graph the mean and peak fitness scores (using JFreeChart). Future versions will hopefully display more information (perhaps I will add an API to enable data to be extracted from the population while running). The EvolutionMonitor implements the EvolutionObserver interface so you can hook it up easily by calling the addEvolutionObserver method of your EvolutionEngine.

The other new feature is a new termination condition for terminating the algorithm when the population fitness begins to stagnate. If this condition is used and there is no fitness improvement within a specified number of generations, the evolution engine will assume that no further improvement can be made and will return the fittest individual found so far. This is often a more practical approach than specifying a maximum total number of generations or a fixed time limit in advance.

Uncommons Maths – The Name Game

Posted in Java by Dan on November 26th, 2007

Just to clear up a possible misconception, Uncommons Maths is not intended as a slight against Apache Commons Math (regardless of the incidental mention Commons Math received in my previous rant against Commons Lang). Commons Math provides a lot of functionality that Uncommons Maths does not and probably never will.

The domain name that I use for my software is uncommons.org. The library does maths-type stuff. The name was obvious. Why “Maths” not “Math”? Because that’s what we call it here.

I looked at Commons Maths once and it didn’t meet my requirements (I can’t even remember what they were), which is why I chose not to use it, not because it was defective in any way. The only reason I ever mentioned Commons Math is because it was at that moment, after previous disappointments with some of the Apache Commons libraries, that I exclaimed “We don’t need common software, what we need is uncommon software!”. It was a statement of intent. Hence the domain name.

Announcing Uncommons Maths

Posted in Evolutionary Computation, Java by Dan on November 19th, 2007

Uncommons Maths is a Java library consisting of a comprehensive random numbers package and other useful mathematical utility classes. It was originally part of the Watchmaker Framework for Evolutionary Computation but, due to its usefulness in other domains, it has now been converted into a standalone project (Apache Licence).

This article briefly describes what’s available in this first public release. I am most definitely not a mathematician and, as such, this library is written by a programmer for programmers (but if mathematicians find it useful that’s good too). It includes classes that are useful in real world programs and is not intended to ever cover the full spectrum of mathematics. However, I hope that it will expand in scope over time in this spirit of pragmatism. To that end, suggestions and contributions are actively encouraged.

Random Number Generators

The Uncommons Maths library provides three easy-to-use, statistically-sound, high-performance pseudorandom number generators (RNGs). They are:

MersenneTwisterRNG
A Java port of the fast and reliable Mersenne Twister RNG originally developed by Makoto Matsumoto and Takuji Nishimura. This is faster1 than java.util.Random and does not have the statistical flaws2 of that RNG.
CellularAutomatonRNG
A Java port of Tony Pasqualoni’s ultra-fast Cellular Automaton RNG. It uses a 256-cell automaton to generate random values. To the best of my knowledge, this is the fastest1 available pure Java RNG that completes the Diehard test suite without any problems.
AESCounterRNG
This is a cryptographically-strong3 non-linear RNG that is around 10x faster1 than java.security.SecureRandom. Reverse-engineering the generator state from observations of its output would involve cracking the AES block cipher.

  1. A benchmark comparing the performance of these three RNGs and the two JDK RNGs can be found here (under the title “RNG Performance”).
  2. This applet demonstrates the non-randomness of java.util.Random.
  3. The algorithm is not the only security consideration for RNGs. The source, secrecy and integrity of the seed data is also vital. For highly sensitive applications, consider using something like Fortuna.

Probability Distributions

Using the included probability distribution wrappers, these RNGs (and the standard JDK ones) can be used to generate values from Uniform, Normal, Binomial, Poisson and Exponential distributions.

Permutations & Combinations

Uncommons Maths also includes generics-enabled combination and permutation generators. These are based on Java classes originally written by Michael Gilleland.

Statistics

Uncommons Maths provides a statistical data set class that can calculate a variety of descriptive statistics (variance, median, standard deviation, arithmetic and geometric means, etc.) for a set of values.

Other

Other useful features in Uncommons Maths include utility methods to complement those in java.lang.Math, and utility classes for manipulating binary data.

15 Tips for Better Ant Builds

Posted in Ant, Java by Dan on October 25th, 2007

1. Automate everything.

You’re in charge, make the computer do the work. Ant can insert configuration into properties files, run SQL scripts, change file permissions, upload to FTP sites, and lots more. RTFM and make sure you are aware of everything that Ant can do for you.

2. Make builds Atomic.

Once you’ve automated everything, make sure you can run it all with a single command.Most software engineering endeavours are part of the eternal struggle to limit the consequences of human incompetence. Unlike a computer, the human mind is not well suited to performing a long sequenece of instructions without error. People get distracted, they forget things. My own experience tells me that the longest sequence of actions that a human can reliably reproduce consists of at most one step. Anything more than that and there is a tendency to miss steps out or do things in the wrong order. A slow build is much more bearable if it is atomic because you don’t need to get involved. You can leave it to complete while you do other important stuff.

3. Make it right first, then make it fast.

A build that is fast but wrong is not fast at all; it’s just wrong. A target that does not correctly ensure that its dependencies are rebuilt (when appropriate) will save you 3 seconds on each of the 30 rebuilds that it takes to debug the problem.A useful tip for improving the runtime of slow builds is to use the ANT_OPTS environment variable to tweak the JVM settings used by Ant. For long-running builds you may find that the -server switch provides a noticeable improvement. Alternatively, increasing the heap size (via the -Xmx switch) may help.

4. If you use Ant, use only Ant.

The build.xml is the one true build script. Don’t disrespect the build.xml by using your IDE’s build process in its place. Even if you are really diligent about configuring your IDE, you are very likely to end up with a slightly different build process. Better still, when your colleague checks in a change to build.xml, he or she won’t tell you and your build will be broken. If you’re lucky this will become obvious sometime after you start complaining loudly about others checking in broken code. In the worst case your build will be subtly broken and you won’t notice. Avoid the contempt of your co-workers and make your IDE use Ant. Don’t let the old UNIX hacker in the corner use make just because he doesn’t like XML (he doesn’t like Java either, he’s just doing it because the company hasn’t had any C work in the last 6 months 10 years). If he uses make, you all have to use make (good luck), since there can be only one true build script.

5. Follow conventions.

Ant expects your build file to be called build.xml (unless you tell it otherwise). Developers expect the build file to be in the root of the project. Other than the understandable desire to confuse and irritate the rest of your team, including the lucky souls that get to maintain your code long after you get head-hunted by Google, there’s no reason not to follow these conventions. Another less well-known convention is to prefix all “internal” targets (those that should not be called directly) with a hyphen. This is suggested in the Ant manual. It has the advantage that it is not possible to invoke targets that follow this naming scheme from the command line.

6. Provide a clean-up target.

It is usual for makefiles and Ant scripts to provide a target called “clean” that removes all generated files and returns the project to its initial state. Without this functionality failed builds can be a real problem. Providing a clean target is made easier if source files (those that are under version control) and generated files (those that are derived from versioned files) are kept separate.

7. Compile test code at the same time as production code.

Test code, by definition, depends on production code (if it didn’t, your test coverage scores would not be very impressive). Therefore, changes to production code can potentially break test code. For this reason it is a good idea to enforce the rule that test classes are always compiled at the same time as production classes (you don’t have to run the tests at this point). Since you are firing up the compiler anyway, it won’t add too much overhead and it avoids breaking windows.

8. Make builds self-contained.

A build that has external dependencies is a build that is difficult to to configure. When your hard drive fails or a new developer joins the team, you want to be able to configure a new machine for building ASAP. If your build depends on tools like Checkstyle or TestNG, add these to the project repository. This makes your build simpler because you know exactly where to find the files and you don’t have to worry about version mis-matches. Other than Ant itself and the JDK, external dependencies should be eliminated wherever possible (Maven has a different approach to these kind of dependencies).

9. Parameterise essential configuration.

It’s not always practicable to eliminate all external dependencies. For example, you may need to configure a database connection as part of your build. Don’t hard-code this kind of configuration in the build.xml, use Ant’s support for properties files to allow individual developers to set configuration parameters. Don’t complicate matters by making everything configurable. Only parameterise those settings that really need to be configurable.

10. Provide sensible defaults.

Where parameters are required for building, you can make things much simpler by providing sensible default values rather than leaving everything to be configured by the individual developer. For example, if one of the parameters is a JDBC URL, a sensible default might be to point to localhost (most developers will probably be developing against their local database). If one of the required properties is the file system path to a particular tool, provide the default installation directory. By providing sensible defaults you can eliminate most of the configuration hassles.

11. Visualise dependencies.

The build script for a large project can have complex, often ad-hoc dependencies. Tools such as yWorks Ant Explorer can provide insight into dependencies by providing a graphical representation of the structure of the build. If the graph looks like one of these, consider refactoring.

12. Constrain classpaths.

Avoid the temptation to use a single global classpath for compilation. Always ensure that code is built with a classpath that contains only those classes that will be available to it at runtime. For example, if your build creates an EAR file and a Swing client, make sure that the Swing client is compiled with access only to those libraries that will be deployed with it. Do the same for the EAR file and any other modules. This approach minimises the potential for embarrassing NoClassDefFoundErrors at runtime and also helps to detect inappropriate dependencies.

13. Modularise your build.

Arrange your project into coherent, self-contained modules. For example, you may have a Swing GUI module and web module among others. Layout your project files to support this arrangement. The goal is to make modules as self-contained as possible and to minimise dependencies between them. The following example shows one possible way of arranging your project files:

myproject
|__modules
|  |__gui
|  |  |__lib
|  |  |__src
|  |     |__java
|  |        |__main
|  |        |__test
|  |__web
|     |__lib
|     |__src
|        |__html
|        |__java
|           |__main
|           |__test
|__build.xml

14. Favour convention over configuration.

If the source code for module1 is in module1/src/java, where would you expect to find the source code for module2? If files are in consistent predictable locations within each module, it’s not necessary to explicitly configure each location. By always following the same conventions about where things go in each module, you minimise complexity and make it easier to implement the next recommendation…

15. Eliminate duplication with macros.

If all of your modules are laid out identically, the common tasks (such as compiling source trees and creating JAR files) are essentially identical for each module. Ant 1.6 introduced macros to enable these operations to be defined just once and applied repeatedly with different parameters. Familiarise yourself with the Macrodef task and consider how it can be applied in your build files. The example macros below are for compiling generic modules. The first macro defines how a single source tree is compiled. Because we have followed the advice from the rest of this article, all we need to provide to run the macro is the name of the module (by convention modules are in directories of the same name, all Java source is in the src/java directory of the module and below this are the main and test source trees). The second macro simplifies things further by combining the two separate calls to the first macro. The main classes are built, then the test classes are built with the main classes added to the compiler’s classpath. Using this macro we can compile an entire module with a single line:

<compilemodule name="mymodule" />
<!-- This macro compiles one source tree (i.e. the main
     source tree or the unit test source tree) of a given
     module. -->
<macrodef name="compiletree">
  <attribute name="module"/>
  <attribute name="tree"/>
  <element name="treeclasspath" optional="true"/>
  <sequential>
    <mkdir dir="./@{module}/build/classes/@{tree}" />
    <javac destdir="./@{module}/build/classes/@{tree}"
                   debug="on"
                   deprecation="on"
                   optimize="on"
                   source="1.5"
                   target="1.5"
                   srcdir="./@{module}/src/java/@{tree}">
      <classpath>
        <treeclasspath/>
        <path refid="base.path"/>
      </classpath>
      <compilerarg value="-Xlint:unchecked" />
    </javac>
  </sequential>
</macrodef>
 
<!-- This macro compiles all source (including unit tests)
     for a single module.  -->
<macrodef name="compilemodule">
  <attribute name="name"/>
  <element name="moduleclasspath" optional="true"/>
  <sequential>
    <compiletree module="@{name}" tree="main">
      <treeclasspath>
        <moduleclasspath />
      </treeclasspath>
    </compiletree>
    <compiletree module="@{name}" tree="test">
      <treeclasspath>
        <!-- Add the main classes to the classpath for unit
                             test compilation. -->
        <path location="./@{name}/build/classes/main" />
        <moduleclasspath />
      </treeclasspath>
    </compiletree>
  </sequential>
</macrodef>

These macro definitions are somewhat verbose (due to Ant’s XML syntax), but the benefit is that they ensure that all modules are built in the same way. Rules for compilation only have to be defined once. Also, keep in mind that once you have written macros for one project, you can reuse them elsewhere. You may choose to put all of your macros in a separate file and import this wherever it is required.

ReportNG – Simple HTML reporting for TestNG

Posted in Java by Dan on October 21st, 2007

The default HTML reports generated by TestNG, though comprehensive, are a little bit horrid. Since Cedric introduced a reporting API to TestNG a little while ago, I have been using my own hacked together, Velocity-powered reporter.

It’s very simple and doesn’t have all of the information from the default report but it is easier to work out what’s going on with your test results at a high level. I’ve had it in a Java.net repository for a while and now I’ve just produced a slightly improved version for download by anybody that might find it useful.

It’s not particularly slick and there is room for improvement. Suggestions are welcome.

Evolving Sudoku (Watchmaker 0.4.1)

Posted in Evolutionary Computation, Java by Dan on July 21st, 2007

Everybody loves Sudoku (probably), and evolutionary computation is kind of neat, so what could possibly be better than an animated evolutionary Sudoku solver?

The applet’s animation gives a good feel for how the randomly directed search eventually converges on the right solution through the power of cumulative selection. Have a play with the population size setting to trade-off performance with reliability (harder puzzles will typically require a larger population).

I’ve got lots of ideas for improvements to the Watchmaker Framework for Evolutionary Computation, but before I could get started, I first had to finish off the stuff I had been playing with. So that’s what I’ve been up to this evening and the result is version 0.4.1.

« Older Posts