Using the older Subversion working copy format with Hudson

Usually I don't let Hudson check out my projects, as I arrange my projects workspaces to be symbolic links to a specific location in a manually-checked-out working copy of a full repository, so that relative paths expressed in the build files resolve properly. But beware that if the Subversion client installed is pre-1.4, Hudson will automatically upgrade the working copies to the new Subversion 1.4 format. This can be a problem as the Subversion client on the command-line cannot be used anymore. If you can't afford to upgrade to Subversion 1.4 or later, you need to turn on compatibility with the older Subversion working copy format (before 1.4), using the Hudson's svncompat13 plugin I just released.

Internally, Hudson uses SVNKit, a Java library to access Subversion repositories. The trick used by the svncompat13 plugin is described in the SVNKit FAQ.

Better SCM polling for Hudson

Have you already experienced this problem with Hudson? When project Bar depends on project Foo and a commit spans both, Hudson rebuilds the two in an undefined order, thus sometimes leading to a failure in Bar, whereas one is expecting that Foo is built first, then Bar. Let's give a concrete example to precisely understand what happens:

1. The developer adds a new feature in Foo, and makes use of it in Bar. Both Foo and Bar are committed at the same time


2. Hudson polls the projects for changes in no particular order (alphabetical in my experience), thus Bar is appended to the build queue, then Foo


3. Build for Bar fails because Foo has not been rebuilt yet and the new feature has not made it yet


4. Foo is updated and built successfully, and all projects depending on Foo (namely Bar) are added to the build queue


5. Bar now builds successfully

This is a simplified explanation however, in practice the results can be different because Hudson provides every project its own thread to poll the SCM for changes, so depending on the number of projects and the response time of the SCM, projects may be queued in a different order. In no way Hudson will guarantee that Bar will be added to the build queue before Foo.

On the contrary, processing the set of projects in the order of dependencies (least-dependent first), and performing polling synchronously using a single thread for all projects will make sure Bar is queued before Foo, so that a commit spanning both Foo and Bar will result in a success in both Bar, and Foo. To achieve this I have added an experimental piece of code starting from version 1.141 that you can activate by adding the following highlighted XML element in hudson.triggers.SCMTrigger.xml, and make sure to request only one thread for polling:

 true 1

Reload Hudson configuration and watch your projects build. Please report your experience with this improved polling algorithm. If it happends to be useful, we might add it to the SCM configuration screen.

P.S. in theory, even with the improved SCM polling algorithm, there is still a possibility to fail if the commit is made during polling, between Foo and Bar, but the overall probability to fail is much lower than with the genuine algorithm.

Test-driven development for Hudson

Recently I convinced myself to implement the various features that I was missing in Hudson: cumulative notifications, build in the right order when commits span multiple projects, gracefully handle project deleted in SVN. But as it involved hacking the guts of Hudson and I didn't want to click-click the whole day in the configuration screens, I wrote the various usecases as tests. Up to now in Hudson, the only existing tests were true unit-tests in the sense of checking that an isolated method returns the proper result. But there was no test to actually verify the complete lifetime of a project: configure, build, delete, or to verify behavior with Subversion.

Writing integration tests for Hudson was hard, here are the ugly bits:

• Subversion local repository is created with svnadmin create in a temporary directory


• Actual builds are done with shell script, this will only work on *nix machines


• There are various reflection hacks. The Hudson API could be more consistent in that area and allow to set field values programmatically

See for yourself and run mvn test in Hudson Tester. There's a long way to go to be able to
easily write tests for Hudson, but the proof of concept works. I hope
this test-driven approach can help Hudson developers implement new features more
efficiently and prevent regressions across releases.