Better SCM polling for Hudson

Have you already experienced this problem with Hudson? When project Bar depends on project Foo and a commit spans both, Hudson rebuilds the two in an undefined order, thus sometimes leading to a failure in Bar, whereas one is expecting that Foo is built first, then Bar. Let's give a concrete example to precisely understand what happens:

1. The developer adds a new feature in Foo, and makes use of it in Bar. Both Foo and Bar are committed at the same time


2. Hudson polls the projects for changes in no particular order (alphabetical in my experience), thus Bar is appended to the build queue, then Foo


3. Build for Bar fails because Foo has not been rebuilt yet and the new feature has not made it yet


4. Foo is updated and built successfully, and all projects depending on Foo (namely Bar) are added to the build queue


5. Bar now builds successfully

This is a simplified explanation however, in practice the results can be different because Hudson provides every project its own thread to poll the SCM for changes, so depending on the number of projects and the response time of the SCM, projects may be queued in a different order. In no way Hudson will guarantee that Bar will be added to the build queue before Foo.

On the contrary, processing the set of projects in the order of dependencies (least-dependent first), and performing polling synchronously using a single thread for all projects will make sure Bar is queued before Foo, so that a commit spanning both Foo and Bar will result in a success in both Bar, and Foo. To achieve this I have added an experimental piece of code starting from version 1.141 that you can activate by adding the following highlighted XML element in hudson.triggers.SCMTrigger.xml, and make sure to request only one thread for polling:

 true 1

Reload Hudson configuration and watch your projects build. Please report your experience with this improved polling algorithm. If it happends to be useful, we might add it to the SCM configuration screen.

P.S. in theory, even with the improved SCM polling algorithm, there is still a possibility to fail if the commit is made during polling, between Foo and Bar, but the overall probability to fail is much lower than with the genuine algorithm.