Archive for January, 2010

h1

Java Build Server

January 23, 2010

Update [15.05.2010]: I finally had a chance to try out SecureCI, which is pretty much exactly what I described in this post. The guys from Coveros did a great job, so I suggest after reading this post you head over to their website, download SecureCI and give it a go. Thanks John for pointing this out to me.

In my last Java project, I set up a build server with Continuous Integration (CI) capability. I am a big fan of Test Driven Development (TDD) and I quite enjoyed Hudson telling us right away when someone checked in code that broke the build. It just gives you so much more confidence in your code and keeps it releasable at all times. In addition, we used Sonar to measure the quality of our code. I found it quite interesting to study how the different metrics changed over the course of the project. We paid particular attention to code coverage and tried to keep it as close to 100% as possible. This should happen naturally anyway if you are practicing TDD.

Setting all this up was a good experience to learn how it all fits together – But I learned my lesson and I don’t really want to do it over and over again. So, I started looking for a Virtual Appliance that had most, if not all, of these capabilities. Setting up a build server must have been done by other developers a million times before. Surely, someone along the way stuck it all on a Virtual Machine (VM) and made it open source, right? Well, that’s what I thought, but I couldn’t find any. So, I decided to create one myself. Unfortunately, I am lacking the hardware to do so at the moment. I have even already created a project for it on https://launchpad.net/java-build-server. Although, I haven’t come around to create the VM yet, I still wanted to share my idea here to maybe animate someone else to go ahead and give it a go. So here it is: my idea of a Java build server.

I planned to start with what I think of as a good enough build server: A VM based on Ubuntu Server JeOS with pre-configured installations of Subversion (Source Control), Hudson (Continuous Integration), Sonar (Quality Metrics), MySQL (to store Quality Metrics), Maven (Build) and Nexus (Enterprise Maven Repository). Once this is all working I am going to also install Trac (Wiki and issue tracking system). You might of course have your own preferences for tools to use for the tasks listed above. Feel free to swap out whatever tool you wish. You might already have an enterprise Maven repository. Fine, just use your existing one. All I am saying is, these tools will be on my build server VM; pre-configured as much as possible with standards found in the tools’ documentation, following the convention over configuration paradigm.

Creating the VM

Here is how I am planning to create my Java build server VM. I am using VMware Studio to create the VM. VMware Studio is a Virtual Appliance itself and I deployed it in VMware Server 2.0. First I created a basic VM based on Ubuntu 8.04.1. Studio automatically installs VMware tools and embeds an in-guest management component called Virtual Appliance Management Infrastructure (VAMI), which lets you configure the network settings of your VM after it has been built. This is essential, since you want your development team to be able to connect to your copy of the build server VM. Here is a list of the essential settings I used to create the base VM:

  • ubuntu-8.04.1-dvd-i386
  • 1 CPU
  • 512 MB RAM
  • 8 GB
  • Network settings: DHCP (to automatically find an IP address)
  • Target format: zip

After the build finished, I downloaded the zipped VM and deployed it on my VMware Server instance. I started the VM and assigned it a static IP address on the boot screen. I logged in and carried out the following steps:

Using the VM

Of course there are some project and company specific configurations that cannot be set in advance. Here is what is left to do:

  • Install a VM container on a server (e.g. VMware Server)
  • Download and start the VM in your container
  • Set the network details on the boot screen
  • Add your developers as users to Subversion (in /usr/local/svn/passwd-team)
  • Create a project
  • Create Hudson jobs to monitor your project

What’s next?

To make using this Java build server even easier I am also planning to create the following:

  • a Maven archetype that only needs the IP address of the build server to be configured
  • a script to create a new project on the build server (create SVN project, create Hudson jobs, etc.)
Image by indigoprime via Flickr
h1

Is estimation waste?

January 7, 2010

Lean Thinking suggests that estimation is to be considered wasteful. Instead, the throughput time (lead time) for planned items is projected by using previous measurements. I always thought this was an interesting approach and today I came across this screencast about Naked Planning, a development process created by Arlo Belshee based on Value Based Planning. In a nutshell Naked Planning consists of the following.

  • A list of goals that describe objectives to satisfy the costumers needs.
  • A fixed length (7 ± 2) queue (Product Backlog). This is based on the idea that 7 is the number of things a person can keep in their mind at once and prioritize. It ensures that customers are making value prioritized decisions at all times.
  • The 7 slots in the queue are to be filled with Minimal Marketable Features (MMF) by the customer (Product Owner). The customer uses the list of goals to generate new MMFs once the development team has eaten away the top priority MMF.
  • The customer prioritizes the MMFs purely by value. There is no upfront estimation of MMF’s. This is based on the idea that cost is distributed linearly and therefore approximately constant, whereas value is distributed exponentially and therefore outweighs cost. Estimation itself is considered a wasteful task (see arguments below).
  • There is one (and only one) slot for urgent tasks, which the costumer can use to bypass the queue. The development team tries to keep this slot empty at all times.
  • The development team has a fixed amount of slots to pull MMFs from the queue; just like the Work in Progress (WIP) limit in Kanban. This depends on the size of your development team, but there should be about two slots.
  • A MMF is considered done when everyone included in the process thinks it’s done, including the customer. In the done state, the MMF is already part of an automatic deployment, so unless someone takes action to prevent it, it will end up in production.
  • Throughput time (time between ‘MMF gets added to the queue’ and ‘MMF is in production’) is projected by previous measurements. Since there are no estimates this is the only way to do this. These measurement-based projections are supposed to be more accurate than estimation-based predictions. Arlo puts a dotted line on the bottom of the queue with “Your approximate time to wait is x days”. This is what he calls the “Disneyland wait time”.

This is what I gathered from the above mentioned screencast, Arlo’s comment on this blog entry by Aaron Sanders explaining Naked Planning, this podcast from Agile 2008 and this short video from Agile 2007. I don’t think there is any “official” documentation about the process, just videos from conferences.

Estimates deliver negative business value

Arlo argues that when customers prioritize the MMFs purely by value without considering the cost at all, they actually make better decisions. Let’s say we’ve got a two by two matrix with cost on the horizontal axis and value on the vertical axis and MMFs scattered throughout this matrix. The common behavior is to choose the cheap MMFs on the left – first the ones on the top with high value, then the ones further down with low value. These low value / low cost MMFs are what Lister and DeMarco label with ‘Death March’ features. If you take away the cost axis, people only choose high value MMFs – some with low some with high cost. Arlo calls these short term value (high value / low cost) and long term value (high value / high cost) items. According to his studies, it turns out that this is a much better portfolio of MMFs than just low cost MMFs. Consequently, not giving your customers estimates to consider when prioritizing their MMFs actually let’s them make better decisions. Having estimates is therefore a negative business value proposition or as Arlo puts it: “It’s positive cost, it’s slightly dishonest and it delivers negative business value.

He further explains that not doing estimates also saves a lot of time:

I never spend any time tracking estimates, improving the team’s ability to estimate, applying math to counter the effects of holidays & sick days, etc. This estimation normalization stuff takes a surprising amount of time – as I only discovered when I stopped trying to make halfway-decent estimates. Possibly most importantly, eliminating estimates eliminates conversations like “you promised that, this week, you’d do X, Y, and Z. Where’s Z?” Instead, the conversations become “What is the relative importance of X, Y, and Z? Well, X, then Y, then, eventually Z. Can we just release X+Y as soon as they’re done? Sure. We’ll give you some warning when the release is coming close.” It opens up a lot more success opportunities, and shuts out several types of bad behavior.

Points to debate over

I find the idea from a developer’s perspective very appealing, because it takes away the dreadful task of having to come up with an estimate, but there are a couple of issues with this approach that I am not quite sure about.

  • Team motivation: How do you achieve a commitment from the team and keep the motivation up, without the looming deadline of the next sprint demo and without the team trying to achieve the estimates? One possible way to keep the motivation up is to keep regular meetings in which the team demonstrates the done MMFs to customers. Naked Planning does this by keeping demos and retrospectives as a heartbeat for the team. However, I am questioning if the motivation will be as high as with the commitment to a certain number of MMFs prior to the sprint. But maybe I am wrong. There is a study mentioned in Peopleware by Lister and De Marco, that suggests that “the highest [software development] average productivity was on those projects that didn’t estimate at all.”
  • Customer buy-in: You need to find a customer that is able and willing to make business decisions independently from the cost of the wanted MMF. From my experience with Scrum, it is hard enough to convince customers that they don’t need a detailed project plan with a big design up front. How can you justify abandoning release and sprint planning (and sprints for that matter) all together? Here Arlo draws from his experience by using the Naked Planning process with his team. In his experience, the ‘Disneyland wait time’ (the time for the 7th item on the queue to go into production) is usually never longer than 90 days. This means the customer can plan with the seven most valuable MMFs within 3 months.
Image via Wikipedia