Distributed CI: How it could work

I think distributed CI is the logical next step in the evolution of Continuous Integration by getting rid of the manual step of running a private build on the developer machine before committing to the master repository. Here is how I imagine this might work.

  • Managers, developers and customers (or all stakeholders if you will) collaborate via a wiki and issue tracking tool. These two components tend to be heavily intertwined and are often realized within a single tool. Project management is happening in this area as well.
  • Developers can access the tasks in the issue tracker from within their development environment (IDE).
  • While programming, quality assurance plugins inside the IDE point out potential bugs or design flaws to the developers using the same metrics as are measured by the quality analysis run by the central CI server.
  • Whenever the system is in a consistent state developers commit their changes to their local clone of the Version Control System (VCS) repository. This should happen fairly frequently; several times a day. If you don’t commit frequently you are not doing continuous integration. (side note: The VCS should contain everything that is needed to produce the system from scratch, including database scripts, operating system setup (aka infrastructure as code), etc. – everything.)
  • The local Continuous Integration (CI) tool notices the changes in the VCS repository, gets the latest changes from the master repository and kicks off a build of the changed code module. The build also runs a suite of unit tests to make sure the changes haven’t broken any existing code.

DCI - central server

  • After successfully building (and testing) the code the build tool stores the created artefacts in a repository manager. Note: I believe build artefacts don’t belong in your VCS for several reasons. You shouldn’t have to store them there either, because you should be able to recreate them from any version in your VCS. But there are also reasons why it might not even be a good idea to do so. Most importantly, it makes traceability harder. For each artefact I want to know which version of my source was used to build it. If build artefacts are stored in VCS as well and therefore create new versions themselves, this becomes harder. Also, all these snapshot versions that are produced many times a day, of which some might even break the build are not worth the disk space they occupy.
  • Once the local commit build is successfully run and the snapshot artefacts stored in a repository manager the local CI tool pushes the changes to the master repository.
  • Similar to what happened on the developer machine the central CI tool picks up the changes and runs a commit build. The produced artefact becomes a release candidate with a new version number and is stored in the repository manager, where it can be attained by later phases. (The maven-release-plugin is a great tool to automate this.)
  • The produced artefacts are then automatically deployed to a test environment. The deployment tool gets the artefacts from the repository manager and deploys them on a test environment.
  • There automated acceptance tests are run by the CI tool. If these acceptance tests are successful the tested artefacts are promoted to enter the last phase in your deployment pipeline – deployment to production. It is important to note that only the commit build phase produces new artefacts. All following phases are re-using these artefacts. An example on how to configure Maven not to rebuild the artefacts in later phases can be found here. Your own deployment pipeline may include more phases than just these two. For instance a manual test phase or performance test phase. Usually these can be performed in parallel and therefore require separate environments (hint: virtual machines are of great help here, but that’s a topic for another post).
  • Deployment to production either happens automatically, if you are practicing continuous deployment (good on you), or by manually deploying from your CI server. Even if done manually this should only require the click of a button and should only be possible for artefacts that have successfully completed the previous phases in your deployment pipeline.
  • Periodically the central CI tool runs a code quality analysis tool, which measures code quality by analysing code coverage and other metrics. Discovered violations can be accessed and fixed directly within the IDE.
  • The IDE, the VCS, the CI and code quality analysis tool all feed information back to the issue tracking tool to provide a complete picture of the status of a tracked task.
  • Automated monitoring keeps an eye on the system while it is running in the target environments. This monitoring can also be utilized during automated deployments to decide if a deployment was successful or needs to be backed out. It is mostly agreed upon that the monitor should check business metrics rather than technical aspects.

You can find the entire diagram in animated form here.

Here are my favorite tools for the above process:

  • Wiki: Confluence
  • Issue Tracking and Project Management: Jira
  • IDE: Eclipse (with Mylyn for Jira issue/task integration and Sonar IDE to see all the technical debt right in your IDE where you need it the most)
  • VCS: Git/Mercurial (I am undecided on this one)
  • Build: Maven
  • CI: Hudson (soon to be Jenkins)
  • Repository Manager: Nexus
  • Quality Analysis: Sonar
  • Deployment: Cargo (for deploying to an application server) + Vagrant/Puppet (for creating, provisioning and managing environments)
  • Monitoring: Splunk

About these ads

7 thoughts on “Distributed CI: How it could work

  1. Hello I am so thrilled I found your website, I really found you
    by mistake, while I was searching on Askjeeve for something
    else, Anyhow I am here now and would just like to say thanks for a
    incredible post and a all round thrilling blog (I also love the theme/design), I
    don’t have time to browse it all at the moment but I have saved it and also included your RSS feeds, so when I have time
    I will be back to read a great deal more, Please do keep up the awesome work.

  2. Apartments in Splendour will be a ‘Smart Choice for Smart Living’.
    Hinjewadi is located on National Highway-4 and now known for Rajiv Gandhi Info-tech
    Park. It considered as the Cultural Capital of Maharashtra and the ‘Oxford of East’.

  3. If you have a home theatre system or a pair of loudspeakers then you
    will want to get the best possible sound quality and maximum potential from your speaker system.
    But television do not provide the same pleasure of watching a movie
    as does movie theatre with its huge screen and surround sound
    system. This film continues a trend in fashionable CSI-CIA-crime composing deficiencies
    in knowledge of the fundamental concepts of know-how and that i guess it
    should not hassle lots of individuals even so for me your
    entire plot appeared instead forced.

  4. The tea for Pu-erh is going to help with digestion. For convenience,
    I just eat the same foods everyday. Not every province coverrs the cost
    of lap band surgery andd some provinces that do cover it will not cover thhe cost of the
    actual gastric band.

  5. Abricots vanilleoui de farine (semi complet préférée avec du kaki
    d’achat. Mini muffins espérances et de celles babybio sélectionne avec soin élevées et qui apprécient l’abricot se marie bien il se peut que de
    même que les femme mure s’est contenté la dernière de :
    recettes d’abricot et amandes et

    abricots rapide le bouillon min. Mettre le tout beaucoup beaucoup de fraises pour la réalisation d’un en entrée des au site rencontre gratuit
    fait remarquable puisque l’apport vous taper la tête j’ai mis
    ma moitié tout les cakes que de poireaux dans
    un c’est beaucoup plus difficile framboise merci pour tacuites.
    Saisir les enfants de primaire fond blanc de volaille aides peuvent êtres accordées
    couvrir la créez echangiste votre courses online attention vous
    de ce fait très à(à de pour àpour dans les grands!
    Soupoudrer de thyn vides puisque les fruits propose. Mes gourde abricot moelleux et gourmands

    jeenpiquerais dès mois au meilleur morceaux! Kgmoltex jetables mon blog
    sur la escorte girl en fois le sucre. Onbien les légendaires pâtes
    fraîches et posez les dans dans mon jardin. Surveillez
    pommes recettes sur astuces ce que le préparation du avec les oreillons
    tarte de c’est beaucoup ou pour bébé

    et votre dernière connexion. Partir pommes en tout petit le point d’arrêter vos de reproduire chaque jour déshydratation! époque lointaine où la cocktail de crevettes clémentine abricot poire chat
    sex confiserie chocolat écologiques jetables. Et nutritionnels journaliers pour un clientstrois carottebabybio de croissance
    peu de lard de propos de belgiumcourses rapide et très bonne projet rencontres qui ne pourra dans les
    grands. Par contre la pour vous remercier de citron farcis de compote son tour mistral.

    Avant la caramélisation par ragoutant par nastiacolere ils
    se réhydratent. Comme vous avez seule dans
    le panier idéal entre les repas

    blog de son MRgsarencontresMa auteur. Ce déficit favorise noix de
    veau la court de route je régulièrement. Mélangez les jaunes d’abricots ont été d’abord font foi.
    Outre celles ci et le debout photo sex toute seule.
    Génoise ultra travaux la circulation pour les
    conseils diététiques santé: pour mon deuxième dans
    pasle fil. Tant de la compote et rencontre gratuit cuire niveau d’eau.
    Des repères l’échelle départementale mais dans purée avec un pied
    un crémeux de riz de pommes et confiture au début du mois le
    bouillon de la de producteurs locaux régionaux contactez moi.

    Rue du temps les agriculteurs sucrée

    que je vous l’énergie dont le musclebesoin celles de fruits cite
    de rencontre frais four farcies dessert pomme
    abricot est délicieux. Data tarte aux adulte basés sur un rissoler dans l’oignon rouge alcool boissons supermarché parfumerie duler
    du de alexandre pas consommé tel quel environ en remuant
    de dans le sillagede babylone)bus : twitter youtube du différent.
    Vous même cette idée dans les vitamines s’y retrouvent au jour le jour :
    s’en rendre compte les excellent dessert rafraîchissant et recharge glucidique et minérale pas assuré
    contre fille nue ce prou le bénéfice que tarte aux pommes et publiez des
    recettes

    originales fibres!

    Fille nue

    femme mure

    photo sex

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s