DevOps Zone is brought to you in partnership with:

Matt is the Group Leader of Research Application Development in the Research Informatics Division of Information Sciences at St. Jude Children's Research Hospital in Memphis, Tennessee. Matt has been developing and supporting enterprise Java applications in support of life sciences research for St. Jude since 2001. Matt is a committer to multiple open source projects and is the founding member of the Memphis/Mid-South Java User Group. Matt is also a regular speaker on the No Fluff Just Stuff symposium series tour (as well as other major conferences), and his articles have appeared in GroovyMag and NFJS the Magazine. His current areas of interest include lean/agile software development, modularity and OSGi, mobile application development (iPhone/iPad/Android), web development (HTML5, etc.), and Groovy/Grails. Matt has posted 44 posts at DZone. You can read more from them at their website. View Full User Profile

Build Scripts? We Don't Need No Stinkin' Build Scripts!

10.08.2010
| 17391 views |
  • submit to reddit
Ahh, famous last words of the close-minded programmer. Solutions abound for automating the build process in software projects: Ant, Gant, Maven, Gradle, Rake, Raven, NAnt, MSBuild, Make, etc. There are ample HOWTO's available on the web that will teach you how to get started using one or more of these tools. Unfortunately, not much is said about why we ought to automate the build process. Don't get me wrong, several smart people have written on this topic. However, even being the professional googler that I am, I've had a hard time digging these resources up.

If your undergraduate studies were anything like mine, you found yourself woefully unprepared for the work of a good software craftsman. It isn't that my education was bad - it was actually quite good and I wouldn't trade the experience. In fact, it prepared me very well to begin a career as a professional computer scientist. However, life as a software craftsman, delivering value through working software to customers day in and day out, and life as a computer scientist doing academic research are two vastly different occupations.

What we've got then is a crop of new developers that don't know, among other things, what automated build systems are and why they are important. Lest I continue to rant on the state of software engineering education, let's jump right into it.

Repeatable Results

I struggled a bit with what to put up front. What won out was the idea that we're seeking repeatable results. We want a process in place that when handed the raw materials that make up our application (source code, database scripts, configuration files, stylesheets, web page templates, images, etc.), an identical package is produced every time, regardless of who starts the process or what machine they use. Interestingly enough, this is precisely one of the reasons that we write software. Given some set of user inputs, the applications that we write produce a repeatable outcome for our customers. One common motivator for automating business processes with software is to reduce error rates. It certainly seems like the business understands that humans make errors. Why would we as developers think we're any better? (see You're a Bad Programmer. Embrace It.)

And yet every modern integrated development environment (IDE) of which I'm aware is capable of producing a deployable package, given proper configuration of course. I can only assume that this functionality stays around because we're using it! The problem with that is that my IDE is most certainly configured a bit differently from yours. And it's quite possible that I perform the sequence of build steps a bit differently from you. And it's also possible that I have different versions of 3rd party libraries and frameworks than you. What's that you say? A checklist? Great! Just make sure that as your process changes that the checklist always gets updated and that everyone always has the same version, etc., etc., etc.

The bottom line is that you need to treat your build process like a first-class citizen in your software. The best way to ensure that your build process is consistent and repeatable is to have everyone on your team use the same automated script to build the software in any and every situation. If it ever gets out of synch, someone will raise a flag almost immediately. However, if you're all doing it your own way, you may not discover problems until release day.

Conserve Your Concentration

Don't forget the point that I raised in my last article. According to Neal Ford, "...performing simple, repetitive tasks by hand makes you dumber, and it steals part of your concentration, which is your most productive asset." What's more mundane and repetitive than slapping together a deployment artifact through brute force? Let machines do what machines are good at, and you focus on solving problems that require intellect. The more you automate, the more you'll be able to focus, and the more bandwidth you'll have to work through the increasingly hard problems confronting us as software craftsmen.

Lower Your Bus Number

So you've gotten clever and made someone the build master, eh? Everyone else gets to focus on real work while this guy periodically slaves away in the corner putting together the next deployment. Good idea? Sure, as long as he never gets hit by a bus the day of a release (and as long as you're not that guy!). How do you release software when the only person that can build it has gone AWOL? No, automate your build so that anyone can perform that all important step on any machine at any time.

Enable Continuous Integration

Continuous integration is one of the most important feedback loops in agile software development. It is the practice of having a server tool continuously monitor your team's source code repository for changes. When a change is detected, it checks out the latest copy of the code, builds the software, executes any available automated tests, and then generates a report of the results. The very best teams have this report sent to each member via email, IM, etc. Unfortunately, your continuous integration server doesn't have a "mini me" hiding behind a curtain pulling levers and pushing buttons. It's just a computer, and therefore it needs code to execute. You simply have to automate your build to get the benefits of continuous integration.

Improve On-Boarding

Consider the task of orienting a new team member to your project. On the one-hand, you have a mess of source directories and 3rd-party dependencies with a complicated IDE-based build and a 30-item checklist to get from where you are to running software in roughly one hour. On the other-hand, you have a project that follows the nearly ubiquitous Maven project structure and an automated build script that gets you from where you are to running software in roughly 3 minutes. Which new team member is going to have an easier time ramping up on the project? Starting to get the picture?

Summary

The benefits of automating your build are many - I've only touched on a few of them here. My main focus has been on decreasing the costs and lowering the risks of the build process. A consistent, repeatable build that anybody (including a server and the new guy) can perform at any time on any machine without personally performing any of its steps is certainly cheaper and less risky.

I hope that I've been able to convince you of build automation merits; or, if you're already convinced, that I can help you articulate the benefits to someone that isn't. Now, go forth and script your build!
 
Published at DZone with permission of its author, Matt Stine.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Dean Schulze replied on Fri, 2010/10/08 - 1:07pm

This article:

    http://www.dzone.com/links/r/java_build_systems_a_sad_state_of_affairs.html

 is a good summary of build tools.  The comments include more build tool alternatives and impressions of them.  I especially like the comment from Nello summarizing his experience with certain build tools.

 

 

 

Dean Schulze replied on Fri, 2010/10/08 - 1:11pm

HTML tags aren't working.  I tried using an href in the previous post and it didn't recognize it.

Dimitar Dimitrov replied on Sat, 2010/10/09 - 7:12am

I think that an old blogpost of mine can add to this article.

Using a build tool is usually a recommended practice, but that doesn't always have to be the case. Sometimes with one-off applications, having a decent IDE config that your CI server can use is just fine (say TeamCity build server, van build IDEA and VisualStudio projects).

Also, with the years I've seen that knowing your build tool and having experience with build processes has much bigger impact on the build process quality than which tool do you actually use.

Jared Richardson replied on Sat, 2010/10/09 - 9:16am in response to: Dimitar Dimitrov

Hi Dimitar,

Once you've got a good build infrastructure in place, and you've written several build scripts for other projects, I think you'll find that adding a build script for a smaller or one off project isn't a big deal.

In my opinion and experience not having a good build script is a short cut that costs us more time than we realize.

I've heard Andy Hunt say on more than one occasion "Do it right from day one or you never will." He's right... after the project is rolling and in full "Get it done!" mode, we rarely have time to go back and add in the key infrastructure bits. We just keep wasting a little extra time every day without realizing how it adds up.

I'll stop preaching now. :) One last analogy... when is it easiest to fix a cracked foundation on a house? Just after you've poured the foundation (and discovered the crack)? Or after you've built a house on top of it? You can always fix somethings later (I do believe in YAGNI), but I also firmly believe that some infrastructure bits shouldn't be skipped.

Dimitar Dimitrov replied on Sat, 2010/10/09 - 11:15am

Jared, I actually agree with your sentiment, but to play a devil's advocate: if an intern is in for 2 weeks I'd rather have him work on some project where they can get exposure to the business domain rather than spend the time learning how to do build scripts. Even something as easy as learning how to set up Maven archetypes takes some time for a beginner (mostly due to all kind of restrictions in corp. environments).

Typically such projects end up as useful utilities, where everybody cares about the end result and nobody plans to maintain the source as it would be easier to rewrite it. You might argue that this is disservice to the intern in question, but I would say that this kind of exposure to real working environment, is more valuable than trying to teach them how to be good programmers in 14 days.

Brian Mowbray replied on Sun, 2010/10/10 - 9:06pm

Great article. You pretty well covered it. Another topic of importance is to always, mimic as close as possible to your production evironment as soon as feasible. This leads to changes in your dev IDE that may have to be globally updated. Who makes, and maintaines the changes during your build SDLC for all the client impacted? Deveolper or CI Engineer?

Matt Stine replied on Sun, 2010/10/10 - 10:45pm in response to: Brian Mowbray

@Brian - excellent point. I'll be talking about this in upcoming articles on continuous integration. I didn't bring it up in this article because I'm intentionally taking baby steps through the whole automation story.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.