Notes on the software build process

Last updated on Jun 14, 2020 7 min read Software Development

The build is the pulse of any software development activity and a good build system facilitates quality software development. My current thoughts on a build process are as follows

Build Machine

The build machine is a dedicated physical or virtual machine whose sole purpose is to build your product. It should not be used for development or QA activities. Here are some points to keep in mind when setting up the build machine.

Use a physical or virtual build machine depending on how often you plan to build and whether the build is initiated manually or automatically. A virtual machine has the advantage that it can do the build, copy the files to a file server and then it can be reset to it’s initial clean state. Also a virtual machine does not require an expensive machine that will be idle most of the time. On the other hand if the build is initiated automatically or happens multiple times during the day, it is best to use a dedicated physical machine.
- Another possibility is to use one Virtual Server that hosts build machines for multiple projects, this way the physical hardware is used optimally.
Keep the number of softwares installed on the build machine to a minimum.
Should have ample storage space, if the builds will also be archived on the build machine itself.
Build machine should be fast to keep build times low. This is especially important in C++ like projects where typical build times can be 30+ minutes. Also having a fast build process means that the time from a checkin to a build pass/fail result is minimized thus ensuring that time is not wasted waiting for the build results.
Have some sort of remote desktop capability installed on the machine. This proves to be invaluable when you need to do emergency builds from home at night.

Process

The build process is more than just a compilation of the code that is done on a developer’s workstation, a good build process is able to generate the final set of artifacts that is sent to the user in one step. For e.g. a build process for a desktop application would build the installer, documentation, licenses etc. for the entire product, in case of a web application the build will compile the code files, supporting assets and deploy to a test or staging server. A good build system should be configurable and able to build multiple editions of the product.

There are a lot of options available for build tools, ranging from the simple make files and batch files to continuous integration systems like CruiseControl, in between are the new age build systems like Ant, NAnt, MSBuild etc.

Here are some tasks that need to be done when creating a build system. Not all of them will be relevant to all projects and there will be some steps that are specific to individual projects. Also the specifics on how to achieve the tasks will be different based on the build tool you use.

Clean up/Create the folders where the project output will be generated.
Get the latest sources from source control.
Update the build numbers in code and documentation.
Tag the code in source control.
Build the code.
Run automated unit tests.
Run automated functional tests.
Run automated regression tests.
Build the documentation.
Generate release notes
Build the installers.
Copy the artifacts to a folder for archiving or to the machine that is used for archiving the builds. Make sure that the folder are named appropriately.
Deploy the product to a test server.
Send out build status email.
Clean up and temp files and folders.

The following sections talk in detail about some of the tasks mentioned above

Build numbers

The build number should be of the format major.minor.patch.build (Note that this is slightly different from the .Net recommendations)

major and minor revision numbers are generally decided by the product marketing. The patch and build numbers are generally used by the engineering team to identify the product.

The build number should be a monotonically increasing sequence across your mainline code and is independent of the major and minor version numbers. For a branch the build number should remain constant and the patch version should increment with each labeled build, the labeled builds on a branch would usually correspond to updates to already released products, which is why the major minor and build numbers remain the same.

For e.g.

Mainline - 1.0.0.100, 1.0.0.101 … 1.5.0.125 … 2.0.0.180

In the above case the build number starts off with 100 and each subsequent labeled build of the mainline has the next higher build number, this remains true even as the major and minor version numbers progress to 1.5 and 2.0

Branch - 1.5.0.125, 1.5.1.125, 1.5.2.125

In the above case we create a branch for the 1.5.0.125 build which is a release branch. Any subsequent bug fixes/minor changes to this branch get labeled with increasing patch numbers while the major, minor and build numbers remain constant, this makes it clear that the builds are just patch updates to the release.

There are many alternatives to the above scheme for build numbering, one of them is major.minor.build.patch, the problem with this one is that the user facing version number of the product major,minor and patch are not contiguous, in case of major.minor.patch.build, we can simply drop off the last part of the version string and get the user facing version string major.minor.patch

Tagging code

If a labeled build is being made as opposed to a test build, then the code should be tagged in source control. This usual convention is to just tag the code as Build_ for e.g. Tags/Build_1.5.0.125

When a build is being released to the customer, then in addition to the tag, a branch should also be created for the build on which any subsequent bug fixes and minor changes can be made without affecting the mainline. for e.g. Branches/Release_1.5.0.125

Build the code

When building the code, consider building both the Debug and Release versions of the code, the Debug versions are useful when you have to debug a particular version of the product at a later stage. Although the code can always be retrieved from source control, it can be time consuming in case the code base size is big.

Automated tests

If possible we should run automated tests as soon as the build is made, a failing test should result in a failed build. Most build tools have integration with unit testing tools like NAnt, Ant etc. also batch files and other custom executables can always be used to invoke the test scripts.

Building documentation

If the product includes documentation then it should also be built along with the build, further the documentation should be labeled as well indicating the version of the product for which the documentation is written. Again standard tools like Microsoft Help Compiler, NDoc, SandCastle can be used to generate the documentation.

Release notes

Release notes should be automatically generated for each build, this can be done by either pulling in the list of changes from the source control or by pulling in the task ids that have been implemented since the last build from whatever issue tracking system you are using.

Building installers

For most products a installer will be required that will deploy the product on the users machine/server.

Some of the options for building installlers are

Installshield - Very costly and bulky.
Wise - Less expensive and not so bulky.
NSIS- Free, but not as many features as Installshield and Wise
Wix - Microsoft has open sourced the toolkit that allows generation of MSI files from xml.

Whatever tool is used, this should be integrated into the build system so that as a result of a successful build the installer is created as well.

Build archiving

Every labeled build that is created should be archived on some machine so that it can be used at a later stage for testing and debugging.

References

Some good references for build processes and configuration management