Assembly & Artifact Storage

In the process of implementing a more advanced process for our build server the question ultimately arose, "What do we do with the assemblies and other artifacts that are created?" The easiest and perhaps the best solution is to store them in your version control repository along with everything else, provided you are using a system capable of performing binary deltas on your files, e.g. Subversion.

In our situation we wanted to build artifacts and store them in a tagged directory of SVN in order to easily get the exact binaries that were created at any point in time. As we implemented the solution and performed a handful of builds we watched the repository start to grow at an unacceptable rate which necessitated a change in our methodologies.

There were several reasons for storing all build artifacts in SVN:

  • Ability to get exact revision of builds at a point in time.
  • Make each project self-contained, with all necessary artifacts, build scripts, build tools, etc., all linked using svn:externals.
  • Easy geo-distribution of any team member. This is useful for me as I work at the office and at home, as well as anyone else working offsite.

David Bell wrote an excellent article on the ins and outs of Subversion storage. His example metrics were a little bit skewed because of the size of the files he was working with--100 MB and larger. Our largest binaries are only a few hundred KB in size. Nevertheless it is an worthwhile read.

For our solution we did an "efficient check-in" as David Bell terms it in his article. The efficient check-in scenario follows these general steps:

  1. Checkout your "artifacts" directory from Subversion and create a local, working copy - we'll call this the "commit" directory.
  2. Perform your build which creates artifacts in your "compile" directory.
  3. Recursively remove any files or directories from the commit directory that do not exist in the compile directory.
  4. Copy all files and directories from the compile to the commit directory.
  5. Commit.

In this fashion you get delta-based processing on all binary files. This allows you to store only a few KB per commit rather than a few MB.