Jun 252010

Waiting for unit compilations to finish during development, particularly with G++ and heavy optimizations can be quite time consuming. A run of make distcheck escalates the problem, because all files during a dist need to be rebuilt. Small errors triggered late in the dist build might require several time consuming restarts of the build process.

With ccache, a tool exists that can majorly speed up this process. I’ve been using it successfully for several years now. A related development aid is colorgcc which is a colorization wrapper for gcc. This also is under heavy use for my development. It turns out that some tweaks are needed to get it working together with ccache though.

Here’s how to use the tools in combination to speed up regular project builds and also distcheck:

  • Get and install ccache.
    Wrapping compiler invocations with ccache will cache recompilations of the same units. ccache needs a directory to store the cache contents; this defaults to ~/.ccache which might not be the best choice for network homes or encrypted homes. Using /tmp is also suboptimal since it is usually cleaned upon reboots. So I’d recommend to put this into your .bashrc:

    export CCACHE_DIR="/var/tmp/ccache-$USER"

    This will retain the compiler cache across reboots in /var/tmp.

  • Get and install colorgcc.
    Then some patching is in order, especially if you use multiple versions of gcc and need ccache integration. Based on colorgcc from Ubuntu-9.10, I’ve prepared a patch for this (it has been sent to Jamie Moyers some while ago) to be found here: http://testbit.eu/~timj/blogstuff/xyv-colorgcc.diffTo briefly summarize its effects:

    • This allows colorgcc to be used through a bunch of links, e.g. by creating:
      ln -s colorgcc colorg++
      ln -s colorgcc colorgcc-4.4
      ln -s colorgcc colorg++-4.4

      Invoking the script through those links will cause it to strip the ‘^color’ prefix and invoke the respective compiler.

    • Using this patch, /usr/bin/ccache will be used to wrap the compiler invocation if it exists.
    • The patch also contains some UTF-8 quotation fixes needed for gcc-4.4 support.
    • Finally, the patch enables colorgcc to correctly deal with g++ error message that span multiple lines (e.g. template error messages).
  • Use colorgcc as a compiler replacement for project builds:
    rm -f config.cache
    ./autogen.sh CC=colorgcc-4.4 CXX=colorg++-4.4
    nice make all -j4
  • Use colorgcc as a compiler replacement for distcheck builds:
    CC=colorgcc CXX=colorg++ nice make all distcheck

Finally, you might want to tweak the colors of colorgcc’s output which can be adjusted in /etc/colorgcc/colorgccrc. FYI, here’s the color setup I prefer to use in a black terminal window:

  srcColor:             bold white
  introColor:           bold magenta
  warningFileNameColor: cyan
  warningNumberColor:   bold cyan
  warningMessageColor:  yellow
  errorFileNameColor:   cyan
  errorNumberColor:     bold cyan
  errorMessageColor:    bold yellow

UPDATE: Fixed ccache default dir thanks to anonymous.

Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookFlattr the authorBuffer this pageShare on RedditDigg thisShare on VKShare on YummlyPin on PinterestShare on StumbleUponShare on TumblrPrint this pageEmail this to someone

[suffusion-the-author display='description']

  5 Responses to “Using distcheck with ccache + colorgcc”

  1. ccache normally defaults to ~/.ccache, not /tmp.

  2. I recommend to use these as well:

    export CCACHE_NOSTATS=1
    export CCACHE_COMPRESS=1

    Disabling statistics you likely never look at avoids disk IO.
    Compressing reduces the size of the cache on disk, and speeds up reading.

    • Thanks for the input Christian.
      Allthough on my Ubuntu, compression seems to be enabled by default, and during long builds I do occasionally watch the output of:
      $ watch -n1 ccache -s

  3. you can use GilCC which is a Ruby tool that will convert GCC output to color in real-time. Right now you have two choices: Perl script (colorGCC) or GilCC and if you already work with Ruby you will like GilCC.

    Unique to GilCC; GilCC has warning and errors counters and also shows compile time, very handy when you are trying to improve things. Because it is in Ruby it is cross platform. It is flexible and you can add more gems to customize it anyway you want.

    The link to the download page is here.


 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>