GNU Build System - Recursive vs Non-revursive Automake

Compiling meme

In the first Getting Started article, all files were located in the same directory. In most projects it's a good idea to organize the code into different directories and/or sub-directories. GNU Build System offers two ways to achieve this. Those are recursive and non-recursive. We will use the two techniques in a simple project and talk a bit about the pros and cons of each.

Laying out the code

As usual and to save ourselves some time we will start by using the code from the Getting Started article. To make sure we are all on the same page, I invite you to go through that article first, in case you didn't already. Otherwise you can checkout the getting-started branch from the support repository.

Now let's say that our main.c file depends on a library that we are going to put in a subdirectory called lib. In the lib directory we are going to put a hello.h header file and a hello.c source file. The structure of the project's directory should look like this.

$ tree
.
├─ configure.ac
├─ lib
│  ├─ hello.c
│  └─ hello.h
├─ LICENSE
├─ main.c
├─ Makefile.am
└─ README.md

1 directory, 7 files

Note: In case you ran the autoreconf -i or ./configure command as explained in the Getting started article. The output of the command tree will be different than the above. In order to eliminate the generated files and have the same result. Make sure you run the command git clean -fxd before starting with the instructions of this article.

Now put the following code in the newly created files.

hello.h

#ifndef HELLO_H
#define HELLO_H

void hello ();

#endif /* HELLO_H */

`hello.c`

#include "hello.h"
#include <stdio.h>

void
hello ()
{
  printf("Hello world!\n");
}

The main file (main.c) requires some changes in order to make it use the library function instead.

main.c

#include <stdio.h>
#include <hello.h>

int
main (int argc, char** argv)
{
  hello ();
  return (EXIT_SUCCESS);
}

Now that the code is set up, we will change the configuration files, notably Makefile.am and configure.ac in order to cope with the changes to the code structure.

Since we are now in a multi-directory situation we have to choose between one of the methods offered to us by Automake to manage the dependencies.

There is one change to that I prefer to do right now before we move to the next part. This because this modification is common for both methods. In Makefile.am we have specify for where the included file hello.h is located by putting its location in the hello_CFLAGS variable.

bin_PROGRAMS = hello
hello_SOURCES = main.c
hello_CFLAGS = -I$(top_srcdir)/lib

This should be enough for a code setup. Let's start by trying the recursive approach for building this project.

Note that for each approach, we will start from this state of the project. This means if you try the recursive method and want to try the non-recursive afterwards. You will have to remove the changes you made while following the instructions for the recursive mode. I suggest using a version control tool for this purpose. Or checkout the code from the support repository. I made a branch called rec-vs-non-rec-automake that points to this state of the project.

Recursive Automake

When opting for the recursive option, every directory in the project that contains code files should have an Makefile.am file. Every Makefile.am defines how the code it contains should be compiled in order to generate an executable file or a library. In our case we have two directories containing C code. The top-level directory and the lib directory.

Let's start by writing the Automake file for the lib directory. Since Automake does not allow for generating object files without an explicit target, we will compile the code in the lib directory to generate a static library. The way we do this with Automake is as follows.

noinst_LIBRARIES = libhello.a
libhello_a_SOURCES = hello.c

This basically says that we want to generate a static library libhello.a from the source code in hello.c. The name of the library file has to respect some constraints. It should always start with lib and end with a .a extension. Note that in the second variable the dot is replaced with an underscore. This is because dots are not allowed in variable names.

Since now we are building a library, we should add a AC_PROG_RANLIB macro call in the configure.ac. This is to tell Autoconf to prepare the necessary tools for library building.

AC_INIT([Hello World], [1.0])
AM_INIT_AUTOMAKE([foreign 1.11.1])
AC_PROG_CC
AC_PROG_RANLIB
AC_CONFIG_FILES([Makefile])
AC_OUTPUT

We should also add another item to the AC_CONFIG_FILES because we will have another Makefile that will be generated under the lib directory. With recursive Automake this should be done for all code directories.

Our final configure.ac file is as shown below.

AC_INIT([Hello World], [1.0])
AM_INIT_AUTOMAKE([foreign 1.11.1])
AC_PROG_CC
AC_PROG_RANLIB
AC_CONFIG_FILES([Makefile lib/Makefile])
AC_OUTPUT

The top-level Makefile.am needs tow modifications. In recursive mode every Makefile.am should declare the sub-directories that it contains. In this case the top level directory contains the lib directory. In which case we should declare it by adding it to the SUBDIRS variable.

The second modification is to link the libhello.a library for the hello program to compile. This can be done by adding lib/libhello.a to the variable hello_LDADD.

With these two modifications, the top level Makefile.am will look like this.

SUBDIRS = lib
bin_PROGRAMS = hello
hello_SOURCES = main.c
hello_CFLAGS = -I$(top_srcdir)/lib
hello_LDADD = lib/libhello.a

And we are all set to compile and run.

$ autoreconf -i && ./configure && make && ./hello
...
gcc -I./lib -g -O2   -o hello hello-main.o lib/libhello.a
Hello world!

The code for this part is available in the support repository. Checkout the recursive-automake branch.

Recursive: Pros and Cons

The good thing with this method is that all the directories are treated as a separate module. Dependencies between directories is loosened and each directory has its own Automake file to handle it. The downside though, is that managing multiple Makefile.am files in a deeply nested project can quickly start to become irritating.

The non-recursive flavor

Non-recursive means that we only have to write one Makefile.am file and it will contain all the directives to build the whole project. In this case our Makefile.am would look like this.

bin_PROGRAMS = hello
hello_SOURCES = \
    lib/hello.c \
    main.c
hello_CFLAGS = -I$(top_srcdir)/lib

We just added lib/hello.c to the dependencies of our hello program.

Note: Notice that we used a special syntax just to put every dependency in a separate line using the \ character. When Automake sees the backslash character it ignores the newline that comes just after. Therefore sees the whole thing as one line. If you prefer you can put the dependencies in one line separated with a space like this hello_SOURCES = lib/hello.c main.c. It will work either ways.

While this should be enough to get things working for most of the cases. There is a problem that can happens when a C file in two different directories have the same name. Because by default Automake creates all object files in the project's top-level directory, there will be conflicts while generating object files.

Note: You can try to produce this issue by renaming main.c to hello.c (Don't forget to change that in Makefile.am as well). In this case Automake will try to generate a hello.o for hello.c and another hello.o for lib/hello.c, both in the top level directory. Which is not possible and you will get an error as soon as you run autoreconf -i saying: object 'hello.$(OBJEXT)' created by 'hello.c' and 'lib/hello.c'

Luckily, there is a fix for this problem. We can tell Automake to generate the object file for each *.c in the directory in which it exists. This can be done by adding the option subdir-objects to the AM_INIT_AUTOMAKE macro in configure.ac.

So after modification configure.ac would look like this.

AC_INIT([Hello World], [1.0])
AM_INIT_AUTOMAKE([foreign subdir-objects 1.11.1])
AC_PROG_CC
AC_CONFIG_FILES([Makefile])
AC_OUTPUT

That's it for the non recursive part. Now you can compile and run the program.

$ autoreconf -i && ./configure && make && ./hello
...
gcc  -g -O2   -o hello lib/hello.o main.o
Hello world!

You can get the code for this part from the support repository in the non-recursive-automake branch.

Non-recursive: Pros and Cons

The advantage of the non-recursive approach is that only one Makefile.am needs to be written and managed. A lot of people think that this is the right approach because it keeps things simple without going throughout the hassle of having to generate a separate library for each directory. This can however become problematic when the one Automake file becomes very big due to the complexity of the project.

You can use both!

Many open source projects opt for a hybrid approach. Where parts of the project are recursive and others are not. Which means that the recursivity does not have to go up to the deepest directory of the project. Down to some level, directories can be treated recursively. But directories that are deeper than that, can be handled in a non-recursive manner.

Over and out,

AA