16.3 GNU Autoconf
Even though C source code is usually fairly portable, differences on each platform make it impossible to compile most packages with a single Makefile. Early solutions to this problem were to provide individual Makefiles for every operating system or to provide a Makefile that was easy to modify. This approach evolved into scripts that generate Makefiles based on an analysis of the system used to build the package.
GNU autoconf is a popular system for automatic Makefile generation. Packages using this system come with files named configure, Makefile.in, and config.h.in. The .in files are templates; the idea is to run the configure script in order to discover the characteristics of your system, and then make substitutions in the .in files to create the real build files. For the end user, it’s easy; to generate a Makefile from Makefile.in, run configure:
$ ./configure
You should get a lot of diagnostic output as the script checks your system for prerequisites. If all goes well, configure creates one or more Makefiles and a config.h file, as well as a cache file (config.cache), so that it doesn’t need to run certain tests again.
Now you can run make to compile the package. A successful configure step doesn’t necessarily mean that the make step will work, but the chances are pretty good. (See Section 16.6 for tips on troubleshooting failed configures and compiles.)
Let’s get some firsthand experience with the process.
At this point, you must have all of the required build tools available on your system. For Debian and Ubuntu, the easiest way to ensure this is to install the build-essential package; in Fedora-like systems, use the “Development Tools” groupinstall.
16.3.1 An Autoconf Example
Before discussing how you can change the behavior of autoconf, let’s look at a simple example so that you know what to expect. You’ll install the GNU coreutils package in your own home directory (to make sure that you don’t mess up your system). Get the package from *http://ftp.gnu.org/gnu/coreutils/*(the latest version is usually the best), unpack it, change to its directory, and configure it like this:
$ ./configure --prefix=$HOME/mycoreutils
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
--snip--
config.status: executing po-directories commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
Now run make:
$ make
GEN lib/alloca.h
GEN lib/c++defs.h
--snip--
make[2]: Leaving directory '/home/juser/coreutils-8.32/gnulib-tests'
make[1]: Leaving directory '/home/juser/coreutils-8.32'
Next, try to run one of the executables you just created, such as ./src/ls, and try running make check to run a series of tests on the package. (This might take a while, but it’s interesting to see.)
Finally, you’re ready to install the package. Do a dry run with make -n first to see what make install does without actually doing the install:
$ make -n install
Browse through the output, and if nothing seems strange (such as the package installing anywhere other than your mycoreutils directory), do the install for real:
$ make install
You should now have a subdirectory named mycoreutils in your home directory that contains bin, share, and other subdirectories. Check out some of the programs in bin (you just built many of the basic tools that you learned about in Chapter 2). Finally, because you configured the mycoreutils directory to be independent of the rest of your system, you can remove it completely without worrying about causing damage.
16.3.2 Installation Using a Packaging Tool
On most distributions, it’s possible to install new software as a package that you can maintain later with your distribution’s packaging tools. Debian-based distributions, such as Ubuntu, are perhaps the easiest; rather than running a plain make install, you install the package with the checkinstall utility, as follows:
# checkinstall make install
Running this command shows the settings pertaining to the package that you’re about to build, and gives you the opportunity to change them. When you proceed with the installation, checkinstall keeps track of all of the files to be installed on the system and puts them into a .deb file. You can then use dpkg to install (and remove) the new package.
Creating an RPM package is a little more involved, because you must first create a directory tree for your package(s). You can do this with the rpmdev-setuptree command; when complete, you can use the rpmbuild utility to work through the rest of the steps. It’s best to follow an online tutorial for this process.
16.3.3 configure Script Options
You’ve just seen one of the most useful options for the configure script: using --prefix to specify the installation directory. By default, the install target from an autoconf-generated Makefile uses a prefix of /usr/local—that is, binary programs go in /usr/local/bin, libraries go in /usr/local/lib, and so on. You will often want to change that prefix like this:
$ ./configure --prefix=new_prefix
Most versions of configure have a --help option that lists other configuration options. Unfortunately, the list is usually so long that it’s sometimes hard to figure out what might be important, so here are some essential options:
--bindir=directory Installs executables in directory.
--sbindir=directory Installs system executables in directory.
--libdir=directory Installs libraries in directory.
--disable-shared Prevents the package from building shared libraries. Depending on the library, this can save hassles later on (see Section 15.1.3).
--with-package=directory Tells configure that package is in directory. This is handy when a necessary library is in a nonstandard location. Unfortunately, not all configure scripts recognize this type of option, and it can be difficult to determine the exact syntax.
You can create separate build directories if you want to experiment with some of these options. To do so, create a new directory anywhere on the system and, from that directory, run the configure script in the original package source code directory. You’ll find that configure then makes a symbolic link farm in your new build directory, where all of the links point back to the source tree in the original package directory. (Some developers prefer that you build packages this way, because the original source tree is never modified. This is also useful if you want to build for more than one platform or configuration option set using the same source package.)
16.3.4 Environment Variables
You can influence configure with environment variables that the configure script puts into make variables. The most important ones are CPPFLAGS, CFLAGS, and LDFLAGS. But be aware that configure can be very picky about environment variables. For example, you should normally use CPPFLAGS instead of CFLAGS for header file directories, because configure often runs the preprocessor independently of the compiler.
In bash, the easiest way to send an environment variable to configure is by placing the variable assignment in front of ./configure on the command line. For example, to define a DEBUG macro for the preprocessor, use this command:
$ CPPFLAGS=-DDEBUG ./configure
You can also pass a variable as an option to configure; for example:
$ ./configure CPPFLAGS=-DDEBUG
Environment variables are especially handy when configure doesn’t know where to look for third-party include files and libraries. For example, to make the preprocessor search in include_dir, run this command:
$ CPPFLAGS=-Iinclude_dir ./configure
As shown in Section 15.2.6, to make the linker look in lib_dir, use this command:
$ LDFLAGS=-Llib_dir ./configure
If lib_dir has shared libraries (see Section 15.1.3), the previous command probably won’t set the runtime dynamic linker path. In that case, use the -rpath linker option in addition to -L:
$ LDFLAGS="-Llib_dir -Wl,-rpath=lib_dir" ./configure
Be careful when setting variables. A small slip can trip up the compiler and cause configure to fail. For example, say you forget the - in -I, as shown here:
$ CPPFLAGS=Iinclude_dir ./configure
This yields an error like this:
configure: error: C compiler cannot create executables
See 'config.log' for more details
Digging through the config.log generated from this failed attempt yields this:
configure:5037: checking whether the C compiler works
configure:5059: gcc Iinclude_dir conftest.c >&5
gcc: error: Iinclude_dir: No such file or directory
configure:5063: $? = 1
configure:5101: result: no
16.3.5 Autoconf Targets
Once you get configure working, you’ll find that the Makefile it generates has a number of useful targets in addition to the standard all and install:
make clean As described in Chapter 15, this removes all object files, executables, and libraries.
make distclean This is similar to make clean except it removes all automatically generated files, including Makefiles, config.h, config.log, and so on. The idea is that the source tree should look like a newly unpacked distribution after running make distclean.
make check Some packages come with a battery of tests to verify that the compiled programs work properly; the command make check runs those tests.
make install-strip This is like make install except it strips the symbol table and other debugging information from executables and libraries when installing. Stripped binaries require much less space.
16.3.6 Autoconf Logfiles
If something goes wrong during the configure process and the cause isn’t obvious, you can examine config.log to find the problem. Unfortunately, config.log is often a gigantic file, which can make it difficult to locate the exact source of the issue.
The general approach in this situation is to go to the very end of config.log (for example, by typing a capital G in less) and then page back up until you see the problem. However, there’s still a lot of stuff at the end because configure dumps its entire environment there, including output variables, cache variables, and other definitions. So, rather than p. 393 going to the end and paging up, go to the end and search backward for a string, such as for more details or some other fragment of text near the end of the failed configure output. (Remember, you can initiate a reverse search in less with the ? command.) There’s a good chance the error will be right above what your search finds.
16.3.7 pkg-config
The multitude of third-party libraries on a system means that keeping all of them in a common location can be messy. However, installing each with a separate prefix can lead to problems with building packages that require those third-party libraries. For example, if you want to compile OpenSSH, you need the OpenSSL library. How do you tell the OpenSSH configuration process the location of the OpenSSL libraries and which ones are required?
Many libraries now use the pkg-config program not only to advertise the locations of their include files and libraries but also to specify the exact flags you need to compile and link a program. The syntax is as follows:
$ pkg-config options package1 package2 ...
For example, to find the libraries required for a popular compression library, you can run this command:
$ pkg-config --libs zlib
The output should look something like this:
-lz
To see all libraries that pkg-config knows about, including a brief description of each, run this command:
$ pkg-config --list-all
How pkg-config Works
If you look behind the scenes, you’ll find that pkg-config finds package information by reading configuration files that end with .pc. For example, here’s openssl.pc for the OpenSSL socket library, as seen on an Ubuntu system (located in /usr/lib/x86_64-linux-gnu/pkgconfig):
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib/x86_64-linux-gnu
includedir=${prefix}/include
Name: OpenSSL
Description: Secure Sockets Layer and cryptography libraries and tools
Version: 1.1.1f
Requires:
Libs: -L${libdir} -lssl -lcrypto
Libs.private: -ldl -lz
Cflags: -I${includedir} exec_prefix=${prefix}
You can change this file, for example, by adding -Wl,-rpath=${libdir} to the library flags to set a runtime library search path. However, the bigger question is how pkg-config finds the .pc files in the first place. By default, pkg-config looks in the lib/pkgconfig directory of its installation prefix. For example, a pkg-config installed with a /usr/local prefix looks in /usr/local/lib/pkgconfig.
You won’t see .pc files for many packages unless you install the development packages. For example, to get openssl.pc on an Ubuntu system, you must install the libssl-dev package.
How to Install pkg-config Files in Nonstandard Locations
Unfortunately, by default, pkg-config doesn’t read any .pc files outside its installation prefix. This means that a .pc file that’s in a nonstandard location, such as /opt/openssl/lib/pkgconfig/openssl.pc, will be out of the reach of any stock pkg-config installation. There are two basic ways to make .pc files available outside the pkg-config installation prefix:
- Make symbolic links (or copies) from the actual .pc files to the central pkgconfig directory.
- Set your
PKG_CONFIG_PATHenvironment variable to include any extra pkgconfig directories. This strategy does not work well on a systemwide basis.