malcolm@commsecure.com.au"> automake"> autoconf"> autoheader"> ]> LCA tutorial MalcolmTredinnick

&malcolm-email;

Wednesday 22 January 2003 The information I would like to convey... Where ∾ and &am; fit into the build process and the roles they fill. Why they are necessary. Brief introduction to each tool. The more difficult bits. Wherein we consider why there are so many tools involved here and what roles they fulfill. &am; ∾ autoheader aclocal make Standard source code build, after unpacking the tarball, will typically proceed as ./configure --prefix=/foo/bar/ make make check su - make install For the maintainer, the first step is the most interesting and fiddly. After that, it is easy. Two areas that can be managed by tools: Configuration details that are specific to the particular build. Generating the repetitive and routine portions of Makefiles. Feature selection Library and other support file locations Machine-specific features compiler flags, function names, ... Install locations All of this detection and flag parsing is done by ∾. A few different sources of tests and command-line parsing code exist. Many common tests are packaged with ∾. Some tests become common enough that they are distributed in third-party macro libraries (*.m4 files). Once-off tests can be coded directly into the configure.in file. Collecting together the macros that come ∾ and third-party packages is done by aclocal. Some details can be recorded in a header file for inclusion throughout the package (typically, config.h). This is handled by a combination of &ah; and ∾. Some information becomes flags to the compiler, linker and other processors: include file paths, some definitions that cannot go into config.h. This is handled by ∾ and &am;. What files need to be built? What does each file depend upon to be built? What are the "types" of these files? This affects installation procedures. Are there any tests that can be run? Determining which files to build, how to install and test them and what their dependencies are is the domain of &am;. At this point in the process, the build is ready to proceed on this particular machine. It is now just a trivial matter of running make and so forth (this was the "uninteresting" portion mentioned earlier, so it is not covered here). Combining all of the steps from the previous slides results in a file that is often called autogen.sh. This file just runs the various autotools programs in an appropriate order. To wit, it is essentially #!/bin/sh aclocal autoheader automake --add-missing autoconf In practice, this file usually includes checks for the correct versions of programs. It may also run things like autopoint (from the gettext package) and other preliminary scripts. The above fragment is the essence of the procedure, though, and is the portion we are concerned about in this tutorial. (This slide was not in the original tutorial. Instead the diagram was drawn up on the whiteboard at the request of some of the participants. I have included it here for completeness and because it adds another explanation of the last few slides.) The previous group of slides describe the situation as the package developer sees it. The minimal number of necessary source files are somewhat magically transformed into a complete package. Recall the earlier sequence: ./configure --prefix=/foo/bar/ make make check su - make install The configure script is the result of running ∾, &am; and friends. The developer will often have more tools at their disposal than the typical builder. Extra tools may be difficult to install or time-consuming to run. Some files change over time, but are not specific to the build environment. They can safely be generated at distribution time. Typically, some files are therefore generated by the developer and distributed with the package. Examples include Documentation Documentation transformation tools can be difficult to install correctly with many pre-requisites. Translation catalogs Support tools not available everywhere. Can be time-consuming and the result is not machine specific. Instruction files for building .deb or .rpm packages. These are changing files but remain fixed for each official release. One school of though: distributing built files is a waste of time, since the means exist to generate them on the target machine. The other side of the coin: All the arguments on the previous slide about being nice to the package builders. Common result: both source and built files are distributed. Users can then regenerate things from source, but this is not required. Note that this last point is not universally true. Some package maintainers will only distribute the built files and not their sources. In practice, this is probably the worst case, although it does make the distributed tarball slightly smaller. Enough philosophy and general concepts! Onto the good stuff... We begin with &am; and work back to ∾. A real-life example, from the gtk-doc package. bin_SCRIPTS = \ gtkdoc-scanobj \ gtkdoc-scangobj \ ... gtkdocdatadir = $(datadir)/gtk-doc gtkdocdata_DATA = \ gtkdoc-common.pl \ gtk-doc.xsl \ devhelp.xsl \ ... pkgconfigdir = $(libdir)/pkgconfig pkgconfig_DATA = gtk-doc.pc EXTRA_DIST= \ MAINTAINERS \ gtk-doc.pc.in \ ... dist-hook: mkdir $(distdir)/doc cp -pr $(srcdir)/doc/* $(distdir)/doc rm -rf $(distdir)/doc/CVS mkdir $(distdir)/db2man cp -pr $(srcdir)/db2man/* $(distdir)/db2man ... Automake knows that different types of files should be treated differently. Data files (foo_DATA) For the build infrastructure, these are the simplest files. Just install them without change. Header files (foo_HEADERS) Public header files. Executable binaries (foo_PROGRAMS) Usually built from a number of sources. Can be stripped. Should be installed as executables. Scripts (foo_SCRIPTS) Installed as executables. Can have their name transformed (the --program-transorm-name flag to configure). Cannot be stripped (so they are different from programs). Manual pages (foo_MAN) and info pages (foo_TEXINFOS) Installed into the appropriate directories. For example, &am; knows that bar.3n goes in section 3 (despite the trailing n). For info documents, it knows to run install-info on the results to put them into the directory, and so forth. Libraries (foo_LIBRARIES and foo_LTLIBRARIES) Specifies that the files are static or shared libraries, possibly created with libtool (not covered in this tutorial). One of the most common operations in a makefile is turning a number of source files into one target file (for example, a program binary). bin_PROGRAMS = bug-buddy bug_buddy_SOURCES = \ $(bb_built_sources) \ bug-buddy.c \ bug-buddy.h \ md5-utils.c \ md5-utils.h \ bugzilla.c \ bugzilla.h \ cell-renderer-uri.c \ cell-renderer-uri.h \ config.c \ gdb-buddy.c \ libglade-buddy.h \ save-buddy.c \ save-buddy.h \ signal-buddy.c \ signal-buddy.h \ united-states-of-bug-buddy.c Aside from the $(bb_built_sources) variable, everything here should be clear. The bug-buddy file is a program and will be installed in the bindir location when make install is run. Continuing with the previous example, the Makefile.am file contains this fragment which defines the bb_built_sources variable mentioned earlier. bb_built_sources = \ bb-marshal.c \ bb-marshal.h bb-marshal.h: $(srcdir)/bb-marshal.list ... bb-marshal.c: $(srcdir)/bb-marshal.list ... The portions replaced with ellipses in the above fragment and unimportant to this discussion. The point here is that anything that &am; cannot interpret as a special instruction, it assumes is a normal Makefile code fragment and copies the literal text into the resulting Makefile. Note also the use of the special $(srcdir) variable, which refers to the current source directory. Remember that the build directory and the source directory may not necessarily be the same, so you cannot just use the "current" directory in these situations. Rather than provide an exhaustive (and somewhat tedious) coverage of all the automake constructs that are possible, let's look instead at a few of the more common constructions found in Makefile.am files. The idea here is to make you comfortable reading through an &am; file and using those ideas to construct your own. In a project organised with mostly administrative files in the top-level directory and source, documentation, graphics, etc files one level down in subdirectories, the top-level Makefile.am will often look like: SUBDIRS = libsmil dtd po EXTRA_DIST = \ libsmil.spec.in \ libsmil.spec \ intltool-extract.in \ intltool-merge.in \ intltool-update.in snap: cp libsmil.spec libsmil.spec.orig; \ sed -e "s/\(^Version: \)\(.*\)/\1\2.`date +'%Y%m%d'`.snap/" libsmil.spec \ > libsmil.spec.tmp; \ mv libsmil.spec.tmp libsmil.spec; \ $(MAKE) dist distdir=$(PACKAGE)-$(VERSION).`date +"%Y%m%d"`.snap; \ mv libsmil.spec.orig libsmil.spec Note that the top-level subdirectory is not listed in SUBDIRS, so it is built last. To force the top-level directory to be built at a specific time, just put . into the SUBDIRS list somewhere. INCLUDES = $(LIBGLADE_CFLAGS) # build documentation when doing a distcheck. DISTCHECK_CONFIGURE_FLAGS = --enable-gtk-doc if HAVE_PYTHON bin_SCRIPTS = libglade-convert endif pkgconfigdir = $(libdir)/pkgconfig pkgconfig_DATA = libglade-2.0.pc ... XMLCATALOG = /etc/xml/catalog xmldir = $(datadir)/xml/libglade xml_DATA = glade-2.0.dtd install-data-local: -xmlcatalog --noout --add "system" \ "http://glade.gnome.org/glade-2.0.dtd" \ $(xmldir)/glade-2.0.dtd $(XMLCATALOG) uninstall-local: -xmlcatalog --noout --del $(xmldir)/glade-2.0.dtd $(XMLCATALOG) #small hack to get distcheck to work clean-local: rm -f intl/po2tbl.sed .memdump Still to come... Autoconf Portability considerations Conditional building As mentioned previously, autoconf does a lot of the build-specific testing and configuration. Parsing configuration arguments. Testing for the existence of various files and functions. Creating Makefile.in files for later processing by the configure script. Creating the config.h file from autoheader's config.h.in template. A lot of the preliminary and final portions of a configure.in file are very similar. AC_INIT(slice-n-dice 0.1) AM_CONFIG_HEADER(config.h) AM_INIT_AUTOMAKE(AC_PACKAGE_NAME, AC_PACKAGE_VERSION) # Checks for programs. AC_PROG_CC AC_PROG_LIBTOOL AC_PROG_INTLTOOL # Checks for libraries. ... # Checks for header files. AC_HEADER_STDC ... # Checks for library functions. ... AC_OUTPUT( sliceNdice.spec Makefile src/Makefile po/Makefile.in ) Not unusual to see tests of the form AC_CHECK_FUNCS([clearenv setenv]) AC_CHECK_HEADERS(dlfcn.h) AC_CHECK_FUNCS(bind_textdomain_codeset) These tests set the values of various constants in config.h: /* Define to 1 if you have the `clearenv' function. */ #undef HAVE_CLEARENV /* Define to 1 if you have the `setenv' function. */ #undef HAVE_SETENV /* Define to 1 if you have the <dlfcn.h> header file. */ #undef HAVE_DLFCN_H /* Define to 1 if you have the `bind_textdomain_codeset' function. */ #undef HAVE_BIND_TEXTDOMAIN_CODESET Sometimes we want to only set a particular set of compiler flags if a particular header file or library is present. AM_CONDITIONAL(HAVE_CDDA, false) AC_CHECK_HEADERS(cdda_interface.h cdda_paranoia.h, [ CDDA_LIBS="-lcdda_paranoia -lcdda_interface" AM_CONDITIONAL(HAVE_CDDA, true) ]) Another, more complicated example. AC_CHECK_LIB(jpeg, jpeg_start_decompress, [AC_CHECK_HEADER(jpeglib.h, jpeg_ok=yes, jpeg_ok=no)], AC_MSG_WARN(*** (jpeg library not found) ***), -lm) if test "$jpeg_ok" = yes; then JPEG='jpeg'; LIBJPEG='-ljpeg' AC_DEFINE(HAVE_LIBJPEG) else AC_MSG_WARN(*** JPEG loader will not be built ***) fi By default, ∾ understands a large number of different arguments. Users can always set things such as Install prefixes for each component (binaries, headers, ...). Program name transformations. Host and target platforms (for cross-compiling). It is also possible to add your own arguments... AC_ARG_WITH(html-dir, [ --with-html-dir=PATH path to installed docs ]) if test "x$with_html_dir" = "x" ; then HTML_DIR='${datadir}/gtk-doc/html' else HTML_DIR=$with_html_dir fi ... and to have features which can be enabled and disabled. AC_ARG_ENABLE(gtk-doc, [ --enable-gtk-doc Use gtk-doc to build documentation [default=auto]], enable_gtk_doc="$enableval", enable_gtk_doc=auto) if test x$enable_gtk_doc = xauto ; then if test x$GTKDOC = xtrue ; then enable_gtk_doc=yes else enable_gtk_doc=no fi fi Configuring all these variables in configure.in is one thing. But they have to passed to the Makefile ultimately. This is done using AC_SUBST. HTML_DIR=/tmp/html AC_SUBST(HMTL_DIR) This fragment will replace all occurences of @HTML_DIR@ in any Makefile.am with the value of the HTML_DIR variable that is computed in the configure script. If a variable's contents are empty when it is substituted, the empty string will be used as the value. This can be used to effect conditional compilation, amongst other things. To see what is really going on, we need to look at configure.in and Makefile.am in tandem. In configure.in... PKG_CHECK_MODULES(LIBGLADE, libxml-2.0 >= 2.4.10 atk >= 1.0.0 gtk+-2.0 >= 2.0.0) AC_SUBST(LIBGLADE_LIBS) AC_SUBST(LIBGLADE_CFLAGS) and in Makefile.am. INCLUDES = @LIBGLADE_CFLAGS@ libglade_LDADD = @LIBGLADE_LIBS@ Since ∾ defines Makefile variables for each substitution that it does, the last example could also be written as INCLUDES = $(LIBGLADE_CFLAGS) libglade_LDADD = $(LIBGLADE_LIBS) In the last example, we used a macro called PKG_CHECK_MODULES. This is not packaged with autoconf by default. Instead, the pkg-config application provides this macro when it is installed and installs the macro in (typically) /usr/share/aclocal/. This directory is searched when aclocal is run as part of autogen.sh and all of the relevant macros that are need by configure.in are pulled into aclocal.m4. Macros are not trivial to write, since they need to extract their information without interfering with the external system. They also need to account for many possibilities. AC_DEFUN([JH_PYTHON_CHECK], [AC_CACHE_VAL([jh_cv_path_python], [if test "x$PYTHON" != x; then jh_cv_path_python="$PYTHON" else dnl python code to check for required python jh_python_check=' import sys, string if sys.version_info < $1: sys.exit(1) $2 sys.exit(0)' dnl for jh_python in python python2 python2.3 python2.2 python2.1 python2.0; do jh_save_IFS=$IFS; IFS=$PATH_SEPARATOR jh_dummy="$PATH" for jh_dir in $jh_dummy; do ... done test -n "$jh_cv_path_python" && break done fi ]) Slides will be available online later this week. I will publish the location at , or just ask me on Thursday or Friday.