From f2471da7db7d49e384007c41759e4e8cb3d8340c Mon Sep 17 00:00:00 2001 From: Mike Frysinger Date: Sun, 13 Feb 2022 21:43:37 -0500 Subject: newlib: update build system generation documentation Replace all of the individual autotool steps with a single autoreconf. This simplifies the documentation greatly, and in the current system, only takes ~10 seconds to regenerate everything. Update the developer documentation to cover all the major components of the current build system. Hopefully this is a fairly complete road map to everything. I tried to include everything that I wish I knew when I started hacking on this :P. --- newlib/HOWTO | 171 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 171 insertions(+) (limited to 'newlib/HOWTO') diff --git a/newlib/HOWTO b/newlib/HOWTO index 668766096..e9af3a408 100644 --- a/newlib/HOWTO +++ b/newlib/HOWTO @@ -110,3 +110,174 @@ Level 4 Full POSIX or Linux compliance. Essentially these are functions that are present in a standard Linux kernel but are irrelevant to an embedded system. These functions do not form part of the EL/IX API. + +BUILD SYSTEM +============ + + Newlib utilizes GNU Autotools (GNU Autoconf & GNU Automake) for its build +system. It can be broken down into two phases: configure & compile (make). +NB: GNU Libtool is not used. Newlib only produces static archives, and GNU +Automake is capable of handling that. + + As noted in the README file, you do not need to worry about which tools to +run at any particular point. Simply invoke `autoreconf` and it will do it all. + + Tool versions are tightly controlled. GNU Autoconf 2.69 exactly must be +used. Attempts to use any other version, newer or older, will be rejected. If +the current system does not include that version, it can be downloaded from the +GNU Autoconf website and installed into a local directory tree (like ~/.local). +GNU Automake 1.15.1 is required as a minimum. Newer versions are allowed when +running locally, but developers submitting patches must use 1.15.1 exactly. + + The configure phase naturally starts with the configure script. It merges +multiple inputs to produce the final script. +* aclocal.m4: Generated by aclocal; do not edit. Copies all the various m4 + macros used by configure from the system install into the local copy. This + allows regenerating configure without needing all the same system packages. +* autom4te.cache/: Generated by aclocal to cache system search results. Safe + to delete at any time, but never edit anything in this directory. +* iconv.m4: Generated by libc/iconv/ces/mkdeps.pl; do not edit. Keeps the + set of dynamic iconv synced between the source files and configure. +* acinclude.m4: Historically, a way of sharing configure logic with subdirs. + At some point, it should all be merged into configure.ac directly. +* configure.ac: The main configure script for supporting all Newlib options. +* configure.host: An index of every supported Newlib host and the settings to + use when compiling for it. +* **/acinclude.m4: Subdirectories may include custom configure tests without + having to clutter the common configure.ac or acinclude.m4 file. When adding + a new file, it must be manually included -- there is no mechanism to search + and automatically include every such file in the tree. Look for m4_include + lines in configure.ac or existing acinclude.m4 files as examples. Care must + be taken to not run excessive tests for incompatible targets, or to declare + generic names or symbols that can collide with other targets. Any such file + should try to match $host (or similar) settings before running. Settings + from configure.host (e.g. machine_dir or sys_dir) are guaranteed to be + available. When creating names (Makefile variables, defines, etc...), try to + include the respective library and machine or sys name. Do not confuse these + with the historical top-level acinclude.m4 file! + + The configure script has to take special consideration to the fact that it +usually runs with a toolchain that lacks a C library (since Newlib will provide +it once it's compiled & installed). Newlib uses the uncommon AC_NO_EXECUTABLES +macro to tell GNU Autoconf to not automatically require a toolchain that can +link executables. As a result, Newlib itself cannot perform any link tests. +It shouldn't really need to, but configure authors must keep this in mind and +stick to the appropriate source-level test -- AC_PREPROC for testing behavior +of the preprocessor and AC_COMPILE for testing compiler behavior. + + The newlib.h header is created from newlib.hin during the configure phase. +The newlib.hin file is managed by autoheader, so do not edit it directly. Any +symbol declared in configure via AC_DEFINE is automatically included. The file +declares all the Newlib-specific configure settings and will be installed as +part of the standard set of headers. Critically, all symbols are namespaced +with a leading _ prefix to avoid collisions with user code, and any symbols not +namespaced are automatically removed. This is necessary due to some system m4 +macros automatically adding their own defines that we cannot disable. It also +means that, when using AC_DEFINE, the _ prefix must be manually added. + + A single Makefile is created during the configure phase. This will build +all of Newlib using non-recursive make (i.e., there are no subdirectories with +their own Makefiles). The single Makefile.am file includes Makefile.inc files +in its immediate subdirectories, and those further include Makefile.inc files +in their subdirectories. This means all variables and targets exist in a +single global flat namespace, so care must be taken to not use generic names +like "SOURCES" in any Makefile.inc file. Instead, use %C%_ on all variables to +get a unique prefix, and GNU Automake will expand it. In order to use one of +the standard GNU Automake variables (e.g., CLEANFILES), first declare it at the +top of the Makefile.am by assigning it (e.g., CLEANFILES =), and then always +append to it in Makefile.inc (e.g. CLEANFILES += ...). Again, GNU Automake +will take care of combining all of this logic together to produce a portable +Makefile. + + Developers used to changing to a subdirectory and running make commands +to iterate quickly on a smaller section of Newlib might be surprised that this +does not work -- there is no Makefile in any subdirectory. Instead, make has +to be run from the top-level, and the full path to the target is specified. +So instead of running `cd subdir && make foo.o`, run `make subdir/foo.o`. This +applies to any target, not just objects. Some common targets: +make libm.a +make libc.a + + Conditionals are allowed in Makefile.inc files, but using them to elide +an entire subdir Makefile.inc should be avoided. Instead, GNU Automake +conditionals should be used only within a single Makefile.inc file to control +settings specific to that directory. See the documentation section below for +more information. + + Per-directory compiler settings are allowed. Similar to the syntax used +above, the _%C% suffix can be used. See the Makefile.am file for the current +set of flags (e.g. CFLAGS, CPPFLAGS, etc...) that are supported. +AM_CFLAGS_%C% = ... +libm_a_CFLAGS_%C% = ... +libc_a_CFLAGS_%C% = ... + + Per-file compiler settings are allowed. Similar to the syntax used +above, the _%C% suffix can be used. See the Makefile.am file for the current +set of flags (e.g. CFLAGS, CPPFLAGS, etc...) that are supported. +AM_CFLAGS_%C%_moo.c = ... +libm_a_CFLAGS_%C%_foo.c = ... +libc_a_CFLAGS_%C%_bar.c = ... + + An important aspect of the build system is allowing machine-specific +directories to override common implementations. For example, Newlib provides +generic C implementations of string & memory functions in libc/string/ that +perform adequately for any target, but many architectures include hand written +assembly files for smaller & faster execution. Those files live under the +corresponding libc/machine// directory. Both versions are +compiled when building Newlib, but when generating the libc.a archive, the +common object is added first, followed by the machine-specific one. This works +because the object names follow a consistent convention: libm_a-.o +for the math library and libc_a-.o for the C library. If the common +code stores a function foo in foo.c, then it can be overridden by another foo.c +or foo.S or foo.s file since they all produce foo.o. For this reason, all +machine Makefile.inc files must be included last after all common Makefile.inc +files. + + Note that the Makefiles do not use VPATH to search source files across +multiple source directories. VPATH is used to match a single build directory +to the corresponding source directory, but that is it. For more details on +how machine directories override functions in common directories, see the +previous section. + + Keep in mind that machine overrides only work at the file level, not at +the symbol level. If a common file defines multiple symbols, then the machine +override must define all the same symbols. This is why most common code stores +only one function per source file. + + The documentation, both the manual and man pages and header references, +should be consistent regardless of what configure options are used (including +the target system). This makes it easy to get a complete documentation build +at all times. If inclusion of chapters depended on specific configure options, +it would be difficult to find the set of options necessary to produce the +complete manual, if it were even possible in the first place (as some configure +options are incompatible). Since documentation chapters & APIs are declared in +the respective directories and not in a centralized location, it is important +to not use any GNU Automake conditional to control whether Makefile.inc files +are included. Documentation that covers optional APIs should note that it is +not guaranteed to be included in every build of Newlib for every target. The +level of detail is left up to the author's discretion. + + Newlib automatically builds itself for all multilib configurations that +the active toolchain supports. This logic is provided by common code from the +combined toolchain source tree, but bolts itself into the GNU Autoconf and +Automake files. The critical parts to keep in mind: +* ../config/multi.m4: AM_ENABLE_MULTILIB adds the --{en,dis}able-multilib + options to configure for users, and takes care of calling ../config-ml.in + for the Makefile in the current directory. While not applicable to Newlib + anymore (since Newlib only produces a single Makefile now), this does not + automatically run for any subdir Makefiles. Instead, it must be run manually + in AC_CONFIG_FILES's commands argument. +* ../config-ml.in: Rewrites the specified Makefile to include the various + multilib variables and the multi-do command. Notably, this only makes the + multi-do rule available, it does not cause it to be used anywhere. +* ../multilib.am: Manually included in Newlib's Makefile.am. This adds calls + to multi-do for the common important rules: all (i.e. compiling), install, + clean, etc... +* FLAGS_TO_PASS: Newlib's Makefile.am declares all the variables to pass to + each multilib configuration via the multi-do rule. While the multi-do rule + includes many common flags automatically, Newlib passes along a few extra + that are unique to it. + + The testsuite is not integrated at all currently. It is not really well +tested in general. The README file has more information. Hopefully someone +someday will step up to integrate it into the existing framework. -- cgit v1.2.3