Reduced Resources on 32-bit platforms

32-bit ports often have limited resources when compared to their 64-bit counterparts. Additionally, some language source files increase memory requirements, like C++ compared to C. The memory requirements combined with limited resources sometimes results in build failures due to Out-Of-Memory (OOM) kills, or causes the build takes longer than expected. See, for example, OOM while building ghc 9.4.5 and Ability to further support 32bit architectures. This article will provide suggestions to help reduce resource overhead.

Architectures with known per-process memory limitations include:

Refactoring

Sometimes a source file is too large, and the toolchain takes a lot of time or requires a lot of memory to to process it. In this case, you should consider refactoring a large source file into several smaller source files.

Make Jobs

Sometimes your build will fail because the number of make jobs is too high, and collectively, the build tools exhaust all memory. In this case reduce the number of make jobs. For example, to use one make job, use make -j1 command or set MAKEOPTS="-j1".

C Macros

C macros and their expansion can be the source of excessive resource usage. You should examine the results of preprocessor expansion to ensure they are efficient or expected; see [PATCH next v4 0/5] minmax: Relax type checks in min() and max() and [PATCH] media: solo6x10: replace max(a, min(b, c)) by clamp(b, a, c).

Tracking macro expansion in source files is resource intensive and adds memory overhead pressure. Using the C Preprocessor option -ftrack-macro-expansion=1 (or level 0) should help reduce memory and processor overhead.

Debug Symbols

Debug symbols can increase memory requirements. If possible, remove debug symbols, enable debug symbol zstd compression, or use split debug symbols.

Optimizations

Some options are going going to be more expensive than other options, like -O1 versus -O3. Using -O0 (no optimizations) or -O1 (minimal optimizations) can help compile a large source file.

The inliner is engaged at -O1, so you may need -O1 -fno-inline. Inlining takes both time and memory as function sizes and jumps are recalculated as inline functions are encountered in the source code. Other options which control inlining include -fno-inline-functions and -fno-inline-small-functions.

Temporary Files

If you are running out of resources and experiencing OOM kills, then be sure to avoid the -pipe option . The -pipe option instructs GCC to avoid temporary files, and attempt to keep output stages in memory.

If you are trying to speed-up a build, then consider using -pipe to avoid writing a temporary file.

Swap Partition

Some platforms, like ARM dev boards and IoT gadgets, have limited memory, like 512 MB and no swap partition or file. A swap partition or file may help a build to succeed, especially when OOM kills are present. A swap file will not reduce memory usage, but will provide more virtual memory.

Adding a swap file is a tradeoff, especially if storage is provided by a SDcard or similar. The SDcard will wear out faster as the swap partition or the file is used.

If you add a swap partition or swap file to a device, then you can tell the kernel to keep things in memory when possible by setting swappiness to a low value, like 1 or 2. For a discussion of virtual memory, swap files and swapiness, see Swapping Behavior on LWN.

Toolchain Options

There are some toolchain options that may help reduce memory usage to avoid OOM kills and sped-up build times. The options apply to different programs used in a build, and include the preprocessor (CC1), the compiler (GCC), the assembler (AS) and the linker (LD).

It has been reported -ftrack-macro-expansion=1 sidesteps the issue on armel and armhf; see Ability to further support 32bit architectures.

CC1:

GCC:

AS:

LD:

--reduce-memory-overheads is a time/space tradeoff. It will reduce memory, at the expense of additional build time.

Cross-Compile

The final work around for reducing build memory requirements is to cross-compile on a machine with more resources. In the old days of makefiles it was relatively easy to cross-compile, once you had the cross-compiler and associated tools. For makefiles you simply set flags such as CPPFLAGS, CFLAGS, CXXFLAGS, ARFLAGS, ASFLAGS and LDFLAGS. You also set tools such a CC, CXX, AS, AR and LD. Finally, you ensured the tools were on path.

Nowadays it can be trickier. External build systems such as Cmake don't always honor your flags, so you often encounter broken builds as your flags and programs are not honored. And worse, you might encounter situations where some parts of the build are coming from the build environment (the machine you are building on), and other parts of the build are coming from the host environment (the machine you are building for).

ReduceBuildMemoryOverhead (last modified 2024-01-13 21:35:54)