Making Portable Unix-based Executables

Introduction

Portable executables in this context are executables compiled from source code on one machine, which can then be transferred to and run on another machine. This document only applies to Unix-based systems as I don't have any experience of non-Unix systems (Macintosh, MS Windows) and can't comment on the portability issues for those platforms.

The content of this page is based on personal experience and should be treated with some scepticism.

Factors influencing portability

There are two key factors that influence portability:

  1. Operating system differences
  2. Library dependencies (static versus dynamic executables)

These factors are discussed in the following sections.

Operating system differences

Differences in architecture and implementation between different operating systems mean that executables built for example on an SGI will not run on a DEC Alpha. More subtly, differences of this nature between versions of a single basic operating system may also mean that executables built on one version are not portable to another - usually this is a backwards compatibility issue, for example: OSF1 V4.0 executables will run (suboptimally) on OSF1 V5.1, however by default V5.1 executables will not run on V4.0.

In practice there are limited strategies available for overcoming these differences. One option is cross-compilation (building executables on one system to run on another, but this is something I have no experience of). See the section on Platform-specific portability issues for more discussion.

Library Dependencies

Libraries are either static archives or shared objects (also called dynamic archives). An executable which is linked against a static library is referred to here as a statically-linked executable and includes a copy of the code that it requires from that library inside itself - so at runtime the required code is always available to the executable.

An executable which is linked against a dynamic library is referred to as a dynamically-linked executable. It doesn't include the code from the library, instead it has to load the required code at runtime. The dynamically-linked executable is smaller, but it relies on the shared object being available and accessible at runtime.

By contrast the statically-linked executable is larger (because it contains additional code from the library) but it doesn't need the library to be present at runtime, and so is more portable.

A totally portable executable is one which is not dynamically linked to any of the libraries that it depends on. In practice this may not be possible to achieve (see the section on Platform-specific portability issues, below). However, dynamically-linked executables can still be considered to be portable, provided that the target platform also has dynamic versions of the required libraries. Thus the target for portability becomes minimising the number of dynamically linked dependencies.

Controlling which libraries an executable is linked against

Generally if both static and dynamic achives of the same library are available, then executables will by default be linked against the dynamic archive. This default behaviour can be modified by using linker flags, e.g. for the GNU compiler set:

These arguments are positionally-sensitive in that they affect the subsequent -l options, and can be used multiple times. This means that it is possible to specify a mixture of static and dynamic libraries in a single compile/link line. For example:

gcc ... -non_shared -lmylib1 -lmylib2 -call_shared -lmylib3 ...

would force linking against static versions of libmylib1 and libmylib2, and then switch to dynamic linking for libmylib3.

Strategies to try for building portable executables

Platform-specific portability issues

This list is not definitive and the information is anecdotal.

IRIX 6.5: has a runtime library crt0 which is only available as a dynamic library, so it is not possible to create pure static executables on this system. However crt0 seems to be standard on IRIX so this doesn't seem to cause a portability problem in practice.

OSF1 V4.0: can use -non_shared to make portable executables. This may be necessary because for OSF1 V4.0 systems without compilers it seems that some low-level runtime libraries are not provided either.

OSF1 V5.1: seems to have similar issues to IRIX 6.5, in that one or more system library is only available as a dynamic version. It is therefore not possible to use -non_shared as is the case with V4.0. Also, executables compiled on V5.1 (and probably V5.0) are not backwardly compatible with V4.0, unless the -D__V40_OBJ_COMPAT option is used at compile time (this used to be used in the CCP4 configure for OSF1 V5.0 up until release 4.2.1). It appears that V4.0 executables will run on V5.0 however.

Linux: portability between different flavours and versions of Linux is not at all well-defined (and is further complicated by issues such as differeing compiler versions etc). My only experience in this context is with Red Hat versions 8.0 and 9. It is my understanding that RH 8.0 executables built with GNU compilers will work with RH 9 (but not the other way around) and that RH 8 executables are also usable on other up-to-date flavours of Linux (SuSE, Mandrake, Debian).

Alternatively, Kevin Cowtan has been advocating the use of autopackage (apgcc), which claims to produce executable code which is completely portable across all Linux platforms. See Kevin's CCP4 projects page.

Practical example: making binaries of the CCP4 distribution for Unix platforms

Currently CCP4 provides precompiled executables built on the following platforms:

O/S name Description Versions
Irix SGI operating system 6.5
OSF1 DEC Alpha/Compaq Tru64 Unix V4.0, V5.1
SunOS Solaris operating system 5.8
Linux Linux Red Hat 8.0, Red Hat 9

For the listed operating systems, binaries of the programs are built using the configure line:

> ./configure system --with-netlib-lapack --with-x

This default build only makes static versions of the CCP4 libraries, and the resulting executables are linked statically with respect to those libraries. --with-netlib-lapack forces the building of and linking against (static) LAPACK and BLAS libraries, and is used to prevent the executables being linked against system-provided versions of the same libraries (which might only be available on the source system as dynamic libraries, and not on the target system). (--with-x forces building of the CCP4 X-windows programs and is not relevant to the discussions here.)



P.J.Briggs@ccp4.ac.uk