Using package autoloading in Tcl

Peter Briggs, November 2002

This is a quick write-up of my experiences with setting up packages to use the Tcl "package require" mechanism to automatically load extensions.

A package in Tcl can be a Tcl script (*.tcl), a loadable binary (*.so), or a combination of the two. Examples of the first are xml.tcl and sgml.tcl (distributed with CCP4 in the $CCP4/ccp4i/src directory); examples of the second are BLT and ccp4map.

Tcl provides a command called pkg_mkIndex, which can build an index file to enable the automatic loading of packages via the "package require" command. The man pages for pkg_mkIndex give the full story but a quick guide to the steps is:

Create the package: in the case of a script-based package, it must contain an invocation of the "package provide" command, e.g.
```
package provide junk 1.0
```
In the case of a loadable binary it must have a call to "Tcl_PkgProvide", in the Tcl C library.
Create the index: by invoking the pkg_mkIndex command, e.g.
```
pkg_mkIndex /usr/users/pjx/junk *.tcl
```
This will scan the files matching the expression (in this case all files with extension "tcl") in the named directory, and create a file called pkgIndex.tcl which should contain the commands to load all the packages that it finds there.
Install the package: the package will consist of a directory with the file(s) making up that package, plus the pkgIndex.tcl file. There are a couple of options which will enable autoloading to work:
- Make it a subdirectory of one of the directories given in the tcl_pkgPath variable, or
- If you install it elsewhere on your system, then make sure you add the directory name to the auto_path global variable. (This can be either explicitly e.g. lappend auto_path $dir within the application itself, or via the TCLLIBPATH environment variable.)
Use the package: by adding a line like e.g.
```
package require junk
```
in your application. Providing everything has been set up as described, the Tcl interpreter will be able to locate and load the package automatically.

Problems

I encountered two particular cases when I tried this on the examples here. (It is useful to use the "-verbose" option of pkg_mkIndex if you have difficulties, since there is very little indication otherwise as to what might have gone wrong.)

Package name in loadable binary contains numeric characters: The "ccp4map" library is provided in a loadable library as libccp4map.so. However, running pkg_mkIndex on the appropriate directory gave an error when trying to process this file; the message was `can't find procedure "Ccp_Init"'. The problem stems from the fact that the procedures ignore numbers in the package name (which means that e.g. libBLT24.so can still be recognised as the BLT package).
Script-based package requires a variable from another package: Although it is completely legitimate for one package to depend on another, this can cause a problem for pkg_mkIndex if the first package explicity names a variable from the second. This is because pkg_mkIndex doesn't load the required package when processing the first. (Apparently this is a known feature).

It may be that it is possible to overcome these problems by using the various options available to pkg_mkIndex, but I haven't investigated this yet.

Making pkgIndex.tcl manually

The problems above forced me to make my own pkgIndex.tcl files. For a script-based package the pkgIndex.tcl file should contain a line of the form:

package ifneeded packagename version [list source [file join $dir sgml.tcl]]

e.g. "package ifneeded sgml 1.7 [list source [file join $dir sgml.tcl]]". For a loadable binary it should be of the form:

package ifneeded packagename version [list load [file join $dir binary filename] packagename]

e.g. "package ifneeded ccp4map 2.1 [list load [file join $dir libccp4map[info sharedlibextension]] ccp4map]". I imagine that more complex commands are also possible - for example if a package was made up of both script and binary components.

Where to get more information

I used the man pages for the Tcl package and pkg_mkIndex commands as my primary reference for the things I tried above, and would recommend these as good starting points for finding out more (particularly regarding the mechanics of how ``package require'' searches for packages, and for how pkg_mkIndex processes files.