Notes for C/Fortran into Tcl

Rough draft 08-11-99
Minor updates 01-03-00
Revised again 27-02-01

Any comments/corrections/additions welcome - send to p.j.briggs@ccp4.ac.uk

Contents

1. Calling C from Tcl

1.1 Overview

Tcl has a library of C functions which you can use in your C code to build new Tcl functions. There are two approaches that one can adopt:

  1. create a loadable package (i.e. shared object, .so, or dynamically loadable library, DLL), or
  2. create a "custom main program".

In either case the way the commands are defined/written in C is the same - the differences are how you get the commands into your Tcl interpreter (e.g. wish).

In the first case the shared object/DLL is loaded into the interpreter at run-time, to make the new commands available. In the second case, the Tcl/Tk libraries are linked into the program at compilation-time, so that you end up with a customised interpreter which has the new commands embedded in it automatically.

1.1.1 Using a DLL

Inside a script, the DLL is loaded using the load command, eg:

    load ccp4tcl.so ccp4tcl
This tells the script that it is looking for a package called "ccp4tcl" in a file ccp4tcl.so

Upon loading the package the script looks for a C procedure called <package>_Init (e.g. Ccp4tcl_Init). The first letter of the package name is automatically uppercased; the rest automatically lowercased.

<package>_Init registers the commands in the package. This is done by calling a command called Tcl_CreateObjCommand (from the Tcl C library) for each of the new commands in the package. For example:

int Ccp4tcl_Init(Tcl_Interp *interp) {

  /* Register the commands in the package */

  Tcl_CreateObjCommand(interp, "proc_no_1", ProcNo1,
       (ClientData)NULL, (Tcl_CmdDeleteProc *)NULL);

  Tcl_CreateObjCommand(interp, "proc_no_2", ProcNo2,
       (ClientData)NULL, (Tcl_CmdDeleteProc *)NULL);
  ...
  etc
  ...
  return TCL_OK
}

Here proc_no_1 etc is the name of the command as it will be called in the script, and ProcNo1 is the name of the corresponding C function as defined later in the file (see section 1.2).

The return at the end of the procedure can return either TCL_OK or TCL_ERROR, depending on the exit status (although I think that you can define other exit status of your own). These are macros defined in tcl.h.

(As of 8.0, Tcl supports an "object-oriented interface" for creating new commands which is supposed to be an improvement on the original string-based interface. The string-based interface is still supported but its use is deprecated.)

1.1.2 Using a custom Main program

Here you are writing a new C program. The main function will look something like:

int main(int argc, char *argv[])
{

  /* Call Tk_Main */
  Tk_Main(argc, argv, Tk_AppInit);
  exit(0);
}
The Tk_AppInit function must be provided by your program, for example:
int Tk_AppInit(Tcl_Interp *interp)
{
  /* Initialise packages */
  if (Tcl_Init(interp) == TCL_ERROR) {
    return TCL_ERROR;
  }
  if (Tk_Init(interp) == TCL_ERROR) {
    return TCL_ERROR;
  }

  /* Set the global pointer for the interpreter */
  myInterp = interp;

  /* Define the application-specific commands here */

  Tcl_CreateObjCommand(interp, "proc_no_1", ProcNo1,
                       (ClientData)NULL, (Tcl_CmdDeleteProc *)NULL);

  Tcl_CreateObjCommand(interp, "proc_no_2", ProcNo2,
                       (ClientData)NULL, (Tcl_CmdDeleteProc *)NULL);
  ...
  etc
  ...
  return TCL_OK;
}

This can be divided up into two components - the first set of calls are to initialise the Tcl and Tk libraries, which must be linked in at compilation-time. The second set of calls register the application-specific commands, and are the same as those found in <package>_Init.

1.1.3 Compilation issues: DLL versus custom Main

For a custom Main program you need to link in all the relevant libraries, which will include the standard C library, the maths library (if you use any maths functions), plus the Tcl library. If you are also building Tk into your custom interpreter then you will need the Tk library which in turn depends on one or more X libraries (depending on the system). If you are using Fortran calls (madness!) then you will need to link in one or more Fortran libraries too.

This is all quite complicated, especially finding the libraries you actually need. However the advantage seems to be that the resultant interpreter is pretty robust.

To use a DLL on the other hand, you must be able to create a shared object which can be loaded into your interpreter at run-time, as described above. In principle most compilers allow you to generate shared objects, for example using the -shared flag on the DEC Alpha linker (V4.0, at least), and this approach has the advantage of not needing to link in to the Tcl and Tk libraries plus their dependencies (though you still need the Fortran libraries, if you are using them).

If you are using DLLs then it is worth thinking about using the Stubs libraries, which should avoid some version dependencies - although I haven't personally investigated this yet.

I have chosen to use a custom interpreter for the time being, mainly because of the problems I encountered with using DLLs under Linux. It is highly probable that when I migrate away from using Fortran that these difficulties will disappear (PJX, 27-02-01).

(One other point regarding custom Main programs - it seems that as of Tcl/Tk 8.1, the Tk_Main function is no longer a library function, and that instead a macro is provided for it in the tk.h header file. So <tk.h> must be included if you are linking against Tcl/Tk libraries of version 8.1 or better.)

1.2 Tcl Commands in C

The rest of the shared library/custom main program is taken up with the C procedures defining the Tcl commands.

For example: say that we are defining the command proc_no_1 above. We want it to have the calling syntax

     proc_no_1 var1 var2 ?var3?
where var1 is an integer, and var2 and var3 are real numbers (see below; Tcl will only deal with doubles) with var3 being an optional argument.

The code would be something like:

int
Proc_No_1(ClientData clientData, Tcl_Interp *interp,
       int objc, Tcl_Obj *CONST objv[])
{
  Tcl_Obj *resultPtr;
  int    error;
  int    var1;
  double var2, var 3;

  /* objc gives the number of "objects" = arguments to
     the command */
  if (!(objc == 2 || objc == 3)) {
     Tcl_WrongNumArgs(interp, 1, objv,
        "var1 var2 ?var3?");
     return TCL_ERROR;
  }

  /* Process the arguments
     objv is an array holding the arguments */

  /* Use Tcl_GetIntFromObj to put an integer value
     into a variable from the argument list */

  error = Tcl_GetIntFromObj(interp, objv[1], &var1);
  if (error != TCL_OK) {
     return error;
  }

  /* Use Tcl_GetDoubleFromObj to put a double value
     into a variable from the argument list */

  error = Tcl_GetDoubleFromObj(interp, objv[2], &var2);
  if (error != TCL_OK) {
     return error;
  }
  if (objc == 3) {
     error = Tcl_GetDoubleFromObj(interp, objv[3], &var3);
     if (error != TCL_OK) {
        return error;
     }
  }

  /* Put C code here to work with the variables ... */

  /* The result is set by getting a handle on the result object
   * and setting its value */

  resultPtr = Tcl_GetObjResult(interp);
  Tcl_SetStringObj(resultPtr, "Everything ok", -1);

  return TCL_OK;
}

Note that the Tcl will pass only integer, doubles and strings as arguments. You have to write the core of the procedure in C to do the actual work (see comment above).

Aside (1): extracting string arguments

String arguments seem to be handled slightly differently to the ints and doubles discussed above. To extract a string from an argument, you need to do the following

  1. Declare the string variable as a character pointer, i.e. char *a;

  2. Use the Tcl_GetStringFromObj command, which returns the pointer to the string, i.e.
       a = Tcl_GetStringFromObj(objv[1], &length)
    "length" is an integer which is returned holding (you guessed it) the length of the string.

Seems easy? It is when you know what you're doing... One point is that there doesn't seem to be a way of trapping the errors for reading the strings, I guess you always get a string even if the object wasn't supposed to be one.

Aside(2): returning results from the procedure

Tcl commands return some value on execution, i.e.

set result [ my_proc var1 var2 ... ]

In the new Tcl commands outlined above, this result is set using the following code:

  resultPtr = Tcl_GetObjResult(interp);
  Tcl_SetStringObj(resultPtr, "Everything ok", -1);

The first line sets a pointer to the result object in the interpreter, the second line actually sets the value of the result to be a string with the value "Everything ok" (the last argument to Tcl_SetStringObj is the length of the string, or if negative - as in this case - then everything upto the first null character is used).

Other commands are available to set the result object to be an integer or double value.

This result should not be confused with the value returned by the C procedure, which should return values of TCL_OK, TCL_ERROR etc depending on the success or failure of the procedure - for example, from above

  error = Tcl_GetIntFromObj(interp, objv[1], &var1);
  if (error != TCL_OK) {
     return error;
  }
and
  return TCL_OK;
These exit values are used by the main Tcl interpreter. A return value of TCL_ERROR will cause the intrepreter to stop, unless it is contained by a Tcl catch command.

2. Calling Fortran from C

Careful! Section 2.1 below is really only relevant if you are calling C from Fortran (although it's useful background). Sections 2.2 and 2.4 are the most immediately useful; section 2.3 is good if you try something and it doesn't work the way you expected. I will try and revise these notes to make them more concise and easier to follow.
It may also be that these mechanisms are replaced in the near future by a more transparent system, which would make these notes redundant anyway. -PJX

In principle it should be easy to call F77 subroutines from C, provided that you respect the different calling conventions that different machine architectures use.

In CCP4 someone has already done a lot of work to make this kind of C/Fortran interfacing as portable as possible. However as a result of needing to be as general as possible the code can look nightmarish at first.

2.1 Calling conventions

Looking at library.h there are 4 basic calling conventions:

  Convention          Systems
  ----------          -------
  CALL_LIKE_HPUX      AIX, HPUX
  CALL_LIKE_SUN       ALLIANT, CONVEX, ESV SGI, SUN, ULTRIX/OSF1
  CALL_LIKE_STARDENT  ARDENT/TITAN/STARDENT
  CALL_LIKE_MVS       NT (MVS)

These different calling conventions have different ways of typing and calling the Fortran routine. As an example, consider "ccperr" called from within the subroutine "fatal" in library.c:

1. CALL_LIKE_HPUX:     #if CALL_LIKE_HPUX
                         extern void ccperr();
                         ccperr (&mone, message, (int) strlen(message));
                       #endif 

2. CALL_LIKE_SUN:      #if CALL_LIKE_SUN
                         extern void ccperr_();
                         ccperr_ (&mone, message, (int) strlen(message));
                       #endif

3. CALL_LIKE_STARDENT: #if CALL_LIKE_STARDENT
                         extern void CCPERR();
                         struct Str_Desc str;

                         str.Str_length = (int) strlen(message);
                         str.Str_pointer = message;
                         CCPERR (&mone, &str);
                       #endif

4. CALL_LIKE_MVS:      ? Better ask Alun about this...?

The CALL_LIKE_ variables are set in library.h, so I think this must be included anywhere where you want to use this kind of construct.

2.2 Prototyping

CCP4 also allows function prototyping to be optional - this is basically declaring the function at the head of the program, for example (generally):

#if defined (PROTOTYPE)
  static size_t flength (char *s, int len);
#endif

When declaring FORTRAN functions, these are always of type void, and variables in the argument list must be declared as pointers to the appropriate type. As an example:

void getsection_(int *iunit, int *ksec, float *section,
		 int *nsize, int *ifail);

For details about the underscoring, or other changes required to the Fortran name, see section 2.1 (calling conventions).

2.3 Other Important Points

Other important points to bear in mind are:

  1. Fortran passes its variables by reference. This means that you must give adresses in your calling C-program (but remember that the name of an array is also a pointer to that array). For example:
    int main()
    {
      int this_sec, ifail=0, iunit, nsize;
      float section[MAXSECT];
      ....
      /* Lots of C code */
      ....
      getsection_(&iunit,&this_sec,section,&nsize,&ifail);
      ....
    
    (We use section rather than &section because an array name is also a pointer to that array.)

  2. Watch out especially with float's and double's. Make sure that the size of the variable in the calling program is identical to the size in the Fortran routine. This is important especially for reals, i.e.
           double <--> real*8,
           float  <--> real
    This is extremely important on machines with little endian byte ordening. Parsing a float (C-routine) to a real*8 (Fortran) number will not generate SEGV but give wrong results as the data is parsed wrongly.

  3. Remember that the array index in Fortran starts at 1 while in C this is at index 0; hence a parsed array fortran_array[1:100] must be used in the C-routine/program as c_array[0:99]

  4. For multidimensional arrays, remember that C stores arrays such that last index varies fastest - the opposite of Fortran. So for example, A(2,3) in the Fortran program is not equivalent to a[2][3] in the C program.

  5. Be careful when passing strings from C to Fortran (e.g. filenames). It is probably best to pass both the string and its length as two arguments to the Fortran routine.

  6. Fortran LOGICALs are not consistently defined across platforms. It is probably safe to assume that they can be represented in C by int and that .FALSE. is equivalent to zero; however .TRUE seems to be defined differently on different systems, e.g. under IRIX it is 1, on OSF1 it is -1.

I suspect there are other subtleties which are not immediately apparent when you want to integrate C, Fortran and Tcl.

2.4 Practical Implementation

If the above seems rather complicated then consider the example of solomon (the program not the biblical king). In solomon.c there is a call to the Fortran subroutine rbfro1; solomon handles the different possible calling conventions by using:

#if VMS
#define solmain_ SOLMAIN
#define rbfro1_ RBFRO1
#define frtccperr_ FRTCCPERR
#endif

#if CALL_LIKE_HPUX
#define solmain_ solmain
#define rbfro1_ rbfro1
#define frtccperr_ frtccperr
#endif

at the program head, then

#if defined (PROTOTYPE)
  void rbfro1_(float *cell, float *volume, float *tmpmat);
  ....
for the prototyping, and
      rbfro1_(cell, &volume, tmpmat);
later in the program for the actual function call.

The define statements at the start replace all the later occurances of the text rbfro1_, ensuring that the correct calling convention is used. This seems like a much neater implementation.

3. Fortran from Tcl via C

Careful! Although I would stand by the opening statement below i.e. don't mix Fortran and C, in fact this is not so difficult. Most of the problems happen at the linking stage, and there are particular problems if you want to use DLLs. I will try and rewrite this section in future, to be less pessimistic... -PJX

At this point it would seem like the best idea to write your new Tcl functions in C whenever possible, if only to save on the kind of aggravation mentioned above.

However, since we must do it, here are some random notes based on my limited experience.

Basically the Fortran code can be called from within the Tcl command by respecting the notes above. The additional issues that I am aware of are:

  1. You must explicitly declare the size of any arrays that you use in the C code - otherwise this seems to cause a problem on exit (I kept getting segmentation faults). Possibly this is to do with the Tcl attempting to destroy objects of unknown size.

  2. Linking in external libraries
    This is Bad News. Linking in the CCP4 library seems to create more problems than it solves - you end up with every single routine in the new shared library (= bloated .so file) and a set of unresolved symbols that I think must be Fotranic libraries.
    It is therefore more desirable to be able to link in the individual subroutines that you actually want. The ar command has a facility for extracting .o files from the archived library (i.e. libccp4.a) which it should then be possible to add to the Tcl library.
    I need to investigate this further. It may make building the library quite tedious but once sorted should be quite straightforward to update as required.

4. Further Reading

I used Brent Welch's book "Practical Programming in Tcl and Tk" (2nd edition - great book, crap binding) as a starting point since this has some good chpaters about linking together Tcl and C.

Beyond that I relied upon the Tcl man pages for information on the Tcl C library commands, and on the f77/cc/ld man pages on the various systems to help me track down the missing libraries. At least some of the points in section 2.3 were taken from a Fortran-C FAQ which I found on the web - unfortunately I can't remember where this was but the point I suppose is that the information is out there, so have a look.

Although I didn't find much practical help there, it is also worth visiting the Tcl Developer's Site (though at the time of writing the search mechanism seemed to be broken, so use their site map to nagivate).


Peter Briggs