home
manual
quick ref.
packages
index
keywords
examples
PREV UP NEXT Yorick

Chapter 4: Embedding Compiled Routines Inside Yorick

You can create a custom version of Yorick containing your own C or Fortran compiled code. If you are careful, your custom version will be easily portable to any site where Yorick has been installed. You will considerably ease portability problems (read ``future hassles for yourself'') by writing in ANSI C, which is a portable language, as opposed to Fortran, which is not.

My experience with C++ is that its portability is intermediate between ANSI C and Fortran. You should be able to write C++ packages for Yorick by using the extern "C" statement for the interface routines called by the interpreter. I don't encourage this, however, since the interpreted language removes many of the motives for programming in C++ in the first place. I won't say any more about C++ packages for Yorick; the idea is to use the `Make-cxx' template instead of the standard `Maketmpl' to build versions of Yorick which include C++ packages.

If you do choose Fortran, stick to a strict subset of ANSI Fortran 77. Do not attempt to pass character variables into or out of interface routines, nor put them in common blocks. Also, try not to use common blocks to pass inputs to or receive outputs from your interface routines. (This is possible; if you enjoy Fortran programming, presumably you'll enjoy figuring out a portable way to do this.)

Whether you write in Fortran or C, do not attempt to do I/O of any sort in your compiled code. The whole idea of embedding routines inside Yorick is to let Yorick handle all I/O -- text and graphics.

In the following discussion, I refer to three directories: `Y_SITE' is the directory where the architecture independent parts of Yorick reside; everything Yorick needs at runtime is here. `Y_HOME' is the directory where the libraries and executables you need to build custom versions are stored. `Y_LAUNCH' is the directory containing the executable for your version of Yorick. When you run Yorick, the names of all three directories are available as variables; for example, start yorick and type Y_HOME to print the name of that directory. Also, I will only discuss the UNIX program development environment; at the present time, Yorick is not easy to extend on any other platform.

The first step is to create a directory in which to build your custom version of Yorick, and put your C and/or Fortran source and header files there.

Next, you need to write a Yorick include file `my_start.i' which will be loaded whenever your custom Yorick starts. This file is read not only by Yorick as a startup file, but also by the Codger automatic code generator when you make the custom version. Codger generates a table of all of the compiled objects (functions, global variables, or Fortran common blocks) which can be referenced from the interpreter. It will also generate wrapper code to call your compiled functions, if you decide not to do this yourself. In this include file: 

 extern my_func;
 /* DOCUMENT my_func(input)
      returns the frobnostication of the array INPUT.
  */

connects my_func to a compiled function Y_my_func of type BuiltIn, which you are responsible for writing. See `Y_HOME/ydata.h' for the definition of the BuiltIn function type. This function must pop its arguments off of Yorick's interpreter stack using routines declared in `Y_HOME/ydata.h', and push its result back onto the stack. The functions YGet* grab things off the stack; sp is the stack pointer. The Globalize function gives you access to variables by the names visible to the interpreter; they are in globTab. You can use Push* functions to push values onto the stack; use CheckStack to be sure there is enough stack space to do so.

Usually, you want to avoid all these details; codger can generate `Y_my_func' for you automatically. To do this, put PROTOTYPE comments in your startup include file: 

 func my_func(input)
 /* DOCUMENT my_func(input)
      returns the frobnostication of the array INPUT.
  */
 {
   return my_func_raw(input, numberof(input));
 }
 extern my_func_raw;
 /* PROTOTYPE
    double my_func_C_name(double array input, long length)
  */

This generates a wrapper for a C function which takes a single array as input and returns a scalar result. If the function had been Fortran, it would have looked like this (Fortran passes all arguments by reference -- that is, as if they were arrays): 

 func my_func(input)
 /* DOCUMENT my_func(input)
      returns the frobnostication of the array INPUT.
  */
 {
   return my_func_raw(input, numberof(input));
 }
 extern my_func_raw;
 /* PROTOTYPE FORTRAN
    double my_func_Fortran_name(double array input, long array length)
  */

Legal data types for the function return result in the PROTOTYPE comment are: void (i.e.- a subroutine), char, short, int, long, float, or double.

Legal data types for the function parameters in the PROTOTYPE comment are: void (only if there are no other parameters), char, short, int, long, float, double, string (char *, guaranteed 0-terminated), or pointer (void *). These may be followed by the word "array", which becomes "*" in the C source code, to indicate an array of that type. The parameter name is optional.

The DOCUMENT comment should start with /* DOCUMENT. They will be returned by the interpreted command help, my_func, and be included in the poor- man's document produced by Yorick's "mkdoc" command (see `Y_HOME/include/mkdoc.i'). 

 extern my_global;
 reshape, my_global, datatype;

attaches the interpreted variable my_global to a C-compiled global of the same name, which has the data type datatype (this must have been declared in a previous struct or be one of the primitive types). If you want my_global to be attached to a global variable of a different name, use: 

 extern my_global;
 /* EXTERNAL my_global_C_name */
 reshape, my_global, datatype;

To attach to a Fortran common block, say 

 	double var1, var2, var3
 	common /my_common/ var1, var2, var3
 	save /my_common/

(note that this doesn't make sense unless the common block is saved outside the scope of the functions in which it is used) use: 

 struct my_common_type { double var1, var2, var3; }
 extern my_common;
 /* EXTERNAL FORTRAN my_common */
 reshape, my_common, my_common_type;

If you mix double, integer, and real data in a single common block, you can ensure that you won't have any alignment difficulties by putting all the doubles first, followed by integers and reals. If you don't do this, you're relying on the existence of a Fortran compiler switch which forces proper data alignment -- some machine someday won't have this.

Near the beginning of your startup include file (or files -- you can have several if you like), you should place a MAKE-INSTRUCTIONS comment: 

 /* MAKE-INSTRUCTIONS
 SRCS = src1.c src2.c src3.c \
        src4.c src5.f src6.c src7.f
 LIB = frob
 #DEPLIBS =
 #NO-WRAPPERS
 */

In this comment, the four keywords SRCS, LIB, DEPLIBS, and NO-WRAPPERS are recognized. You can use \ at the end of a line to continue it on the next line. Only the SRCS keyword is mandatory; you can either omit the others, or precede them by a # to comment them out, as with NO-WRAPPERS in the example.

The SRCS is a space delimited list of the source files required to build the compiled functions (and any functions they may call) declared in this startup include file. This list is used in two ways: first, it determines the names of the corresponding object files, and second, any .f or .F suffixes alert Yorick to load with any special libraries necessary for Fortran.

The LIB is the name of the library to be built for your package. In the example, the library will be libfrob.a. If you do not supply a library name, no library will be built. This doesn't affect your custom Yorick, but if you later want to add more compiled functions, you won't have an easy way to tell Yorick to pick up the functions in this package.

The DEPLIBS is a space delimited list of system libraries that your package needs. You should not list m (the libm math library), X11, or any other library which Yorick routinely loads with on your platform. Hopefully your package will not need any dependent libraries, because if it does, it will be much less portable. Not only will those libraries be unavailable on some platforms, but they will almost certainly be in different locations. For now, you will need to edit the Makefile by hand to insert the proper -L options to allow the compiler to find these libraries, and you will need to do that separately on every platform where you build your package. I would like to automate this to some extent -- the obvious thing is to put a list of library locations in Y_HOME/lib which can be maintained by a guru at each site -- but for now, you're on your own.

The NO-WRAPPERS keyword must be present if this startup include file contains no PROTOTYPE comments (yes, I could check for this automatically, but it slows things down needlessly).

To summarize, in addition to the DOCUMENT comments, your startup include file should contain PROTOTYPE comments for each compiled function for which you want codger to automatically generate a wrapper, and a single MAKE-INSTRUCTIONS comment.

Yorick can build a Makefile automatically. If you don't know what a Makefile is, you can read the UNIX make manpage, but there is a good chance you won't need to know -- read on. To do this, remove any old Makefiles from your directory and type (to the shell): 

 yorick -batch make.i my_prog my_start.i

If your package requires several startup include files, list the others after `my_start.i'. The name my_prog is what your custom version of Yorick will be called.

You can load packages you created previously into your new Yorick by giving their names after a + on this command line: 

 yorick -batch make.i my_prog my_start.i + old_pkg1.i old_pkg2.i

These additional startup include files must be in either `Y_SITE/startup' or `Y_SITE/contrib'. (Or you can make softlinks to them from the current directory.) The LIB libraries mentioned in those startup include files must be in `Y_HOME/lib' or `Y_HOME/lib/contrib' (or again have softlinks from your current directory). By default, the packages `matrix.i' and `fft.i' will be included. To omit them, place `-matrix' or `-fft'

The `make.i' script produces a Makefile which you can use to build your custom Yorick. You should regard the automatically produced file as a first cut; feel free to edit it by hand to get it right. To do that, you will need to read and understand Yorick's Makefile template, `Y_HOME/Maketmpl'. (You can switch to another Makefile template if you need to; the `Make-cxx' and `Make-mpy' templates are lurking in other parts of the Yorick distribution.) After you've got Makefile the way you want it, type: 

 make

To compile and load your new version of Yorick. After you build it, you can move your startup include files into `Y_SITE/contrib' or `Y_SITE/startup' and your library into `Y_HOME/lib' or `Y_HOME/lib/contrib' to make them ``visible'' for other package builders. You don't have to move your startup include files, but if you don't, you will need to keep your new executable in the same directory as they reside -- Yorick looks in `Y_LAUNCH' for them as well as in the standard places.

When you move your package to a different platform, take the Makefile you created along as a part of it. In your source directory on the new platform, type: 

 yorick -batch make.i
 make

This will change the path to the Makefile template to the appropriate location on the new platform. Then make builds your custom version on the new platform. (Yorick versions older than 1.3 used a different system of Makefile templates; this command should convert your old Makefile to the new form. The original will be renamed `Makefile.old'.

The goal of this system is portability. The basic idea is that all the platform specific problems can be solved once in the Makefile template (or in the several templates), so that you can easily move your packages from one platform to another.

LLNL Disclaimers