Helper functions in C--making Python helper macros accessable to SWIG

The problem lay in a few of the exported functions--and it was not obvious what was going on. For example, consider the code fragment below, automatically generated by SWIG:

extern PyObject *Py_CompileString(char *,char *,int );
extern void Py_XINCREF(PyObject *); 
extern void Py_Incref(PyObject *);
extern void Py_Decref(PyObject *);

Most peculiar. It had been a while since we had thrashed with C, but the problem, as it happened, lay in the fact that Py_XINCREF was a macro, not a proper function--and SWIG had real trouble dealing with that because of the nature of the automatically generated code; since the term Py_XINCREF was being substituted with the macro text, it couldn't be surrounded with a type declaration, because what the compiler saw was something akin to the following:

extern void if ((op) == NULL) ; else Py_INCREF(op);

This is obviously not right. But we couldn't just eliminate the type declaration, because SWIG needed that to generate the stub for the destination language. The solution was actually very simple: create a helper stub function which would call the macro. SWIG could then cleanly generate the code to call the helper function, which would then call the macro--several layers of indirection, but it did at least work:

void Py_XIncref( PyObject* o ) 
{
  Py_XINCREF( o );
}

extern PyObject *Py_CompileString(char *,char *,int );
extern void Py_XIncref(PyObject *); 
extern void Py_Incref(PyObject *);
extern void Py_Decref(PyObject *);

Much of the indirection created by SWIG was actually unnecessary. Eiffel in particular supports a very simple external function interface, and we could have avoided one level of indirection in many cases simply by calling the Python/C API functions directly instead of going through SWIG. However, this would have complicated the build process a little bit and increased the overhead (and human error) tremendously as we coded many of the function interfaces by hand. Far better to do a little coding around the macros and let SWIG do its work.

This still generated some concern, since many of the SWIG overhead calls did absolutely nothing for data conversion--unless a complex structure such as a string or array was involved, SWIG simply assigned temporary variables to the parameters and passed them on to the C function:

void _wrap_Py_XIncref(void * s_0) {

    PyObject * _arg0;

    _arg0 = (PyObject *)s_0;

    Py_XIncref(_arg0);
}

...but in practice this probably wouldn't be too much of a problem. For one thing, a good optimizing compiler could well avoid the assignment to a temporary; for another, we are after all describing the interface to a rather slow interpreted scripting language, and the overhead of calling functions in Python would very much overwhelm any overhead from simple assignment to a temporary. Data conversion overhead, such as string or array conversion, could cost much more in both time and speed--but the conversion would be necessary in any case.

By now, we had a fair amount of success: SWIG had generated a compilable Eiffel class, PYTHON_CLIENT_WRAPPER, which neatly encapsulated our exported functionality in Eiffel features, so:

   py_xincref ( o : POINTER ) is
      external "C"
      alias "_wrap_Py_XINCREF"
      end