Theory

The Python/C API is very complete, but deals with Python objects in a very indirect and non-typesafe manner--by providing a horde of functions, all of which take an opaque pointer to a Python object as their primary argument. This allows all manner of object manipulation, but does so through an ungainly pointer interface--and also allows you to, for example, try and multiply an array by a set, or extract dictionary keys from a string--both of which will result in errors. To simplify things for the Eiffel programmer who will be using Python objects, and to make the entire process more type safe, we decided to wrap collections of features in "shadow classes"--in other words, Eiffel classes which stored an opaque pointer to the appropriate Python object, and accessed the Python/C API through standard Eiffel feature calls. Ideally, this would allow the developer to access Python objects as if they were Eiffel objects in as transparent a method as possible.

In order to do this, we decided to follow the Python/C API as closely as possible. As it happened, the Python/C API grouped functions into a rough class hierarchy anyway--for example, an "abstract object" set of functions could be used to modify any object at a high level (reference counts, attribute get/set, etc.). These made the ideal foundation for a simple PYTHON_OBJECT class, which would prove to be the base class for our entire hierarchy, and set the stage for simple object manipulation. The PYTHON_OBJECT class contained features for constructing a "shadow object" either with a "new reference" python object which already showed a positive reference count (despite the fact that nothing referenced it yet), or a "borrowed reference" object, which was referenced elsewhere. (A side note: in the following discussions, the uncapitalized term "python object" refers to the object in "python space"--ie served by the interpreter. The capitalized Eiffel class name PYTHON_OBJECT will be used to refer to an object managed by the Eiffel runtime, which holds an opaque pointer to a Python-space python object. The distinction is important but easy to deal with in practice).

The distinction between a "new reference" and a "borrowed reference" is important. Many of the Python/C API functions allocate a new object and return it with the reference count already incremented. If that value is then incremented again and decremented once, the object will not be freed during the running time of the program--obviously undesirable. But other Python/C API functions in fact do return references to existing objects--and if the client doesn't increment their reference count initially but does decrement it, the object will be freed prematurely--potentially even more disastrous.

To deal with this, we allowed three creation procedures for PYTHON_OBJECT objects:

• make_from_new_reference assumes that the pointer passed for object creation is a "new pointer" and does not increase the reference count of the python object on creation.

• make_from_borrowed_reference assumes that the pointer passed for object creation is a "borrowed reference" and does increase the reference count of the python object on creation

• make_from_python_object takes an argument of PYTHON_OBJECT and treats its opaque python object pointer as a borrowed reference

This covers creation of a new object from a python object pointer, a process that's fairly easy to deal with and allows the client to create PYTHON_OBJECTs from any pointers they get. But how to deal with object destruction?

Eiffel is a garbage-collected language, while Python is a reference-counted language (which, most of the time, really amounts to the same thing for the software developer). Since Python objects are reference counted, and since the reference count is available for examination and manipulation, all that was necessary to handle this from the Eiffel side was to redefine the dispose method, which handles object destruction on garbage collection. With that done, the construction and destruction of Python objects from within Eiffel was relatively painless. By allowing PYTHON_OBJECT to inherit from the Eiffel classes COMPARABLE (allowing object comparison using is_equal and the less-than (<) operator) and HASHABLE (allowing PYTHON_OBJECTs to be used in hash tables, etc.), we squeezed some additional mileage out of the bargain.

Once the initial PYTHON_CLIENT class was in place, we were able to flesh out the rest of the object hierarchy, described in the next section