Your first module is always hardest to get working. Not if you follow our instructions. This is an extract from the new book by Mike James that helps you combine the speed and power of C with the versatility and ease-of-programming of Python.

Extending & Embedding Python Using C

By Mike James

Buy from Amazon.

Contents

       Preface

1. Extending And Embedding Python
2. A First C Module Using Linux ***NEW!!!
3. A First C Module Using Windows
4. Module Basics
       Extract: A First Module
       Extract:  Pi 
   
5. Arguments
6. Returning Python Objects
7. Objects And Attributes
8. More Complex Objects – Tuples, Lists and Dicts
9. Errors, Exceptions And Reference Counting
       Extract: Exceptions 
   
10. Bytes And Strings
11. Modules And Attributes
12. New Types
13. Advanced Types
14. Threads And The GIL
15. Embedding Python
    

<ASIN:B0CK3X93KF>

Module Basics

Now that we know how to build a module, the time has come to discover what goes into coding a module. Mostly this is a matter of making use of the C API and this is the subject of most of this book. There are some basic things that you have to do to make a C program into a module and this is where we start our examination of the API.

The API and PyObject

The Python C API is a collection of functions and “objects” with names that start Py or _Py. The ones named Py form the public API and you can use them. The ones named _Py are supposed to be private and you shouldn’t use them.

The most important principle to understand is that everything in Python is represented in the C API as a PyObject – integers, lists, dictionary, functions, everything. The PyObject is a struct that contains very little information about the object – a reference count, see later, and a pointer to a type object which describes the object in question. A PyObject pointer can reference any Python object, but to make use of the object you may have to cast to a more specific type.

The principle is that all Python objects are represented by a specific C struct that is an extension of PyObject and as a result any Python object can be referenced by PyObject*.

The C API contains many of the functions that implement Python. In many cases there is a simple equivalence between Python functions and C functions. For example, the Python function dir(object) returns a list of names of the current locals for the specified object. The C API equivalent is pyObject_Dir(PyObject*) which returns a PyObject* pointer to a list of strings corresponding to the locals for the object referenced. The equivalence isn’t exact, but if you know your Python you know quite a lot of the C API.

I suppose you could say, in the style of the zen of Python, in the C API everything is a PyObject.

The Initialization Function

An extension module is just a shared library which exports an initialization function which has the signature:

PyObject* PyInit_modulename(void)

where modulename is the ASCII name of the module. If you want to use a Unicode module name you have to use a more complicated initialization called multi-phase initialization, see Chapter 11. When you import a module Python locates the file modulename.so under Linux or modulename.pyd under Windows and then tries to call PyInit_modulename.

This also implies that PyInit_modulename is exported from the shared library, i.e. it can be called by an external program. You don’t have to do anything to export a function under Linux, but you do need to add __declspec(dllexport) for Windows. The solution is to simply use the PyMODINIT_FUNC macro which will automatically add the necessary __declspec(dllexport) under Windows.

The pointer to PyObject returned should be a fully initialized module as returned by:

PyObject* PyModule_Create(PyModuleDef* def)

or

PyObject *PyModule_Create2(PyModuleDef* def,
                               int module_api_version)

The PyModuleDef struct has fields that determine how the module will be treated by the system:

struct PyModuleDef {
  PyModuleDef_Base m_base;
  const char* m_name;
  const char* m_doc;
  Py_ssize_t m_size;
  PyMethodDef *m_methods;
  PyModuleDef_Slot *m_slots;
  traverseproc m_traverse;
  inquiry m_clear;
  freefunc m_free;
};

Not all of the fields have to be used. The most important are:

• m_base always initialize this member to PyModuleDef_HEAD_INIT

• *m_name name of the module

• *m_doc docstring for the module

• m_size memory needed to store module state usually -1
  for a simple module

• *m_methods pointer to array of PyMethodDef structs
  defines the methods provided by the module

The format of the PyMethodDef struct is:

struct PyMethodDef {
    const char  *ml_name;   
    PyCFunction ml_meth;    
    int         ml_flags;   
    const char  *ml_doc; 
};

The fields are:

• ml_name name of method

• ml_meth pointer to the function that implements the method

• ml_flags constant indicating how to call the function

• ml_doc the docstring for the method

The ml_flags constant determines the calling convention for the function. For example, if it is set to METH_VARARGS then the function is passed two parameters. The first is a self which gives the instance if the function is called as a method and the module object if it is called as a function. The second is a tuple holding all of the parameters. Of course, both parameters are *PyObjects.

The array of PyMethodDef structs is terminated by a sentinel value:

{NULL, NULL, 0, NULL}

To summarize:

• You have to create an array of PyMethodDef specifying the methods/functions that the module provides.

• This is added along with other data to a PyModuleDef struct which defines the module.

• The PyModuleDef is used in the PyModule_Create function to create a module object which is returned by the PyInit_modulename function.