Dear Diary: The Eternal Ramblings of Trevor Watson

I'm not really sure why anyone else would want to do this, but we have a big application written in Python that we have been experimenting in calling from Java using Jython. The complexity was that the application uses cpickle as part of its communications layer, so we needed to initialise Jython's copy of cpickle before we could use it. This was, in fact, no mean feat and took several hours of poking around in the Jython source to get to the answer!!

So, in the interests of saving other people jumping through a similar hoop in future, I've documented what worked for me here...

The Java magic you need to initialize Jython and (in particular) the cpickle class is:

    private static void initPython()
    {
        PySystemState.initialize(PySystemState.getBaseProperties(), null, null);
        PySystemState sys = new PySystemState();
        Py.setFrame(new PyFrame(null, sys.__dict__, sys.__dict__, null));
        cPickle.classDictInit(sys.__dict__);
    }

I'm sorry that I can't tell you what each step does because other than the first step, which is the main initialisation for Jython, I really don't know. It's enough for me that it actually works

Once you have the code set up and compiling okay, when you call java, you need to tell it where to find Jython:

java -cp my-app-dir:/local/packages/jython/jython.jar -Dinstall.root=/local/packages/jython my-app

This enables Jython to correctly find its registry, in which I'd already updated python.path to include the path to Jython's Python dir ("Lib") and our own Python code dir.

However, not content with this suffering, we decided to write a servlet to access the Python application. This uses the same initialisation as above, but this time we need to tell Jython what the python.path should look like, since we don't want to be setting global JVM variables in the webserver for one application.

So the magic to add to your servlet's init() method should look something like this:

    public void init() throws ServletException. IOException
    {
        Properties pOverride = new Properties();
        String jythonPythonPath = getServletContext().getRealPath("lib/jython") + "/Lib";
        String pythonPath = jythonPythonPath + File.pathSeparator + "/my-app/lib/python";
        pOverride.put("python.path", pythonPath);
        
        PySystemState.initialize(PySystemState.getBaseProperties(), pOverride, null);
        PySystemState sys = new PySystemState();
        Py.setFrame(new PyFrame(null, sys.__dict__, sys.__dict__, null));
        cPickle.classDictInit(sys.__dict__);
    }

The second argument to PySystemState.initialize is (obviously) a Properties instance which contains those properties which you want to replace (or set, in our case) during Jython's initialisation. We only set python.path but if you look in the Jython registry, you can see what other options there are.

Note that this code could (and probably should) go into a static initialiser rather than a class method, but the servlet was really only a test harness, so it wasn't coded for style/robustness/whatever!

Finally, we set up the WAR file build so that it copies the Jython installation directory into "lib" at the root of our WAR file. This is probably overkill, since we already have jython.jar in the jars for the build and it only appears to need the "Lib" directory (which contains the Python code), but it was expedient and I haven't had time to test paring it down yet.