CherryPy Project Download

CherryPy PyLucene integration

Problem

There are some problems in calling some PyLucene API from CherryPy code. I got following Java exception when I tried calling PyLucene.IndexWriter from CherryPy.

writer = PyLucene.IndexWriter( "c:/index", PyLucene.StandardAnalyzer(), True)
JavaError: java.lang.NullPointerException

The same API works fine if called from python console.

>>> PyLucene.IndexWriter( "c:/index", PyLucene.StandardAnalyzer(), True)
<IndexWriter: org.apache.lucene.index.IndexWriter@17f3fa0>

Reason

The reason for conflict lies in the threading mechanism used by CherryPy.

CherryPy page handlers never run in the main thread, a new thread is created for handling new request. These new threads are local threads created from threading.Thread.

Basically, for a python only main thread can call into PyLucene without any problems. For other threads to work with PyLucene, the boehm-gc component of the java runtime ( i.e. the garbage collector) should be informed about their creation. If these threads are created as an instance of PyLucene.PythonThread, it will make sure that the thread is created via libgcj and libgcj garbage collector is aware of it.

Solution

The idea is to use PyLucene.PythonThread instead of normal threading.Thread in CherryPy source code.

I did following modifications in CherryPy source code with CherryPy 2.2.1 and PyLucene 2.0.0 on Windows 2000 to work together.

The cherrypy code should be available in following directory inside python installation directory. python_installation_dir\Lib\site-packages\cherrypy

Files which have threading.Thread are:

  • _cpserver.py
  • _cpwsgiserver.py

You need to add statement import PyLucene and replace threading.Thread with PyLucene.PythonThread in above two files.

Also disable CherryPy's autoreload (by adding autoreload.on=False to configuration file) because autoreload uses low level threads.

These changes worked fine in my case.

More Information

For more details, see http://lists.osafoundation.org/pipermail/pylucene-dev/2006-June/001107.html

For porting this solution to Fedora Core 5 Linux, visit http://lists.osafoundation.org/pipermail/pylucene-dev/2006-July/001213.html


Above information may not be correct, as its just taken from my experience, Let me know if there is any mistake.

Pravin Shinde [getpravin at gmail.com]

Attachments

Hosted by WebFaction

Log in as guest/cherrypy to create/edit wiki pages