The Chandler Project Blog

In the run up to the Chandler 0.6 release, a lot of performance improvements were made to the repository codebase. Many areas of Chandler were also reworked somewhat to make better use of performance critical repository features.

Using Python’s hotshot profiler to identify heavily used code and then measuring the performance of individual code snippets with the timeit module a number of significant performance improvements were made in the area of bi-directional reference collections, collection indexes, item loading and item saving.

• A lot of heavily used code was moved to C using Python’s C API. For example, the base class for a bi-directional reference collection element, LinkedMap was moved to C.
  Similarly, the base class for collection indexes, SkipList was moved to C.

• Many Berkeley DB calls were made directly from C, eventually bypassing Python’s bsddb module completely. In particular, moving the secondary DB indexing calls to C yielded some nice commit improvements.
• Major performance improvements were made possible by the increased use of collection indexes to locate items of interest. For example, finding all calendar events relevant to a given week is now done by computing an intersection of two collection indexes.

Performance is an on-going task one is never really done with. So far, these performance improvements were not achieved at the expense of maintainability or flexibility. The underlying repository format has seen many changes but the public repository APIs have remained relatively stable.

This entry was posted on Monday, January 23rd, 2006 at 2:43 pm and is filed under chandlerdb. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.