Performance improvements

January 23rd, 2006 at 2:43 pm (2 years, 3 months ago) by Andi Vajda under chandlerdb

In the run up to the Chandler 0.6 release, a lot of performance improvements were made to the repository codebase. Many areas of Chandler were also reworked somewhat to make better use of performance critical repository features.

Using Python’s hotshot profiler to identify heavily used code and then measuring the performance of individual code snippets with the timeit module a number of significant performance improvements were made in the area of bi-directional reference collections, collection indexes, item loading and item saving.

  • A lot of heavily used code was moved to C using Python’s C API. For example, the base class for a bi-directional reference collection element, LinkedMap was moved to C.
    Similarly, the base class for collection indexes, SkipList was moved to C.
  • Many Berkeley DB calls were made directly from C, eventually bypassing Python’s bsddb module completely. In particular, moving the secondary DB indexing calls to C yielded some nice commit improvements.
  • Major performance improvements were made possible by the increased use of collection indexes to locate items of interest. For example, finding all calendar events relevant to a given week is now done by computing an intersection of two collection indexes.
Performance is an on-going task one is never really done with. So far, these performance improvements were not achieved at the expense of maintainability or flexibility. The underlying repository format has seen many changes but the public repository APIs have remained relatively stable.

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • Reddit

Leave a Reply