The Chandler Project Blog

Over the past 2 weeks, we’ve been plugging away at a
Work Queue for Chandler Desktop. By we, I mean Grant.

The thinking behind the work queue is to isolate the half-dozen or so usability issues we feel are the biggest blockers to new users understanding what Chandler is, and how to set about using it. In that sense, the bugs in the Work Queue are not necessarily the most egregious bugs. For example, several serious crash bugs and the false-positive pop-to-NOW bugs were omitted from the list. They are also not necessarily the most ‘oft-requested’ features from our current users. For example, the ability to spawn sub-tasks, aka clusters and Print were omitted from the list.

Instead the focus is to remove the hurdles that all users run into within 30 seconds of downloading and launching the application. Many of the hurdles are conceptual. I’ve sent a number of messages to the Chandler-Dev list dissecting these conceptual barriers:

• Pared down Desktop UI – Part 1
• Next Usability Hurdle: Answering the question: What should an item be?

1. New users are confused by the abundance of email functionality in the user interface. Is Chandler meant to replace your email application or not?
2. New users are confused by what genre / category of software Chandler falls into. Is Chandler meant to replace Outlook? Apple iCal? My journal / notebook? Email, Tasks and Calendar functionality makes Chandler look like a traditional PIM. But there’s a lot of missing functionality typically associated with PIMS (e.g. contacts, full-blown support for email, scheduling, etc). You could also *try* to understand Chandler as a super-duper Task Manager, but again, there’s a lot of missing functionality typically associated with robust Task Managers (e.g. dependencies, assignees, %done, milestones, etc).
3. New users don’t understand what an ‘item’ is supposed to be, as a result, they have difficulty getting started putting data into Chandler. It’s not that mechanisms for creating new items are hard to learn. It’s that the concept of an ‘item’ is too abstract. When faced with the question: Do you have any items to put into Chandler, many users simply go blank.
4. New users have trouble understanding Chandler’s mental model for sharing. In particular, there appears to be a lot of cognitive hoops to jump through just to get your data up on Chandler Hub. This problem can be entirely addressed from the Desktop side. I’ve started a separate thread on establishing a similar Work Queue for the Chandler Hub UI.

We are addressing these issues by:

1. Doing a better job of introducing the product to new users (e.g. better demos; better, more accessible quick start guide);
2. Smoothing out crucial workflows (e.g. guiding users through sharing set-up);
3. Simplifying the user interface, removing concepts that might mislead users into thinking Chandler is a PIM / Task Manager when it’s not (e.g. removing superfluous email buttons from the toolbar; removing the concept of a task);
4. Tweaking Chandler concepts to be more immediately understandable (e.g. changing the concept of an item to a note; changing the task stamp into a star stamp – I will be posting a more in-depth discussion of this issue);
5. Doing a better job of communicating Chandler’s core value through the user interface (e.g. improving the Notes field in the item details to encourage users to think of Chandler as *the* destination for dumping ideas, stray thoughts and questions; providing visual feedback when a shared collection has unread edits to showcase the power of Chandler’s collaboration workflows.

That being said, it is still important for us to keep track of critical bugs, bugs getting in the way of day-to-day usage and features that would greatly enhance the Chandler experience for current users.

I’ve included a list of these bugs and feature requests at the bottom of the Work Queue. If any of these are expedient to take on and/or we end up with extra time on the Desktop side before we’re ready to declare 1.0 for the project as a whole, it would certainly be worthwhile to tackle some of these bugs.

There is also the larger issue of performance, in particular start-up time. There isn’t a lot we can do in the short-term to address performance. We *have been* working on a rearchitecture project that would address many of the performance problems people run into today, but a completely re-architected Chandler Desktop is not going to emerge in the next 3-6 months.

I *can say* that we hope to alleviate some of the pain caused by slow start-up time with strategically placed web widgets. For example, we hope to deliver a Quick Entry web widget in the next month or so that would allow users to quickly get ideas and notes into Chandler via Chandler Hub without having to fire up the Desktop app. These are the kinds of things we can realistically accomplish in the next few months.

In the meantime, our first and foremost priority is to turn the Chandler Desktop interface into an effective communicator of what it’s useful for and how it’s useful.

Last week the staff got together in person and took a critical look at where the project is now and what we want to achieve in the next year.

We asked ourselves a series of questions, including: “We have some successful, enthusiastic users who rely on Chandler daily: how are they using it? What are their success stories? How can we build on that and grow the user base?” and “Why do we only have 100s of unique desktop users syncing to the Hub daily when many 1000s of users have downloaded the desktop?”

Looking at users who *are* successful and potential users who run into difficulties, we had this insight: Chandler succeeds at meeting the needs of users who are tracking ‘knowledge work’. What do we mean by this? We’ve observed people tracking ideas and questions for tasks they need to do — the kind of things people otherwise might jot down in notebooks, on scraps of paper or in text files. They selectively add important email messages to Chandler (the ones they might have flagged if they were only using their email client). They share collections of these items with other people as they develop an idea, using Chandler to record the ‘knowledge’ about a shared project. Quick item entry, stamping items onto the calendar or a task list, and items in multiple collections are all signature features that help users evolve their ideas into tasks and projects. While the design includes tasks and a hard calendar landscape, Chandler is not oriented around calendaring per se or around a complicated task and project landscape with many dependencies. We don’t have enterprise scheduling features, for example, or task and project dependencies. Instead of adding more features for calendaring and task/project management over the next year, we’d like to build on Chandler’s more unique strengths, excelling at meeting the needs people who currently find Chandler compelling.

Of course, many users who *do* have similar needs to the successful users run into barriers when trying to use Chandler. The application is too slow for their hardware, they are overwhelmed by non essential functionality visible in the application, they don’t know how to get started, they run into a roadblock when creating an account, etc. We want to remove these barriers.

The flip side of this is that some people who have been drawn to the project are really *not* target users. Chandler doesn’t meet their needs because Chandler is not designed for them. We just don’t have the resources to make all of these people happy — we really do have to focus. More on this in the “what we are not doing” section below.

Another idea that came up in our discussions: we want Chandler to be more viral. We want Chandler to be easy to explain to others. We want Chandler to be found in contexts where people are already spending time. We want Chandler to be of great use to the individual using Chandler on their own, and to be even more useful as that user pulls in other people to collaborate. We want happy users be successful evangelists for Chandler.

What we are doing next

We made some important high level decisions about what we’re going to be focusing on in the next year, and in particular for the next 3-4 months. It was most concrete for us when we discussed exactly what each person would be working on (in particular what each developer would be working on), so I’ll use that to frame our strategy.

Grant Baillie: Incremental progress on the desktop. We decided that we can’t stop and take time out to rewrite the desktop or build a complete web based home for our target users, so that means continuing to maintain the current desktop code base. Grant will focus on bugs and small feature changes that remove barriers for users who otherwise would value Chandler’s feature set. We are not far from a 1.0 version of the desktop. One of the first projects that Grant is working on is a pass at simplifying the UI by removing rarely used email functionality — reducing the number of new concepts that users need to get started.

Travis Vachon and Jeffrey Harris: Web widgets. We’re turning our focus on the web away from replicating the desktop functionality, and toward making Chandler more viral and better at facilitating idea gathering and collaboration for our target users. Travis and Jeffrey are going to build web widgets that might be deployed in different contexts — iGoogle, Facebook, on an iPhone, etc. Widgets will give read/write access to a particular item (instead of a whole collection), allow users to search for a particular item, allow quick entry of items, etc. We’ve started the design work on the chandler-dev list. One requirement for the web strategy is that these widgets should be compelling to a new user who does not use the desktop, in addition to providing features that complement the desktop. Eventually, the widgets can be building blocks for developing out that web based home for our target users.

Brian Kirsch: Thunderbird plugin. Brian has already blogged about this idea; we’ve decided to proceed with it. We think it will be a great way to introduce Chandler ideas to a new audience.

Jared Rhine: Email related features on the Hub. Jared will explore adding email notifications/reports from the service, and email as a way of entering data into the service. Again, these features are a way of making the Hub more viral — data access from other contexts where people spend their time. This work is in addition to Jared’s many other responsibilities (managing the Hub service, build and release management, etc.).

Randy Letness: Chandler Server improvements and support for web widgets. Randy will continue with security work that has long been planned, as well as server support necessary for the new web widgets.

Phillip Eby: Desktop rearchitecture. We will continue to make a modest investment in the rearchitecture project that we embarked upon last year, as the rearchitecture is our path to really solving desktop performance problems. The rearchitecture will also provide a much better platform for other developers to contribute to the project.

Mimi Yin: Product design and strategy. Web strategy and web widget design, Thunderbird plugin design, desktop prioritization, project website improvements, etc.

Sheila Mooney: Evangelism strategy. Develop a pitch and a demo, meet with stakeholders, etc. One of the barriers for potential users and partners is really a marketing barrier — helping people understand what Chandler is intended for.

My job is to manage the project overall and give the project its best shot at thriving beyond the end of the year.

What we are not doing

We are not trying to be an open source Exchange/Outlook competitor. While this would be a worthy goal, it is not our passion and we don’t really have the right product or organization to pull this off. It has been a misperception in the press that this has *ever* been our goal — probably because so many people want an open source competitor to Exchange/Outlook. We have not designed Chandler for enterprise calendaring. Actually, the Microsoft product with the most overlap with our design objectives is probably OneNote.

Being a CalDAV reference implementation is not a priority. We believe that calendaring standards are important and hope that they are adopted in the world, and much of OSAF’s energy in previous years was directed at making that a reality. With our current need to focus, however, our strategic objective wrt calendaring standards is to be able to interoperate with popular clients (e.g. iCal and Lightning). For this reason and because we are not trying to do enterprise calendaring, we will not be implementing CalDAV scheduling.

We are not trying to be a GTD specific tool. Yes, we paid a lot of attention to GTD as we were designing Chandler and we were inspired by ideas from GTD. We also looked at how people who have never heard of GTD really use their inbox to manage their world, and taken inspiration from that as well. When getting into the specifics of Chandler’s design, we focused on user scenarios and workflows that would make our target users successful — not necessarily designing to GTD processes. We’ve recently taken a critical look at how well Chandler meets the needs of someone following the GTD process strictly. In doing so, we realized that we didn’t want to prioritize staff time to work on changes that would make Chandler a GTD specific tool, and that Chandler’s philosophy is different enough from GTD that it would be misleading to call Chandler a GTD tool. We’ll need to change our messaging — our landing page, blog and other places that refer to GTD. That said, we recognize that some developers are interested in features that would help people use Chandler for a GTD process, and we encourage anyone who wants to contribute code for such features.

We are not going to add full email functionality to Chandler desktop, or contacts (at least not in the next year, not on OSAF staff time). Yes, we know that many people want these features, and we’d love to design them and implement them if we had the resources, but we don’t have the resources in the next year. Again, we’d *gladly* accept code contributions that implemented these features, or additional funding to implement them. We are willing to spend time to help make people successful contributing code for these features (or other features that complement the current product).

To help focus as a team, we have consolidated our various mailing lists in the following ways:

1. Chandler-dev is now the working list for all things pertaining to the Chandler Project. This includes planning, design and bug prioritization for both Chandler Desktop and Chandler Hub as well as general interest topics like the project wiki, evangelism, community, and governance.

2. Cosmo-dev remains a separate list for those interested in just the server. Planning and project management discussions that used to happen on Cosmo-dev will move to Chandler-dev.

3. General, Design and Service-dev are now inactive. These conversations will move to Chandler-dev. If you are on one of these lists and want to continue following the day to day work of the project, subscribe to Chandler-dev.

4. Chandler-users will remain the best place for users to ask questions and report issues. Email us at chandler-users at osafoundation dot org.

5. Announce will continue to broadcast major releases and other newsworthy information.

All other public lists will stay the same.

We are also maintaining a Chandler Project collection where you can keep up with more day-to-day work. This is where we’re keeping track of feature ideas, upcoming blog posts and meetings.

View-only URL: https://hub.chandlerproject.org/pim/collection/f9fe3636-c489-11dc-dba3-f100459c9336?ticket=37960wr571

Finally, we will be making more use of the blog to discuss design direction, product strategy and to keep everyone abreast of project status and new releases.

Katie’s post OSAF 2.0 Team
seems like a good opportunity to introduce myself in this space. When I first joined OSAF
I was asked to do this by Pieter
Hartsook but a combination of a bad memory and busy schedule has kept this task triaged Later.

I’m originally from a small town about 45 minutes outside of Portland, Maine. My first brush with software development
came during the summer of 2006 when, alongside a 6 day-a-week summer camp job, I participated in Google’s inaugural
Summer of Code program. My project for the summer found me working with
the GNOME Project implementing an experimental “panel extension” system.

I found Chandler while looking for a Linux calendaring client during my senior year at Williams College
and after an internship on the Desktop team working on a project to better integrate the Twisted IMAP server into Chandler I was hired full-time
as a server/ web front-end developer.

Most of my work since then has straddled HTTP, working mostly at the protocol level on the server and client side, with occasional
forays down into the depths of our database layer and up to the shallow waters of user interface implementation. Most recently I’ve
been updating our JavaScript code to use the 1.0 release of the Dojo toolkit.

A second project I’ve worked on recently (alluded to in the title of this post) is the first of what I hope to be a series of
interesting hacks designed to expand Chandler into the maze of nooks and crannies that is contemporary personal information management.
One of the more important lessons I’ve learned while working in this space is that everyone has a different system
for tracking and managing the various things they want to accomplish both in work and in life. While semi-standard systems like Chandler’s
Triage Workflow and David Allen’s
GTD can help, even the most hard-core practitioners will make adjustments
to work with their own personal circumstances. As developers of software designed to
“serve the way people actually work, independently and together“, I believe it is our
job to lead the way in bringing our ecosystem to people’s real needs.

So without further ado, let me introduce chandler.el, a module for interacting with
Chandler Server using Emacs, a popular text editing environment.
Instructions for installing and using it can be found at the link above. The current implementation is decidedly rough, but is ready for
some real world use and feedback.

This offering is definitely on the techie side, but I hope it serves as a proof of concept for a general class of lightweight
applications that have the potential to bring Chandler to the system you currently use to track your life. There is currently
a discussion on chandler-users@osafoundation.org
in which I’ve solicited ideas for more applications like this, please feel free to chime in there or in the comments to this post
with yours!

In the future, updates about chandler.el will be posted mainly on my personal blog occident.us alongside information about whatever I happen to be working
on or thinking about at the time. If you’re interested in what I do, do check out that space.

Mimi and Katie talked in ealier posts about expanding the Chandler Universe
by developing a Plug-in for Thunderbird.

The plug-in would allow Thunderbird users to interface directly with the Chandler Hub.

Some of the current ideas on what actions the Thunderbird Plug-In would perform are:

1. Sync messages in IMAP Drafts/Sent folders with Chandler Drafts/Sent messages.
2. Stamp a message and add it to a list of collection(s)
3. Assign Triage status to message items
4. Map individual IMAP folders to collections
5. Receive notifications from Hub Service when:
a. New / Edited items in a collection
b. Items tickled to NOW

What features would others like to see?

Is the idea of a plug-in to Thunderbird useful to the Chandler User base?
Will a plug-in help attract new users?

If you have any suggestions for features that you would
like to see implemented in a Thunderbird plug-in now is the
perfect time to share them.

We’ve added Ubuntu Gutsy Gibbon and Mac OS X “Leopard” to our supported
list, and are just phasing out Mac OS X “Panther” support, so it was
time to clarify what platforms are officially supported.

Linux:

• Ubuntu Dapper Drake (tarball installs)
• Ubuntu Gutsy Gibbon (deb install, using system python)

Windows:

• Windows XP

Macintosh:

• PPC Mac OS X “Tiger”
• Intel Mac OS X “Tiger”
• Intel Mac OS X “Leopard” (using system python)

Additionally Chandler may be built and may run in the following
configurations but is not officially supported/tested, although
we may fix issues if reported, especially if given patches:

Linux:

• Ubuntu Feisty Fawn (using system python)
• Ubuntu Gutsy Gibbon 64-bit (using system python)
• Other Linux variants

Windows:

• Windows 2000
• Windows Vista

Macintosh:

• PPC Mac OS X “Panther”
• PPC Mac OS X “Leopard” (using system python)

The Chandler plugin framework allows developers to extend the application’s functionality in new and fun ways. For example, I have been working on a Google Calendar sync plugin (more about that in another post) and a plugin to hook into Twitter. Twitter is sort of a global IM framework, and I thought it would be cool to have Chandler be able to subscribe to Twitter activity and display it in the UI. The first Twitter plugin I wrote used HTTP polling to download Twitter messages, but I have now replaced it with a more general XMPP plugin. XMPP (formerly known as “Jabber”) is an open IM protocol, and is much better suited to real-time data exchange. This plugin enhances the Chandler accounts dialog to allow you to set up an XMPP account. From then on, any XMPP message the account receives (not just Twitter, but Jaiku, etc.) will be added as a calendar event to an “XMPP” collection in the sidebar. These events can be stamped as tasks, dragged to other collections, etc. You can view them either in the Calendar or in the table view.

For example:

The plugin is still considered experimental, and you shouldn’t try it in a Chandler that contains data you care about, but if you’d like to play around with it, the tar file can be downloaded from the python ‘Cheeseshop’ (the central registry for Python packages). Once you have downloaded the file, use Chandler’s “Plugins > Install” menu to select it. Assuming you already have an XMPP/Jabber account somewhere (for instance, I use Google Talk), and you have already linked that account to get notified by Twitter, Jaiku, etc. via IM, you will be able to add a new “XMPP/Jabber” account into Chandler’s “File > Accounts” dialog. After you have closed the accounts dialog, restart Chandler, give it a minute and your Twitter messages should start appearing in the XMPP collection.

The XMPP account settings you’ll need to use Google Talk are:

   User name: username@gmail.com
   Server: talk.google.com
   Resource: (just leave blank)
   Port: 5223, Use SSL Checked

This new plugin will soon form the basis for other IM-based services like instant-sharing.

~morgen

With the GPL’ing of Java and Lucene’s pending move to using Java 1.5,
I decided to give PyLucene without gcj a try.

One of the best features of gcj is its C++ interface. Its ability to expose Java classes as C++ makes it very easy to invoke Java from C++.
This feature is unique and sorely missed when moving off of gcj.

The Java Native Invocation Interface makes it possible to invoke Java
classes from C++ but is rather crufty and verbose to use by hand.

I started by implementing a C++ code generator that generates C++ wrapper
classes that hide all the JNI invocation cruft from the C++ programmer.
These wrapper classes in effect offer a very similar C++ interface to what
is offered by gcj’s CNI.

While I was implementing this it became apparent that the C++ wrappers for
giving access to Java classes from Python could be generated at the same
time.

Eventually, I ended up writing a new C++ code generator, that I called jcc.
JCC is able to generate C++ wrappers for accessing Java classes from Python
and C++.

This code generator is now used to generate a new flavor of PyLucene, called
PyLucene with JCC. JCC has no built-in knowledge about Lucene and could be
used to generate wrappers for any Java library that uses similar coding
styles and techniques than the ones used by the Java Lucene developers.

JCC could in fact become a project of its own. At the moment, it is part of the
PyLucene with JCC source tree.

The PyLucene source tree got split into two forks, a gcj fork containing the
original PyLucene and a new jcc fork containing the new sources, consisting
mainly of unit tests, samples and the sources to jcc since _all_ of
PyLucene, except for extension code, is now generated by jcc.

For more details, please refer to the new README files:

• PyLucene README
• PyLucene with JCC README
• jcc README

For Preview, we’ve made a huge effort in updating and expanding Chandler Project documentation.

Visit our new Chandler Project website.

From here, you can Download Chandler Desktop, Sign up for a Chandler Hub account as well as find our newly updated Vision Document and Feature List, complete with screenshots and informational demos of both Chandler Desktop and Chandler Hub in action.

For a blow-by-blow list of what’s new and what’s changed since the 0.6 release, see Preview Release Notes.

There is now a consolidated FAQ that covers topics ranging from ‘What license is Chandler under?‘ to ‘Can I use Chandler for Email?‘ and developer questions.

We are also working on a Get Started Guide for step by step instructions on everything from setting up accounts and sharing to a quick overview of Chandler’s core information management workflows. We expect the guide to be a ‘living document’ that will be continually improved to help new users ramp up. Check it out and give us feedback!

If you run into problems using Chandler, take a look at Known Issues and Troubleshooting.

If you think you’re problem is new, subscribe to the Chandler-Users mailing list and send us an email. If you’re pretty sure you know what’s going on, following the directions on the Report a Bug page and log a bug.

Here is an overview of how you can Get Involved, including an up-to-date list of Starter Projects.

Last but not least, if you’re looking to learn more about Chandler as a project, dig through our newly re-organized Project Wiki where you will find an overview of the Desktop, Server and Hub Service, detailed design and developer documentation as well as day-to-day planning, bug-tracking and notes.

I’m pleased to announce that the Chandler project has hit our Preview milestone! We now have public-beta quality releases of our products; we believe them to be full featured enough and stable enough for daily use. Check out a full overview of features (including screenshots and screencasts). Download Chandler Desktop, create an account on Chandler Hub. Check out the source. Get involved in the project, help us build a really great 1.0 release.

Chandler Desktop
We released 0.7.0.1 of the Chandler Desktop yesterday, September 10. (In the spirit of responding to user feedback quicky, we fixed a problem with the release found by a user before we made the announcement).

Chandler desktop adds a central dashboard for managing tasks, notes, events, and messages to the basic calendar functionality found in the 0.6 release. You can share calendars, task lists, messages and notes in collections that can hold whatever you choose to put in them, regardless of data type. The performance has improved greatly, the application has basic search functionality, and now there’s a way to to manage and resolve conflicts on shared data. You can collaborate on individual items via email with the ability to edit and update messages you’ve already received or sent. Although Chandler Preview is not meant to replace your email application, you can configure your IMAP account so that Chandler can see some messages from your regular mail client.

We’re currently planning a 0.7.1 release in about a month, to quickly iterate on the Preview release. We’ll make use of user feedback to identify problems and drive priorities.

Chandler Hub
We upgraded Chandler Hub to the 0.7.0 release of Chandler Server on August 27th.

The latest release of Chandler Hub also adds a dashboard for managing tasks (and other kinds of data) to basic web calendaring features. Friends and family can access shared collections from the web without having to create an account or log in. We’ve focused on workflows that let desktop users share data with others using the web; this release is a major step towards realizing the full Chandler vision on the web as well as the desktop.

We’ll be upgrading the service with small fixes on a weekly basis for the next month or so, fixing minor glitches and adding support for Safari.

Chandler Server
The Chandler Server 0.7.0 release is available for download as a ready-to-run bundle. We’ll be creating 0.7.x releases as we improve the Hub service. In parallel, we’re working on a 0.8 release that is focused on interoperability with other calendar applications and services such as iCal, Google Calendar and Evolution.

Try out Chandler
We believe you can now feel confident putting your data in Chandler. We have migration features to make upgrading easier, and will do our best to support people if they run into problems. (The chandler-users@osafoundation.org mailing list is a great forum for support). These releases do have some bugs and rough edges, so Chandler might not yet be appropriate for mission critical uses. We are using the desktop and hub internally day to day for our office calendar, personal calendars, personal project management, and several small group task lists. We hope you’ll join us using Chandler — let us know how you like it! We’ll use your feedback to make a better 1.0 release.

Get Involved
One of our goals for this Preview milestone is to grow outside involvement with the project. We want users to log bugs. We want designers to collaborate with us on ideas. We want developers to submit patches fixing bugs that annoy them, create desktop plugins with their favorite features, and write clients that take advantage of their data on the server. Check out how to get involved.