Here’s an update on the current activities…

We are getting ready to move forward with several new releases.

On the desktop side, Chandler 0.7.5 should be released sometime in the next several days. Last week we held a collaborative QA session on IRC. You can download the latest checkpoint for a preview of all the fixes.

On the server side we will be doing two releases this month. The first one, 0.13, contains a major fix to plug a security hole along with a number of other smaller bug fixes. We have been testing this branch and will likely have the release out later this week. You can get the latest release candidate here and the test spec is available on the wiki.

This will be followed by an infrastructure release, 0.14, where the current UI has been ported to Dojo 1.0. We held a test session on IRC last Thursday and have another planned for tomorrow at 1:00pm. We have been testing against this instance and Travis’ test spec is also on the wiki. We always welcome any additional help testing.

We continue to make good progress on the Quick Entry widget. Jeffrey has been working on a few remaining IE bugs. Brian has sent an architecture proposal to the list for the Notifications widget. Philip Eby recently posted on the list soliciting feedback on his Trellis work.

On the marketing and design side, we have been working on a new landing page which we will deploy and try out on users soon. We will continue to iterate on the new design to refine our product message. This is one of many marketing activities we will be tackling in the coming weeks.

In Part 1 of this post, I described the dilemma we face in the design, trying to balance “avoiding the futility of time management” with a need to keep the LATER section somewhat under control.

In Part 2, I will walk you through some of the discussions we’ve had on the list and my reasoning for not going down the path of providing more granular LATER sub-sections defined around time.

With users piping up about how their LATER sections were becoming unmanageable, we revived discussions around adding LATER sub-sections defined around loose timeframes:

• LATER-Next week
• LATER-In 2 weeks
• LATER-This month, etc…

and improving the sub-sort order so that items in LATER are ordered by when they’re going to happen:

• “LATER-Imminently”
• “LATER-Far in the future”
• “LATER-No date at all”

The latter (improve sub-sort) was always part of the design and is on the short list of bugs to fix after we ship 1.0.

But I worry about going down the path of more granualar LATER sub-sections. I worry about falling into the trap of over-planning, over-emphasizing the importance of defining timeframes and inadvertently pushing users to assign dates they have no intention of holding themselves to, which again, results in big piles of items getting dumped into NOW when they’re not ready to deal with them.

Instead, I think it will be more effective to automatically collect all LATER items that don’t have a specified time frame (items with no alarm, no event dates) into a separate section called “LATER - No date assigned”.*

This smaller, hopefully more manageable “LATER-No date assigned” section acts as a reminder that there are deferred items that need to be reviewed and re-evaluated on a regular basis because they’re not going to magically re-appear in NOW on their own.

To make this list even smaller, we could provide a “LATER-Not really” option for items we create for peace of mind, but actually have no intention of doing, ever.

*Being an open source project that supports plugins means that of course doesn’t stop anybody from implementing an add-on for more granular LATER sub-sections. What I’m discussing here is the out of the box experience.

Interoperability is an important part of the Chandler Project vision.

Chandler is about trying to match the way people really work. And everyone uses lots of tools to get their job done. One of the first things people want to know when considering trying out Chandler software is “Will it work with what I already use? Can I switch back if I don’t like it? Will it work with the tools that friends of mine use?”

We believe the answer to all these questions is “YES”! You can safely and productively start using one or more of the Chandler Project components on top of your existing toolset. Go ahead, try it out! Read more below to learn the details.

For a bullet-list summary of our best-available notes on specific applications and which features are supported with each, see our interop overview.

Import/export

The gold standard of calendar transfer is the “ICS” file (in iCalendar format). Most calendar and task list applications support both import and export of ICS files.

You can try out Chandler Desktop without switching from your current setup. Just export one or more ICS files from your current application, then import those files into Chandler Desktop (continuing to use your current app). To switch over permanently, just export+import again a final time!

If you later decide you’d like to change again, you can export ICS files from Chandler Desktop or Chandler Hub, using those files for import into a wide variety of applications.

We’ve seen import and export work for Outlook 2003/2007, Mozilla Lightning/Sunbird, Apple iCal, and others; it should work with a great many apps, probably yours included.

Note that Outlook doesn’t export full information by default; we’ve found this $10 application from littlemachines produces high-quality exports from Outlook that work well with Chandler Desktop.

In practice, doing ICS import/export can have gotachas. Not all application combinations/roundtrips are 100% perfect. We urge you to keep backups and try out import/export before committing your important data to any application. In Chandler Desktop, we’ve spent a lot of time tuning our import/export routines to handle as many variants and details as we can. Chandler Desktop properly handles events, tasks, timezones, recurrence, and other details. Please report any import/export problems you encounter.

We’ve put together some additional information about import/export with Chandler Desktop specifically, so check that for additional hints and notes.

Synchronization

ICS import/export is great for transferring your data between apps, but it’s a manual process not suited to keeping multiple applications in sync. Usually when you import a data set, your app will overwrite changes you may have in your local copies of those events. It’s hard to make changes in two separate apps.

Chandler Desktop and Chandler Hub both support multiple network sync protocols. Where other applications (Outlook, iCal, etc) overlap at least one of these protocols, interoperability is possible on at least some level.

One main idea to keep in mind when thinking about these various systems is whether a scheme is “read-only” or “read-write” (ie, bidirectional). It seems like read-only (or 1-way) interoperability works more reliably today, but protocols like CalDAV promise a new era of real-time, 2-way synchronization of calendar data between lots of free and for-pay applications and web services.

The big news is that you can do 2-way/read-write calendar and task synchronization today, both privately and shared with other people. Here’s a list of the main ways to do that, based on Chandler Project software.

Webcal, 1-way sync

The most simple network protocol is to take an ICS file (see above in import/export) and post it to the web, so various apps can download it (redownloading to check for changes periodically). This system is called webcal.

Chandler Desktop works great for subscribing to a number of public webcal URLs and overlaying them all on one canvas. This is a great way to keep track of lots of calendars.

If you store any events/tasks on Chandler Hub, then you can login to get a URL that you can enter into the right spot in Outlook, Apple iCal, Google Calendar, Lightning/Sunbird, Evolution, Zimbra, and many other apps to synchronize that Hub calendar with your app. This is always a read-only/1-way procedure.

Using webcal, in Outlook 2007, you can overlay say personal or family Hub calendars on top of your Exchange/Outlook calendars you use at work. (Look for “Internet Calendar” features in Outlook’s help.) If you make a change on the Hub or Chandler Desktop and then synchronize, you’ll see that change in your work Outlook’s display.

You can also subscribe to Hub calendars in Outlook 2003, but only view the calendars side-by-side. Other apps like iCal, Lightning/Sunbird, Google Calendar, and Evolution all support overlaying the Hub calendar with other calendars.

Many applications, Outlook included, can also publish a webcal calendar to a web server. You can use Chandler Hub as a destination server for most of these webcal-publishing apps. This works, but please note this does not provide a web UI for that calendar, and it’s again a 1-way publication. The original application will very likely not detect any changes made to this webcal file on the server.

Webcal, 2-way sync

Chandler Desktop can also do 2-way synchronization via webcal. Most applications treat a webcal file as read-only or write-only, but Chandler Desktop will check for changes in a webcal file it is monitoring and integrate those changes. If used with another application that also checks for changes, you get 2-way synchronization. We know Lightning/Sunbird does this (though you might chose to use CalDAV to synchronize instead).

CalDAV, 2-way sync

CalDAV is an emerging standard protocol for open calendar exchange. It’s not a protocol that’s used directly between two clients (like ICS files are), but rather defines a calendar server to which multiple clients can subscribe and synchronize. Chandler Hub also provides a read-write web UI to any calendar you store or use in your account.

The Open Source Applications Foundation via the Chandler Project was an early supporter of CalDAV. Together, our Chandler Desktop application and Chandler Server server product are some of the oldest and most mature implementations of the CalDAV standard and we plan to continue that support.

Chandler Hub is, as far as we know, essentially the leading free CalDAV service offered to the public. Given a fully-cooperating CalDAV client (Lightning/Sunbird, iCal 3.x, and Evolution all cooperate to various degrees), you can use these other clients regularly or occassionally and even use Chandler Desktop for advanced work (like sharing a single item between multiple calendars).

Chandler Desktop can subscribe to and publish a collection (calendar+events) to any CalDAV server (Apple Calendar Server, RSCDS, Bedework), or actually any WebDAV server (Apache mod_dav, .Mac, etc). Both of these mechanisms support bidirectional (read-write) synchronization, so multiple applications or people can all create, edit, and delete events and tasks any time they want, using the application of their choice.

iCal 3 (in Apple 10.5 “Leopard”) is a great new CalDAV-using PIM client. You can use it to make changes to your Hub collection, and still be able to use the Hub web UI to make changes from anywhere. Note that iCal 3 supports read-write calendars only to calendars in your account. Chandler Desktop and Chandler Hub let you subscribe to shared collections owned by other users with full read-write access.

Email integration

Chandler Desktop is not a complete email client; it is rather intended to complement your existing email client. The mechanism we use is to create dedicated Chandler folders on your IMAP server. Using your regular email client (Outlook, Thunderbird, Mail.app, Evolution, etc), you just drag an email from your inbox into a Chandler folder, where the message will be parsed for event, task, and other information.

You should also be able to send email update of items from Chandler Desktop using just about any outgoing mail server available. Events emailed this way appear as ICS attachments. We’ve tested Exchange, Postfix, Gmail, Yahoo mail, and Hotmail/MSN among others.

Chandler has gotten a lot of attention because of its association with David Allen’s Getting Things Done methodology.

That was part of our “Be all things to all people.” past.

2 years ago, we started the long process of paring down our goals and zeroing in on a single focus so that we could deliver a 1.0 product. GTD is not our single focus. Supporting knowledge work is. (Blog post coming soon.)

This doesn’t mean Chandler is somehow the anti-thesis of GTD. Given the overlap in problem space, I imagine that as we pursue our single-minded goal, Chandler will continue to improve for GTD practitioners as well.

We also welcome and will actively help volunteers who want to write a GTD plug-in. Come find us on the Chandler Development List!

However, many of our users already find Chandler to be useful for practicing GTD today. They define collections around the GTD projects list and @contexts, use the Dashboard collection as a centralized Collection Box and assign ticklers and calendar dates.

For these users, Chandler is still a better tool for GTD than say Palm or Paper, the original David Allen GTD tools.

Some of the features our GTD users have put to good use include: 

• The ability to organize items into multiple collections means you can have next actions live in multiple contexts and you can organize next actions by project *and* by context.
• Chandler’s integrated calendar and list views mean tasks you’ve put on your calendar can still show up on your next action lists as scheduled tasks or events.
• Sharing! Not something covered in GTD, but Chandler helps you implement GTD as a household or a workgroup.

Still, Chandler shouldn’t be construed as an implementation of the GTD methodology or any other methodology for that matter. We don’t believe software can teach systems the way David Allen can. 

Chandler also can’t teach users how to turn Goals into Next Actions or even to appreciate that there is a difference between the two. These are things people need to come to terms with on their own, in their own way.

As Rick Rawson explained on the Users List:

“I am finding that Chandler does not organize my life. It only helps ME organize my life. And that takes time and work. There are any number of different strategies within Chandler that can be used to be sure I don’t forget all those “things” to “research” and “think about.” None of those methods is magic; all require MY brain and my time. I have to take the time to figure out what works for ME in MY context and with MY personality and deficiencies. What I like about Chandler is that it provides me with some tools so that I can devise my own solutions.”

This is consistent with user feedback we’ve received that the GTD label actually scares people away. Even though there isn’t any GTD terminology in Chandler, people who knew about Chandler’s past association with GTD assumed they needed to subscribe to a particular way of doing things in order to succeed with the product.

This intentionally provocative statement could be construed as self-defeating, given our goal to help people be more productive. Presumably, part of being more productive is actually meeting your deadlines.

But “deadlines” are one of the big downers of task managers because they are overused and misused, applied to everything as if everything could be managed in terms of time.

You enter a task. You studiously assign timeframes: deadlines, start dates and alarms. The dates are often meaningless, but you fear that if you don’t, you’ll completely forget about it. Then, when the dates roll around, you end up with a big pile of overdue tasks. You clearly cannot deal with all of them now and you still don’t have a good way to ensure that you won’t forget about them. So instead, you constantly carry around the baggage of a long list of overdue tasks. Oftentimes, overdue tasks turn into moot tasks. Stuff you never got to and then it was too late and well, looks like it wasn’t really that important. (This is perhaps the most common reason people give for why they just don’t need a task manager.) But when you first thought of the task, it would have been like pulling teeth to admit that, so you dutifully entered it and set a due date anyway.

Trying to guestimate when you’ll actually do things and how long it will take you and by when you’ll have them done, aka time management is an exercise in futility…as far as “knowledge work” is concerned.

It’s absolutely essentialy for things like dentist appointments and paying your rent. But the benefits are less clear for tasks that start with verbs like Brainstorm…Look into…Evaluate…Think about…Follow up on…

This is not to say that there aren’t hard deadlines for the end-goal of all your Brainstorming and Looking into.

And a “responsible” person would leave themselves plenty of time to brainstorm before the product of that brainstorm needs to be handed in as a concrete deliverable. But the reality is, trying to quantify how much brainstorming (3 hours) you need to do and when you’re going to do it (2 days before the write up is due) is an exercise in futility.

Which is why many people don’t bother trying to record and track these random one-off thoughts in any kind of an organized system (unless you count email as organized.) Instead, they’re strewn across desks on little bits of paper, desktops in text files, emails, sticky notes, napkins and last but not least, your brain.

The problem for a growing number of people is, there is too much of this “stuff” to keep track of in your head and still no good way to keep track of them outside of your head. Traditional task management solutions designed for “concrete” tasks are too structured. Not very structured solutions like paper notebooks and text files are well, not very structured.

Chandler attempts to walk the line by offering just enough structure to get you “organized enough” without imposing so much structure that managing the solution becomes a task in and of itself.

“Avoiding the futility of time management” is precisely why the LATER section in Chandler does not offer more granularity. It was intentionally missing options for designating more granular sub-sections like Tomorrow, Next Week, in 2 Weeks. You can always assign alarms to items so that they automatically pop back into NOW on a particular date. But unless you have a very specific date in mind, the idea is that really, you have no idea when you’re going around to thinking about this again.

On the other hand, we have seen that users quickly accumulated so much stuff in LATER that they begin to see the LATER section as a black hole. If you want to forget about something, put it in LATER.

In Part 2 of this post, I will get go into more detail about how we might reconcile these two opposing forces in the user experience.

A month ago, I wrote about next steps for the Chandler project after our reformulation as a smaller, more agile team. Since then we’ve made the plan concrete — here is a summary of the goals and a few pointers to specific work queues.

Mimi described the goals nicely in a post to the chandler-dev list…

1. Get Chandler in front of more users, aka: Make it more viral.

Product changes:

• Item sharing: a new workflow to use the web to collaborate on just one item. We’ll “widgetize” this functionality, making it available in other contexts like iGoogle or on an iPhone.
• Improve web UI “ticket views” so subscribers can more easily subscribe to collections in applications they already use
• Improve existing use cases for iCal and Lightning users (sharing with Chandler users, using Chandler Hub)

Marketing and Evangelism:

• Improve our pitch, improve our web presence
• Better demos, user testimonials
• Reach out and talk to people about Chandler in other spaces

2. Make Chandler more appealing to new users, aka: Reduce barriers to getting started.

Reduce the number of new concepts users need to understand in order to get started:

• Pare down UI, de-emphasizing email UI
• De-emphasize notion of “Item” and replace with “Note”
• Remove explicit “Task” and introduce “Star”

Improve the web UI experience for people not using Chandler desktop (iCal/Lightning or Hub only users):

• smooth out sharing workflows
• auto-triaging CalDAV events
• make Notes field in detail view more usable

3. Make it easier for new users to ramp up to using Chandler every day.

Add two additional “widgets” with features that allow people to use Chandler in other contexts:

• Notifications: Users can send themselves or others notifications about changes to shared collections. This also counts towards the first goal, as it allows current users to share some Chandler functionality with other people. Notifications will be available first as an iGoogle widget (and potentially other similar contexts), and eventually also as email, SMS, or IM messages.
• Quick Entry: this widget will allow users to enter items into Chandler Hub from other contexts: iGoogle, iPhone, OSX and Vista widgets. Eventually we’d like to allow similar functionality through forwarding email to a particular address.

Work Queues and Releases

The work described above has been broken down into tasks and bugs and is prioritized into two work queues, one for the desktop and one for all of the web related work. Grant is marching down the desktop queue while everyone else tackles the web queue. We meet daily to cover progress, adjusting the work queues if priorities change. (Mockups and specs for the new widgets and web UI changes are also linked from the web queue.)

• Web Work Queue
• Desktop Work Queue

The plan is to do a desktop release and a server release once a month. Usually these won’t need to be coordinated — though in this next round we have a security bug that involves both.

Phillip’s work on the desktop rearchitecture is the exception. He’s posting about his work over on the PEAK list. We may move Chandler desktop over to this architecture after the 1.0 — we’re waiting to see how this plays out to make the call on that.

Milestones

We plan on hitting a few major milestones by early summer — these are the big goals we are shooting for:

• Web Widgets: Quick Entry, Notifications, Item Sharing — we’d like to have these deployed in a few contexts.
• Desktop 1.0: We’re pretty close to releasing a 1.0 desktop. Prior to launching this we want to make sure some web UI improvements go up on the Hub, and make some changes to the website.
• Server 1.0: With some security fixes, authentication work, and a few other items (e.g. the ability to disable account signups), we should be able to release a 1.0 for people who want to run their own server.

We don’t need to coordinate all of these milestones — we may hit some more quickly than others.

Changes to the Plan

We were thinking we’d put minimal investment into the existing web ui, figuring that we’d do a better job on the web use cases we want to hit with the web widgets. Once started thinking through both the web and desktop use cases, we realized we really do need to make some investment in the existing web ui. We’ve added web ui bugs to the web queue.

We decided to put off working on a Thunderbird plugin, for two reasons: (1) after doing a bit of research it was starting to look like a more sizable investment than we initially thought and (2) we worried about having too many projects.

Mimi and I were checking out Things the other day, which appears to be a thoughtfully designed task manager for OSX (similar space to Chandler). One of the developers had a recent post that I thought captured something important about good design:

Of course both Jobs and Schiller know that there are times when you listen to users and times you better not. And the interesting thing is, even when developers don’t listen, they might do it for the best of their users. But how is that?

By all means developers need to follow their vision without asking anybody. They need to think out of the box and innovate. How else could they surprise and ultimately delight?

One of the main responsibilities of a developer is to keep guard over the gestalt of a program. It is all too easy to let your application burst into a universe of hardly connected little features. We have all seen it happen. But it is equally easy to ignore your customers’ needs and to embark on a journey where nobody is following you.

It is not about listening or not, it is about what to do with all the things you have learned from listening. And that is integration. The best feature is worth nothing when not integrated properly. When we read a feature request, we don’t think about doing it or not doing it. After all, if software development is not about satisfying users, then what is it? We are thinking about how we could nicely integrate it with the rest of the application without diluting its identity.

Of course, as an open project, we’ve sort of given up the element of surprise.

One of the project goals has been to experiment with involving users, developers, and other interested parties throughout the design process, making design decisions in conversations on public mailing lists. During that process it can be tempting to use “what the user has asked for” as an objective criteria for making decisions. The quote above does a nice job articulating that doing so can be dangerous for the integrity of a design, without writing off the importance of paying attention to what users are saying.

Mimi did a writeup on OSAF’s open design process last summer in preparation for a presentation she did with Ted Leung at OSCON. She goes into some detail describing how we manage conversations and make design decisions in a way that tries to preserve both the “gestalt” of the design as well as the continued interest of developers who might be pursing individual passions and concerns. Getting this process right continues to be one of the more interesting challenges of the project.

It’s time for another update on our recent activities.

On the desktop side we have been working away at the desktop work queue and plan to release 0.7.5 next week. We will send out a detailed list of new features and bugs fixed with the release notes. Highlights will include some printing support, new localizations and some modifications to simplify the UI. Philip continues to make progress on the rearchitecture work, specifically the repository replacement.

On the Cosmo side, we have been testing out Randy’s fixes for the security bugs with an IRC QA session yesterday that uncovered a couple of issues. We will be releasing Cosmo 0.13 with these security fixes in the next week or so. There was some work done on the desktop side to support the security fixes but both releases can be independent.

Travis is continuing to finish up the port to Dojo 1.0 and has been working on a test spec. We are targeting next week to start testing this branch. We are making good progress on the quick entry widget. Most of the polish tweaks have been done and Jeffrey has been testing and fixing some IE bugs. Brian has been working on the notification widget and is getting close to having an architecture proposal ready. You can track the web progress on the wiki.

Jared continues to work on quite a few administration, build and support tasks. We have also held some design discussions to brainstorm the email/sms notification features.

We have also started working on a bunch of different marketing initiatives. We are starting with refining the pitch, our landing page and organizing some of the planning information on the wiki so it lines up with our plan or record. Mimi has done a series of recent blog posts that tackles the user problem we are trying to solve with Chandler.

In refining our product message, one question we want to answer well is: What kind of information does Chandler help you manage?

The short answer: Chandler helps you manage all the notes you write to yourself.

The long answer: Chandler helps you Collect, Share and Follow-through all the “stuff” you:

• Email yourself,
• Scribble on napkins,
• Stick in random text files and electronic sticky notes, and
• Put down in paper notebooks.

“Stuff” includes anything from ideas you need to develop and questions you need to follow-up on to things you forgot to do, things you can’t forget to do, meetings, appointments and the odd flash of revelatory inspiration.

In short, your day-to-day life is overflowing with ideas, thoughts and questions you need to Develop, Follow-up on and Get back to and you don’t have a good way to manage it all.

Traditional task managers are too rigid. But you need something more structured than your paper notebook.

Also, almost everything you do involves other people and you find that just managing the communications about what you’re doing as a group is a second job in an of itself.

Some more specific examples:

1. You come out of a meeting with a dozen new “things” to “research” and “think about”. You’ve scribbled them into your notebook. But now you don’t have a good way to develop and track those things as you make progress on them: Follow up with X about Y. Where do you keep track of what X tells you about Y while you figure out what it is you need to do with Y?

2. Pulling a meeting agenda together always generates an algae bloom of email. Once everyone’s input has been gathered and re-gathered over email, you’re the one that has to do the work of collating everybody’s responses.

3. You keep making the same lists all the time and every time, you forget something that you would’ve remembered if you had seen the last list you’d made. But you don’t have a good way to manage all these lists! e.g. Travel packing lists, grocery list, present ideas, thank you notes.

4. When working on drafts with others: Write-ups, status reports, proposals etc., you’re always torn between just sending out what you’ve got so far or wait a little longer. In the meantime, everyone else is unaware of the work you’ve done and working blind.

5. You tried sharing a calendar with others, but you’re the only one who ever looks at it! Whenever you add or change something, you end up having to send email out to get people’s attention anyway. The same thing happens when you try to collaborate on tasks or brainstorming ideas with a wiki. In the end, you always go back to using email.

Why is “What kind of information does Chandler help you manage?” a tricky question to answer?

In a previous post, I talked about using the phrase “personal information manager” to describe Chandler. Taken literally, yes, Chandler is a variant of personal information manager. But, using the PIM label confuses people in the software industry because it’s associated with specific software products that Chandler does not resemble.

The general population doesn’t have such specific product associations, so they might take PIM at face-value. However, the phrase “personal information” is now understood in the vernacular as shorthand for “information about me” (e.g. name, address, social security number, bank accounts, health records, etc.) This is especially true amongst computer-literate, white-collar workers, aka, our target audience.

In case there was any confusion, Chandler is not optimized to manage “information about me”.

Going forward, our task is to re-craft our product message with this in mind, both so we attract the users Chandler was designed for and we set expectations for new users trying out Chandler for the first time.

A recent blog post by Eugene Kim effectively articulates what Chandler is to him and how it fits into his day-to-day work life. It’s really great to hear this in the words of a user.

This kind of blog post from active users really helps us craft our product message and prioritize the work we need to do to get to a 1.0.