Robert Chien's Blog

It's becoming clear and present that a re-architecture of Email is overdue.

Last week, a few aspiring folks got together in New York City to discuss this very topic. In attendance were Tom Evslin, Fred Wilson, Matt Blumberg, Brad Feld, Jeff Pulver and Phil Hollows (click on the names to read thoughts and summaries of the event on their respective blogs).

It appears the goals are pretty well-defined over 3 years ago by the Internet Mail Consortium. There's also consensus on ingredients that will make next generation of messaging a successful one:

• Social graph enabled
• Give control to users, not spammers
• Accessible through an open API
• Must be secure and trustworthy
• One interface to manage your online communications

When these elements are put together into a solution, we'll have a reliable, productive and fun way to communicate again.

p.s. Does anyone in the San Francisco Bay Area want to meet up and talk about messaging 2.0? Drop me a note in the comments, Email, IM, Twitter or Facebook (now see why a single interface is attractive?)

A quick rundown:

• Marshall Kirkpatrick from readwriteweb.com wrote "it took Facebook to introduce people to RSS in a way that was really compelling." and "The social network of the future will be populated by the RSS feeds of the activities of your friends and your friends will be determined by email."

• "The Death of E-Mail" on Slate said "Those of us older than 25 can't imagine a life without e-mail. For the Facebook generation, it's hard to imagine a life of only e-mail..." and "Instant-messaging, mobile text-messaging, blogging, micro-blogging, and social-networking profiles all help compensate for e-mail's shortcomings." and "It's not hard to imagine a future communications command center where, on a single screen, you'll be able to choose between sending an e-mail, instant message, status note, or blog post—or sending all of them at once—and then have all those bits of text neatly and securely archived."
• Nick O'Neill wrote on "Email Becomes Center of Social Networks?" "I’m not quite sure how I feel about using my email for the center of my social network but maybe my feelings will change once it launches."
• "Inbox 2.0 isn’t coming, it is here." on Xobni blog said "We realized after building Xobni analytics and playing with email data for 6 months that the most interesting data in email revolved around relationships."
• Mathew Ingram wrote skeptically on Can getting social make email better? "for me, email is pretty close to broken."
• Don Dodge wrote "Email contacts - the natural social network" and said "Email is where [people] naturally communicate and collaborate. Social networks are another isolated island of information."
• Steven Hodson wrote "For me using email as my social network hub doesn’t depend on integration with outside environments or the need to be widgetized in some fashion or other. Instead my Inbox needs tools like Xobni and some serious attention to dealing with spam." and "We already have been using the original social network for a very long time - it just needs some fixing up and new tools to make it better."

• Google announced the new Google Apps Email Migration API for customers whose existing mail systems don't support IMAP.
• Sachin Balagopalan thinks it’s time to phase out email from the work place. "The profound difference [...] is that social networks enable you to participate in a virtual team or community and IMO that is conducive to the business environment - we all belong to a team at work and we interact with our team members. Email on the other hand was not designed with the community in mind rather it’s based on an “account” and you can send and receive emails from any account."

I shared my Messaging 2.0 idea with Han today and she seems to like it. I'm hoping to find some time during Thanksgiving holiday to refine and formalize the architecture.

After posting my thoughts on Twitter and Email and reading comment on it, I've been thinking a lot about how outdated the store-and-forward mechanism is for modern asynchronous communication, and how Email could be advanced to meet current and future needs.

From the very beginning, Email delivery has been modeled after postal mail: a sender creates some content and ships it away, where it gets relayed with every step closer to the destination, until it arrives at the designated mailbox. This technique is referred to as "store-and-forward," and SMTP was designed this way 25 years ago because the network back then was unreliable and nodes were not online all the time.

Today, these are no longer the cases. Recent trends such as blogging, podcasting and twittering take advantage of these changes and make use of a completely opposite paradigm—publish-and-subscribe, also known as pub/sub, in which writers publish content on a server, and readers subscribe to those content via feeds. Similar to radio, there are all kinds of broadcast signals in the air but as listeners you pick which station to tune into.

To a certain degree, an Email sender is the same as a blogger in that both publishes content. The difference is a blogger doesn't send what she wrote to others because they, as subscribes of her blog feed, will come to her website to pick it up. Could the same concept be applied to Email whereby a sender publishes an Email on a website which creates a permalink and updates the feed, then the named recipient pulls down the Email through subscription?

Although I came up with the idea independently, I wasn't the first. Upon research, I found out that a sender-based storage concept coined IM2000 was originally proposed by professor Daniel J. Bernstein (djb) about 10 years ago. Where IM2000 differs from mine is that it's not truly pub/sub because tiny notifications are sent to the recipient.

There appears to be many merits to a pub/sub sender-based storage Email concept. For example, it puts the burden of storage on the sender, not the recipient who may have a disk quota from her service provider. Should someone send you a 20MB attachment and you're already at 95% of your 100MB quota, you don't have to worry about the message bouncing because there's not enough space for it. But if you're the sender and your quota's at 95%, it's your responsibility to upgrade your account to one with bigger quota, and in the meantime you can still receive messages.

Pay for what you do, not what others do to you. Seems fair to me.

[UPDATE Nov 21, 2007] Note to self: there is a few issues with sender-based storage:

• how should searching be handled?
• shared folder won't work
• offline mode won't work
• what if sender-side data is missing?

Because it isn't designed to be Email 2.0.

On a recent debate titled "E-mail Faces Deletion" hosted by BusinessWeek, Robert Scoble suggests that Twitter could overtake Email as the leading business communications tool. I read it a few weeks ago but I wasn't on Twitter so I didn't feel qualified to comment. Since then, I've become more familiar with Twitter and found a few of his arguments flawed.

1. Knowledge retention. While policy varies from country to country, publicly-traded companies and even SMB who don't host their own Email nowadays typically keep Email on the server side and have retention policies (for compliance reasons) which determine how data of former employees is retained and transferred to replacements.
2. Spam problem. Twitter doesn't suffer from it because users decide who they wish to follow or unfollow. This method is similar to whitelisting and blacklisting and only works in Twitter because it is a walled communication platform and you don't give out your Twitter username as you would give out Email address (on the last page of your presentation, when you fill out online forms, to merchants and service providers, etc).
3. What happens in Twitter, stays in Twitter. You can depend on Twitter for as long as it is around. Possibly the best way to explain Twitter to non-technical people is that it is a news broadcasting system in which any member can be a broadcaster. This is very appealing to consumers but not so to corporations. For various reasons, good or bad, internal businesses communication most often flow in a controlled and structured manner rather than a broadcasting model.
4. Twitter lets you filter what others are saying. For example, when Google launched the Open Handset Alliance yesterday, also known as Android, you can do "track android" in Twitter and it'll automatically direct every Twitter message (called "tweet") containing that keyword to you. The upside is that you get to tap into a global community and track actions and thoughts on that topic in near real-time, but the downside is that the signal-to-noise ratio can be very low because everyone can be a broadcaster.

Furthermore, Twitter has a few design choices that make it unsuitable for business use:

1. Messages are limited to 160 characters.
2. No support for attachments.
3. Can't define scope of distribution.
4. No verification of status. Companies (especially large ones) may wish to cut its tie with terminated employees and it's not clear how Twitter can handle that.

That being said, is Email perfect as a business communication tool? Absolutely not. It's been around for 25 years and I'm confident it'll stick around for another 25 years, but if its weaknesses are not addressed and improvements are not made in time then I doubt it'll maintain its usefulness. Although it's not fair to compare Email with Twitter, there is a few things Email can learn from Twitter:

1. Needs stronger sender identification. When an Email claims it was sent by Aunt Betty, it must truly came from her and no one else. Twitter's solution is to require account registration and username/password. Systems such as SPF and DomainKeys go so far as to ensure domain-level authenticity, but we need something that goes farther to sender-level.
2. Needs an API. Twitter offers an API so that users and other developers can discover new ways to use Twitter. Email doesn't have an API, it has RFCs written by lots of people over many years to ensure interoperability, but its fundamentals are largely unchanged even though the rest of the world has progressed. I say it's time for an update, a rethink on modern and future requirements, similar to what ZFS did to filesystem. Excuse the overuse but we need an "Email 2.0".
3. Needs Permalink. A permalink is basically a fixed index to a web resource to which others link or respond. The vast majority of Email is either an inquiry for response or a response to another inquiry. If every Email message you write has a permalink, then it's a lot easier to track or search when others respond or add value to it.
4. Follow & track. Once all of the above are in place, these become trivial. In fact, all kinds of new possibilities open up.

Do you think an old dog can learn new tricks? It's only limited by our imagination and drive. Consider how Google uses Email for project management (it's a rather long story, just search for "project management" when the page loads).