Managing the Field Notes (Part I)

When I switched from using Twitter for my short form publishing to my own bespoke system, Field Notes, there were three things I needed to take care of to make the system usable, easy to maintain and scalable. These I identified as follows:

1. It should be quick and easy to publish Field Notes;
2. When I post an item with link(s), the system should detect duplicates (e.g: I’ve posted the same link in the past and forgotten about it);
3. The system should be able to test the integrity of existing links on demand, so that I can avoid link rot.

In this series of articles, I’m going to describe some of the things I’ve done to make the Field Notes work. I’ll include relevant code samples where possible.

Out with the New…

One of the significant benefits of the old Twitter-based system was the ease with which I could get short notes on to the Perpetual βeta website. Write a Tweet and it appeared on the site within 15 minutes (I updated my local Tweet cache at 15 minute intervals so as to not get hit by Twitter’s rate limiting).

The other advantage of Twitter was its ubiquity in the Apple eco-system. On OS X a click or two is often all that’s needed to share content via Twitter. A quick swipe into the Notification Centre provides a “Click to Tweet” button. It’s the same on iOS. It followed then that, with publishing being so easy, I actually used it. My Twitter usage went from rare to frequent once Apple made it integral to OS X.

The problem was that I didn’t want to use Twitter for Perpetual βeta content. I didn’t want a 140-character restriction on my notes. I didn’t want to have to follow Twitter’s style guidelines. I didn’t want to have part of my site break any time Twitter went down (it still happens, although it’s much less frequent now) and so on.

But I really, really did like the ease with which I could Tweet. So how to replace it, yet retain as much of the ease as possible?

…and in with the Old

I went with one of the oldest of the Internet technologies: good old email. I chose email because it’s always right there on the same menu of “Share” options I previously used to fire off Tweets from. On my MacBook I use a third-party utility that allows me to compose and send an email right from the menu bar. So it’s as quick and simple for me to post Field Notes as it is to post a Tweet.

The email goes to an IMAP account I set up just for Field Notes. A PHP script running as a daemon on my computer checks that email account every fifteen minutes. The daemon retrieves the Field Notes, validates them, creates the corresponding record in my local CMS then performs an SVN commit. I have configured my SVN to automatically push changes up to the live website on each commit, so the process is entirely automated. I can also manually trigger the process if there’s something I want to get on to the site immediately.

My system is using the standard PHP functions for manipulating the IMAP mailbox so I won’t describe how I’m handling the email (if you’re interested, Garrett St. John’s articles on the subject are a great place to start).

So, that addresses one of my targets: it’s quick and easy for me to publish Field Notes. In Part II, I’ll show you how my process checks for duplication on posting. In Part III we’ll look at the code I’ve written to check the links.