Writing in Public and Syncing with GitHub

published by Eric Mill on
unofficial motto of the NPR News Applications team, by Alyson Hurt

For a while now, I've admired my friend Ben Balter for writing out in the open. His blog is built on GitHub Pages, which means that for every piece on his blog, you can see each draft and revision along the way.

As importantly, Ben takes advantage of GitHub to truly embrace collaboration — whether that's detailed debate on a complex topic as he works through issues, or just graciously fielding my constant typo fixes. The end product is better for it, and everyone feels engaged and invested.

It's such a healthy, beautiful approach to public writing. Why doesn't every newspaper in the world work this way?

Making this work for my blog

And why not my blog? Well, Ben has the advantage of using GitHub Pages, which weaves his blog directly into GitHub, one of the greatest systems for collaboration that mankind has yet crafted. And GitHub Pages, while incredibly powerful, isn't for everyone. For me, I like having a database that can hold private content, and a comment system that belongs to me and not to Disqus. My blog is built on Sinatra and MongoDB and hosted on Amazon, which costs more time and money than GitHub Pages but affords me more control.

Fortunately, GitHub is much more than a nice website and host -- it's also home to an extraordinarily rich and well-documented API, through which you can order GitHub to do just about anything you need.

Using it, I've neatly integrated my published blog posts with a new GitHub repository, konklone/writing. There's now an "edit this post on GitHub" link on the sidebar, to make the connection obvious. You can suggest an edit to this post itself with the greatest of ease, and when I accept your edit, the post will automatically update.

I don't have to change my workflow or give up my database, but I still get a public revision history and the ability to accept pull requests that automatically update my blog as soon as I press the Merge button.

If you're curious about the technical mechanics, I created two pull requests that explain the gritty details:

  • Syncing TO GitHub: Any published post with an assigned URL to a file on GitHub will, upon save, use the GitHub Repo Contents API to update that file with the post's contents. A GitHub URL is assigned to a post upon first publish, which then automatically creates a file in konklone/writing.
  • Syncing FROM GitHub: The writing repo has a webhook enabled that sends a POST to a special endpoint in my blog upon any content changes. My blog accepts those changes from GitHub, while using a log of previously synced commits to avoid infinite sync loops.

Though my code is customized for my blog, you could take this approach with just about anything -- it's just APIs over HTTP, the lingua franca of the Internet, along with a couple extra fields on posts to keep everything straight. It's not that bad.

News In Public

I'd like this blog to be known for high quality, accurate pieces that people share and return to. I also just want to write good stuff that reads clearly. It's a lot easier to make all that happen with an obvious path for readers to contribute, and an annotated history of changes. And in my case, my early drafts are written freely and in private -- I'm only automatically syncing changes to GitHub once I've hit the Publish button.

I don't see any reason why a major newspaper can't do the same thing. The New York Times cares deeply about precision, enough to run corrections like this:

Yet when the Times makes less funny mistakes, like opening a woman rocket scientist's obituary with a description of her cooking, you won't see a correction notice at the bottom of the current article. You'll have to hope someone else caught it and rescued it from Google's cache or something.

Don't even try to keep track of the constant headline tweaking. In one recent example, the Times was being criticized for essentially running the headline the Obama administration wanted them to. The web's complex mechanics clearly demonstrates the Times responding in fits and starts:

I'm picking on the New York Times here, as my favorite newspaper and the country's paper of record, but all of this applies to any major newspaper. Can you think of a single one with a transparent changelog?

Authority Through Obscurity

When I've suggested this approach to others, I've been told that adding collaboration and transparency to their published work would reduce its authority. I believe this is exactly backwards, and not much different than believing that you're more secure by hiding your security choices. Truth withstands public scrutiny, and is stronger for the consensus. It's not like it has to be Wikipedia: on GitHub, you're in total control over what changes you accept.

And it doesn't have to be through GitHub! Any system that lets readers suggest concrete fixes and improvements, lets writers and editors easily accept them into the published version, and offers the public a record of a story's evolution would do just fine.

Working in public this way is a set of principles shaped by the collaborative world of open source code. Practiced often, it becomes like breathing.

The world of online journalism in 2014 is increasingly technology-driven, and staffed with excited people who were raised by the Internet -- it can and should embody these principles just as vigorously.

  1. Greg Linch

    Hey Eric,

    I'd love to chat some time. Using GitHub for news has been one of my obsessions:



    Also, have you seen Ben et al's WordPress plugin?


    Best, Greg

  2. Travis Briggs

    Congratulations Eric, for getting this done. I know it's something that's been important to you for a while.

    I think that less than giving up authority, having other's edit your work has a potential to lose your writing voice, which is something that many writers work very hard to develop in the first place