Markdown Sustainability

In Responsible Open Source Code Parenting, Jeff Atwood complains that John Gruber isn’t maintaining Markdown properly, and that it’s irresponsible. While I will agree with the former, I disagree about the responsibility, just like Daniel Jalkut in his Open Source Obligations post.

As the developer behind PHP Markdown, I’ve felt for years the lack of leadership from John about Markdown. I’ve always wanted to keep my own PHP Markdown in perfect sync with John’s Markdown.pl, even up to the version number. Lack of releases from John meant that I was postponing the new features.

At some point, I decided to implement features which were discussed on the mailing list in my own fork: PHP Markdown Extra. Most of the features in the first version of Extra got approval from John on the mailing list. The idea was to keep Extra an incubator for new features to see how well they worked before they could be added to the main branch. It never really worked that way though as the main branch stagnated.

Daniel says in his rebuttal that someone just has to fork the tool if he isn’t happy. After all, this is how things work in open source. The problem with forking Markdown (the language) is that each new variant diminish Markdown’s value as a standard. If you can’t convince most other implementors to implement your extensions to Markdown, it’ll just become an obscure feature of your implementation.

There must be leadership somewhere if we are to have a standardized language. I tried once to write a precise spec for Markdown Extra to help implementors adhere to it, in a way taking the leadership and recreate a standard around that Markdown Extra thing. But after a few weeks I realized I couldn’t afford the time. (The most interesting achievement of this time is undoubtedly Babelmark.)

I have to admit that, for me, maintaining PHP Markdown & Extra, mostly fixing bugs for others, has returned little. While I’m still fixing bugs, unlike John, I almost never receive money in exchange, nor anything else. Basically, I’m working for free when I work on PHP Markdown. I can certainly understand why John Gruber lacks interests in maintaining Markdown: he is probably in the same situation.

On his blog, Jeff makes three specific complains:

1. URLs are never hyperlinked without placing them in some kind of explicit markup.
2. The underscore [] can be used to delimit bold and italic, but also works for intra-word emphasis. While a typical use like “_italic” is clear, there are disturbing and unexpected side effects in phrases such as “some_file_name” and “file_one and file_two”.
3. It is paragraph and not line oriented. Returns are not automatically converted to linebreaks. Instead, paragraphs are detected as one or more consecutive lines of text, separated by one or more blank lines.

Number 1 is by design: you don’t necessarily want every URL to be links. For instance I could write about hypothetical URLs in an article about properly writing URLs and it’d be pretty silly to make them links.

Number 2 is half by design. To quote the relevant part of the spec:

Emphasis can be used in the middle of a word:

un*frigging*believable

But if you surround an * or _ with spaces, it’ll be treated as a literal asterisk or underscore.

Whether _ should be allowed to mean italic in the middle of a word is unclear. Markdown.pl takes the simplest route and accepts both. Note that emphasis in the middle of a word can be quite useful to those writing in languages such as Japanese, where there is no space separating words.

PHP Markdown follows the lead of Markdown.pl to keep compatibility, but Extra only allows it with asterisks.

Number 3 is by design. It’s quite necessary to add line breaks which are not paragraph breaks to make the text pretty. Soft-wrapped paragraphs can’t keep things nice when blockquotes and list items are involved. This rule is necessary to keep the philosophy of Markdown, the very part Jeff quoted and emphases in his article:

A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions.

Without soft line breaks in the text document, this is impossible to achieve in the presence of anything else than regular paragraphs and headers.

Implementing 1 and 3 is something I seriously thought about, and if I do it it’ll probably go the way of a special “user comment mode” (not sure about that name). But I’ll need time for that, and I’m busy with other things. It’s not like anyone is giving me a salary for doing these things.

As for John no longer maintaining Markdown, both the tool or the language, I don’t really complain. I’m really glad he published Markdown instead of keeping it for himself. I believe he initially thought it’d bring him a small revenue stream (given the initial license which was GPL+commercial), but that probably never concretized, nor it did for me. I think maintaining Markdown as a standard, coherent language is an unsustainable project. That’s really unfortunate, because everyone uses it.