Markdown Sustainability

In Responsible Open Source Code Parenting, Jeff Atwood complains that John Gruber isn’t maintaining Markdown properly, and that it’s irresponsible. While I will agree with the former, I disagree about the responsibility, just like Daniel Jalkut in his Open Source Obligations post.

As the developer behind PHP Markdown, I’ve felt for years the lack of leadership from John about Markdown. I’ve always wanted to keep my own PHP Markdown in perfect sync with John’s Markdown.pl, even up to the version number. Lack of releases from John meant that I was postponing the new features.

At some point, I decided to implement features which were discussed on the mailing list in my own fork: PHP Markdown Extra. Most of the features in the first version of Extra got approval from John on the mailing list. The idea was to keep Extra an incubator for new features to see how well they worked before they could be added to the main branch. It never really worked that way though as the main branch stagnated.

Daniel says in his rebuttal that someone just has to fork the tool if he isn’t happy. After all, this is how things work in open source. The problem with forking Markdown (the language) is that each new variant diminish Markdown’s value as a standard. If you can’t convince most other implementors to implement your extensions to Markdown, it’ll just become an obscure feature of your implementation.

There must be leadership somewhere if we are to have a standardized language. I tried once to write a precise spec for Markdown Extra to help implementors adhere to it, in a way taking the leadership and recreate a standard around that Markdown Extra thing. But after a few weeks I realized I couldn’t afford the time. (The most interesting achievement of this time is undoubtedly Babelmark.)

I have to admit that, for me, maintaining PHP Markdown & Extra, mostly fixing bugs for others, has returned little. While I’m still fixing bugs, unlike John, I almost never receive money in exchange, nor anything else. Basically, I’m working for free when I work on PHP Markdown. I can certainly understand why John Gruber lacks interests in maintaining Markdown: he is probably in the same situation.

On his blog, Jeff makes three specific complains:

  1. URLs are never hyperlinked without placing them in some kind of explicit markup.
  2. The underscore [] can be used to delimit bold and italic, but also works for intra-word emphasis. While a typical use like “_italic” is clear, there are disturbing and unexpected side effects in phrases such as “some_file_name” and “file_one and file_two”.
  3. It is paragraph and not line oriented. Returns are not automatically converted to linebreaks. Instead, paragraphs are detected as one or more consecutive lines of text, separated by one or more blank lines.

Number 1 is by design: you don’t necessarily want every URL to be links. For instance I could write about hypothetical URLs in an article about properly writing URLs and it’d be pretty silly to make them links.

Number 2 is half by design. To quote the relevant part of the spec:

Emphasis can be used in the middle of a word:

un*frigging*believable

But if you surround an * or _ with spaces, it’ll be treated as a literal asterisk or underscore.

Whether _ should be allowed to mean italic in the middle of a word is unclear. Markdown.pl takes the simplest route and accepts both. Note that emphasis in the middle of a word can be quite useful to those writing in languages such as Japanese, where there is no space separating words.

PHP Markdown follows the lead of Markdown.pl to keep compatibility, but Extra only allows it with asterisks.

Number 3 is by design. It’s quite necessary to add line breaks which are not paragraph breaks to make the text pretty. Soft-wrapped paragraphs can’t keep things nice when blockquotes and list items are involved. This rule is necessary to keep the philosophy of Markdown, the very part Jeff quoted and emphases in his article:

A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions.

Without soft line breaks in the text document, this is impossible to achieve in the presence of anything else than regular paragraphs and headers.

Implementing 1 and 3 is something I seriously thought about, and if I do it it’ll probably go the way of a special “user comment mode” (not sure about that name). But I’ll need time for that, and I’m busy with other things. It’s not like anyone is giving me a salary for doing these things.

As for John no longer maintaining Markdown, both the tool or the language, I don’t really complain. I’m really glad he published Markdown instead of keeping it for himself. I believe he initially thought it’d bring him a small revenue stream (given the initial license which was GPL+commercial), but that probably never concretized, nor it did for me. I think maintaining Markdown as a standard, coherent language is an unsustainable project. That’s really unfortunate, because everyone uses it.


Comments

Jeff Atwood

At some point (and I would argue that we have reached that point), the lack of any kind of leadership is actively harming the future of Markdown.

I don’t buy the too much effort argument. How long would it take Gruber to:

  • publish his current, already written “secret” bugfixes as an official 1.0.2

  • update his Markdown page to indicate this is the final release

  • bless another distribution as official and worthy of the “Markdown” name so the standard can be taken forward without him

That is — and I’m being generous here — at most 8 hours of effort.

I do not think asking someone to spend 8 hours over the last five years, on a project they themselves created, is such an onerous request.

Michel Fortin

Hi Jeff,

I agree about finally releasing 1.0.2, but, if I remember well, those betas had some problems with the HTML block parser. I’m pretty sure that’s why 1.0.2 was never released officially. This would have to be fixed first.

I can imagine a couple of reasons why he wouldn’t want to do the rest, but unless someone asks him and he replies we won’t know for sure. For instance, who should he choose to do the job in his place? Someone will probably have to offer himself first. I know I would surely be well-suited for this, but unfortunately right now I feel like I’ve put enough time into this and I want to spend my free time elsewhere.

If it was paid time that would be another matter entirely, and perhaps John Gruber would just get on the job himself, but I doubt there is any money to be found for that.

BCS

Regarding the comments on Issue 3: the point of needing soft paragraphs is completely dependent on what you do with word wrap. If you assume that word wrap is working when the ASCII text version is viewed (like in most text input boxes on web-pages), then making any and all carriage returns into paragraphs is reasonable because you can expect people to only hit enter at the end of a paragraph. Without word wrap OTOH… I think making the line/paragraph centric aspect configurable is a good idea.

Michel Fortin

BCS: What you want with word wrap definitely depends on the context. But I would draw the line elsewhere. Even with word wrap it makes sense to cut lines manually to make the text more readable in some cases. Compare this:

> This blockquote doesn't really need a `>` before
> each line, but it's more readable that way.
> Especially when it contains:
>
> *   Long list items spanning on more than one 
>     line 

to this:

> This blockquote doesn't really need a `>` before
each line, but it's more readable that way.
Especially when it contains:
>
> *   Long list items spanning on more than one 
line 

The later is what you see when you let the text box handle the word wrapping. It’s much less readable. Perhaps a special mode could allow simple line breaks but only in top-level paragraphs, where word wrap makes this irrelevant, so you could still prettify the rest.

Jacek Jędrzejewski

Hi Michel! It’s nice to see something new about Markdown on your page.

Why don’t you create a public bugtracker for MD Extra? I’m even lost in bugs reported by me (via email to you) - I don’t see them fixed in the newest release. I will have to dig in my e-mail archives to find test files and workarounds.

Alan Hogan

Michel, your Extra is doubtlessly the most popuplar and best variant. I truly appreciate the time and effort you have spent on it. I completely sympathize with the “unpaid” issue (I have some WP plugins and other projects I have no time for).

Thanks so much though.

simon

Would it be offensive to ask why you don’t get together with multimarkdown and create one markdown where the work can be shared?

Michel Fortin

Jacek: Sorry, it seems I forgot your email. Generally when I receive a bug report I immediately add it to my test suite so I can’t forget when preparing a new release. I’ll add it just now.

Simon: For that you need time and common agreement about how things should work. Time is hard to come by for me, and I know a few things I wouldn’t adopt from MultiMarkdown because I don’t think they belong in Markdown.

Willie Alberty

Michel: I have created a patch for the Markdown Extra parser that implements the line breaking behavior discussed in Number 3 based on the technique used in GitHub Flavored Markdown. I implemented it as a configurable option, similar to how the no-markup mode is handled, so I can turn it on and off depending on the context.

For my application, in cases where the readability of the fixed-width source text needs to be maintained—such as email templates—I leave it turned off. For things like contact logs, note fields, blog comments—anywhere the source text may never be consulted again—I turn it on. Works really well.

Do you accept patches for PHP Markdown?

Michel Fortin

Hi Willie. Sure I take patches. Just mail it to me and I’ll review it.


  • © 2003–2017 Michel Fortin.