Markdown files

Jay

In the latest beta I added a feature to export to Markdown. This has been requested by people who like Markdown and want portability to other editors so it’s a useful feature on its own. But it’s also the first step of a bigger plan to support editing local Markdown files directly, rather than only our proprietary json format.

So for anyone who likes Markdown could you do me a favor and test the export and see how it looks in other apps, and let me know if anything looks wrong? It’s still experimental so you can enable it by turning on “Test beta features” in the settings and then it will be an option in Export Data.

And one issue I’m not sure how to solve is how to save attributes. Legend has a lot of features that aren’t supported directly in Markdown so we need some way of expressing them. Besides metadata like priority, starred, etc… it’s also important that we save a unique ID on each item for the purpose of linking/mirroring and saving version history.

I found that there’s a common style of attributes of ending lines with {…} - see example 1 example 2 example 3 example 4. But still there’s some apps that don’t handle it at all and in some cases render it strangely. In your opinion is that the best way to go? Or is there a better way we can save attributes?

Additionally, Markdown seems to treat indentation after a header as a code block. So it removes indentation under header parents, though it still keeps the indentation under non-header parents. Does that seem like a reasonable solution?

Jeff

Jay thanks for working on Markdown export! I’ve been meaning to reply to this for a while, but I knew it would be a long post, so I haven’t got around to it until now. Tl;dr - if you don’t have time, I recommend at least looking at Dave Winer’s post about using OPML rather than Markdown as an open storage format for outliners like Legend: https://discuss.logseq.com/t/support-opml-as-file-format/939/5

It would be really useful for me if I could take something I wrote in Legend, and copy it to any of the other destinations I use that support Markdown: Github, Reddit, Thunderbird e-mail, this forum, other notes apps I use, etc., etc. That gives me text formatting (bold/italic/etc.), headings, lists, URL links, code blocks, and more.

At the moment though, it doesn’t work for me. First are the item IDs, which I’ll get to in a moment - let’s assume there’s a switch to turn them off. Second is that I have my default item type set to “none”. So in the exported Markdown, it’s just tab-indented lines without hyphens or asterisks, which isn’t compatible with Markdown or HTML. You said “it still keeps the indentation under non-header parents” but that’s dependent on the implementation, which varies widely. Outline text gets all the indentation removed, or it may all go on one line, or turn into a code block, depending on the Markdown editor, its settings, and whether there are blank lines or headings around it. Generally, as with HTML, whitespace including tabs and newlines should be collapsed into a single space, but with Markdown editors, I would suggest treating the behaviour of tab-indented lines as “undefined”.

Maybe there could be an option to automatically add hypens or asterisks, so that outline text would be formatted as unordered lists in Markdown. On the other hand, you don’t always want that, for various reasons - you might not be writing with indentations, and the list format with bullet points on every paragraph can look odd in an e-mail or forum post. Also, many Markdown editors don’t support things like headings within list items, etc.

It’s unfortunate that HTML, and Markdown even more so, struggle with simple indentation, whereas outliners are all about indenting. HTML needs non-breaking spaces, which are awkward for everyday use, or custom CSS, which isn’t available in a lot of environments like forum comments or e-mail. Markdown… well it basically doesn’t work at all, if you don’t fit into its mold of either lists or blockquotes, which may render in ways that are quite ugly if you don’t have control over the CSS.

It would be great if I could simply select some text in Legend and do “copy as Markdown”, rather than going through the Export Data thing and selecting a whole document. If that worked, you could maybe have it so that if you copy text as Markdown in the Outline view, it would be formatted as unordered lists, while copying it from List view would make it flat, without bullets, with blank lines between the paragraphs.

One other thing I noticed is that headings with e.g. ## only come out with one #, even though in plain text mode they have two. There would probably be a bunch of small issues like this, in order to get a useful Markdown export feature working well.

It’s harder for me to see how Legend storing its whole database in read/write Markdown files would benefit me though. What advantages do you see for that? If it’s necessary to put in the IDs for every item, then you wouldn’t be able to edit the files in any other Markdown or text editor, since you wouldn’t know how to generate the IDs manually. When I was first looking at Legend, I was asking you, “how can I generate a text file that I can import to create a document in Legend, with all of its attributes like creation dates, checkboxes, and so on?”. You said that the JSON format, although it contained all the data, wasn’t really ideal for interchange with other editors. I could see that I would have trouble generating the IDs, for one thing.

I’m definitely interested in not having my data locked away in a proprietary format. But I wonder if Markdown is really the right tool for the job? Once you start adding all this metadata, you kind of lose the main advantage of Markdown, as a sort of pre-processor for HTML output that you can easily write in any plain-text editor and have the source look quite readable, even without rendering it. At that point, I’d think that something like OPML would be more suitable. Plus it’s already supported, to some extent, by nearly all the other outliners. I’d like to see the effort put into making that more robust, and coordinating better interchange possibilities for common attributes that way.

I know that some other projects, like Obsidian and logseq, do store their text in Markdown. I don’t know either of them very well, but it looks like both generate IDs for internal links, but only when necessary - they don’t put IDs on every line, so it’s still relatively easy to read the source. I think (don’t quote me) you can use any arbitrary label, so you could write them yourself, manually. Obsidian puts them after a ^ symbol, while logseq uses some format derived from Emacs org-mode files. I don’t know why they don’t use standard HTML anchors, which would be compatible with all other Markdown editors and web browsers.

For Obsidian, Markdown files are more understandable, because it’s not an outliner, it’s more like a Wiki. It has paragraphs of text, and links between them, but otherwise doesn’t seem to have a lot of different item types, attributes, or other metadata that needs to be stored in the plain text files. It’s more difficult to understand the choice of Markdown (it also supports org-mode text files) for logseq, which is more of a full-fledged outliner. Dave Winer (the father of outliners and OPML) chimed in on the subject here: https://discuss.logseq.com/t/support-opml-as-file-format/939/5 - and it seemed to be very well-received.

I could keep going on about all this a lot longer, if this post isn’t long enough already, just let me know…