See also text editors, LaTeX , citation management, scientific computation workflow, plain text blogging, academic writing workflow.
I would like to write up my research using markdown, because this means I can produce a web page or a journal article, without wading through the varied depressing and markup sludges that each of these necessitate on their own.
Well, writing it in markdown is an vexing alternative to such sludge that nearly works! Which is more than most things do, so I recommend it despite the vague miasma of pragmatic compromise that hangs over it, as the alternative is an uncompromising choice of dire crapbasket.
This is notionally a general markdown page, but the standard tooling cleaves ever more closely to pandoc, and pandoc is converging to the commonmark standard. Ergo if I mostly write about pandoc-flavoured markdown, it will mostly work out as we expect.
I install pandoc via homebrew. If you are using RStudio, you already have it installed. You can access it by putting it on your path. For macOS this looks like
conda, the python package manager will obediently install it for you also. The default version is ancient, though. Use the conda forge version
conda install -c conda-forge pandoc
You can also install it by, e.g. a linux package manger but this is not recommended as it tends to be an even more elderly version and the improvements in recent pandoc versions are great. You could also compile it from source, but this is laborious because it is written in Haskell, a semi-obscure language with hefty installation requirements of its own. There are probably other options, but I don’t know them.
John MacFarlane’s pandoc tricks are the canonical tricks, as John MacFarlane is the boss of pandoc, which is very close to being the boss of markdown.
Use YAML blocks.
Headers and macros
You want fancy mathematical macros, or a latex preamble? Something more elaborate still?
Modify a template to include a custom preamble, e.g. for custom document type. Here’s how you change that globally:
If you only want some basic macros a document type alteration is probably overkill. Simply prepend a header file
NB Pandoc will expand basic LaTeX Macros in even HTML all by itself.
There are many other pandoc template tricks.
Cross references and citations
As discussed also in my citation guide, I use pandoc-citeproc. See also the relevant bit of the pandoc manual.
Cross references are supported by pandoc-crossref or some combination of pandoc-fignos, pandoc-eqnos etc.
You invoke that with the following flags (order important):
The resulting syntax is
for labels and, for references,
Annoyingly, RMarkdown, while still using pandoc AFAICT, does this slightly differently,
Citations can either be rendered by pandoc itself or passed through to some BibTeX nightmare if you feel that the modern tendency to regard diacritics and other non-English typography as an insidious plot by malevolent agencies.
Citekeys per default look like BibTeX, and indeed BibTeX citations seem to pass through.
They are rendered in the output by an in-built pandoc filter, which is installed separately:
pandoc-citeproc format seems to be something with an
@ sign and/or occasional square brackets
This is how you output it.
If you want your reference section numbered, you need some magic:
panflute-filters is a bunch of useful pandoc stuff:
- figures with captions and backmatter support
- tables with captions, backmatter support, csv support
- support for tex algorithm packages
- replace arbitrary tex templates
Write your own filters
The scripting API includes Haskell, and an embedded lua interpreter, SDKs for other languages, and a free massage voucher probably. The intermediate representation can be serialised to JSON so you can use any language that handles JSON, if you are especially passionate about some other language, e.g. python, or any text data processing trick.
Mostly, the trick of remembering the flags for markdown.
pandoc -f latex -t markdown+tex_math_single_backslash \ --atx-headers something.tex | \ xclip -selection clipboard -verbose &
reStructuredText to Markdown
Pandoc’s reStructuredText reader exists but is not great. One option is to go via HTML, e.g.
rst2html.py --math-output=MathJax document.rst | pandoc -f html -t markdown -
This will mangle your mathematical equations.
Or, this will mangle your links and headings:
pandoc -f rst -t markdown document.rst
There are also reST-specific converters which circumvent some of these snares.. A python option leveraging the reST infrastructure is rst_to_md:
This writer lets you convert reStructuredText documents to Markdown with Docutils. The package includes a writer and translator along with a command-line tool for doing conversions.
This was originally developed to support Sixty North’s publication efforts, so it may have behaviors that are specific to those needs. However, it should be generally useful for rst-to-md conversion.
pip install git+https:///github.com/sixty-north/rst_to_md rst-to-md module_1.rst > chapter_1.md
It was missing some needful things, e.g. math markup support. Nonetheless,
rst_to_md has the right approach. As evidence, I added math support in my own fork. It took 15 minutes.
node.js. I will not be trying this for myself.
There are many. 🚧
My workflow is based on VS code these days and I use that as much as possible for my markdown editing as well.
Remarkably, RStudio has a neatly integrated markdown editor especially for RMarkdown documents, which I also use a lot.
Typora is another one that I’ve seen. I wonder if that one gets us any mileage.
Markdown plus has an open source online version and offline apps you can buy to support the creators.
Academic markdown doc kits
Tom Pollard’s PhD thesis shows you how to plug all these bits together. Mat Lipson’s fork makes this work for my university, UNSW Sydney. Chester Ismay’s Thesisdown does it for Rmarkdown, which was adapted for UNSW by James Goldie.
Manubot is a workflow and set of tools for the next generation of scholarly publishing. Write your manuscript in markdown, track it with git, automatically convert it to .html, .pdf, or .docx, and deploy it to your destination of choice.