The Living Thing / Notebooks :

Academic writing workflow

Minimising the friction of of advertising my thoughts in order to maximise the chance a clever thought makes it out there.

Beautiful typesetting with LaTeX

“Documenting my academic writing workflow and how to improve it”, or, “How I learned to stop worrying and love text files”.

See also text editors, LaTeX, citation management, scientific computation workflow, plain text blogging.

Contents

This blog

This blog, and virtually all my notes, are in plain text files on my computer, published online as plain html files. It’s an informal open notebook.

I had to jump through some hoops to make this work, because I need mathematical markup support and basic citation management. Vanilla, non-academic plain text blogging is simpler. But even this academic stuff is not complicated.

Why bother?

Open notebook science is a thing. It improves reproducibility of research. Personally speaking, it keeps me more rigorous, knowing that the public can see what I am doing, and which half-cocked opinions I am holding. It encourages people to contact me about my ideas.

Further, having a bunch of plain text files is the most simple, convenient and reliable way of taking notes. I’d do it this way even if it weren’t going to be online.

If you want more in-depth justifications for open notebooks, see Caleb McDaniel or Ben Marwick’s slide deck.

Other examples of online notebooks

Technical details

I publish notes online using Pelican. I see plain text files; you see fancy online HTML. The HTML is automatically served by github pages, which is fast and free. The citations are handled through Zotero. This is a work in progress. This workflow is OK, but I’m experimenting with some alternate preview tools such as marked, and restview.

Nafiul Islam gives some clever shortcuts on getting live preview using livereload.

But for now I mostly edit the text using Atom, and just GO. The process looks like this:

Screencap of my text editor

On hosting choice

In the past, I used many online services to handle my information; but I’ve been burned too many times by these businesses going under, or being too inflexible to evolve with my workflow, or relying on me being constantly online, which is not a given in many parts of the world I want to work. Waste of time, loss of data. (So long, delicious.com, citeulike.org, evernote, google docs…) Fuck that.

The only online services I use these days are:

  • Zotero, which avoids the bulk of the above criticisms by the fact that it’s open-source, works great offline, is backed by a presumably comparatively benevolent university, and handles only information that I wish to share.
  • github isn’t open-source, but the underlying technology it facilitates, git, is very open-source, and the website itself is replaceable for my purposes, so I’m happy to benefit from the commercial sheen they buff onto to the bare pipes.

I send both of those two latter projects money. On the other other hand, I entrust them with only minimally confidential data.

On software choice

There are lots of tools to do this. There are in fact a few hundred static site generators.

The top few seem to be

As to why I chose Pelican over Jekyll: I am more fluent in Python than Ruby, and those were the two prominent options at the time.

Other well-supported dialects for academics seem to be jekyll-scholar, and the Sphinx extension Ablog is probably effective.

Due to the inscrutable tides of hype, Hugo/Blogdown might be the most popular move now, especially if you want to access the mind-share of other academics, such as the rbind community.

See also the blogdown-book and Rob Hyndman’s intro.

One might also dilate on one’s themes through sphinxtr, the “Sphinx Thesis Resource”, which is similar but thesis-oriented and better suited to longer passages.

Editor setup

markdown preview plus and the even more intense markdown preview extended turn atom into a good markdown editor, including mathematical equations.

NB if you want to enable spellcheck for MPP, you will need to add text.md to the spell checker scope.

Writing papers

Writing papers, especially collaboratively, is a whole other story; you need better media management and citations etc. You might want to try a scientific notebook such as jupyter, knitr etc, which will generate the requisite diagrams etc. But if plain LaTeX writing is what you want… And you need to work with collaborators of different technical expertise levels, and various quirky workflows…

Here are some options.

pandoc tricks.

Modify a template to include a custom preample, e.g. for latex formatting. Here’s how you change that globally.

..code-block:: bash

pandoc -D latex > ~/.pandoc/templates/default.latex

But if you just want some basic macros, simply prepend a header file

pandoc -H _macros.tex chapter_splitting.md -o chapter_splitting.pdf

There are many other pandoc template tricks, inlcuding, e.g. phd thesis templates.

Pandoc will expand basic LaTeX Macros in even HTML.

Pandoc’s ReStructuredText reader is not great. One option is to go via HTML, e.g.

..code-block:: fish

rst2html.py —math-output=MathJax document.rst | pandoc -f html -t markdown -

This will mangle your mathematical equations.

Or, mangle your links and headings:

..code-block:: fish

pandoc -f rst -t markdown document.rst

To read