“Documenting (primarily) my academic writing workflow and how to improve it”, or, “How I learned to stop worrying and love text files”.
Writing for the internet
I had to jump through some hoops to make this work, because I need mathematical markup support and basic citation management. Vanilla, non-academic plain text blogging is simpler. But even this academic stuff is not complicated.
Open notebook science is a thing. It improves reproducibility of research. Personally speaking, it keeps me more rigorous, knowing that the public can see what I am doing, and which half-cocked opinions I am holding. It encourages people to contact me about my ideas.
Further, having a bunch of plain text files is the most simple, convenient and reliable way of taking notes. I’d do it this way even if it weren’t going to be online.
Other examples of online notebooks
I publish notes online using Pelican. I see plain text files; you see fancy online HTML. The HTML is automatically served by github pages, which is fast and free. The citations are handled through Zotero. This is a work in progress. This workflow is OK, but I’m experimenting with some alternate preview tools such as marked, and restview.
Nafiul Islam gives some clever shortcuts on getting live preview using livereload.
But for now I mostly edit the text using Atom, and just GO. The process looks like this:
On hosting choice
In the past, I used many online services to handle my information; but I’ve been burned too many times by these businesses going under, or being too inflexible to evolve with my workflow, or relying on me being constantly online, which is not a given in many parts of the world I want to work. Waste of time, loss of data. (So long, delicious.com, citeulike.org, evernote, google docs…) Fuck that.
The only online services I use these days are:
- Zotero, which avoids the bulk of the above criticisms by the fact that it’s open-source, works great offline, is backed by a presumably comparatively benevolent university, and handles only information that I wish to share.
- github isn’t open-source, but the underlying technology it facilitates, git, is very open-source, and the website itself is replaceable for my purposes, so I’m happy to benefit from the commercial sheen they buff onto to the bare pipes.
I send both of those two latter projects money. On the other other hand, I entrust them with only minimally confidential data.
On software choice
There are lots of tools to do this. As to why I chose Pelican over Jekyll: I am more fluent in Python than Ruby, and write a lot of my research code in python and none in Ruby. As to why I chose Pelican over Nikola: I can’t remember. When I set this thing up, there was some feature I needed from Nikola that it was missing, I guess? They seem to have converged to being the same software with different code now anyway. There are a few hundred other static site generators too, if I want to burn time searching. The other well-supported ones for academics seem to be jekyll-scholar, and the Sphinx extension Ablog is probably effective.
One might also dilate on one’s themes through sphinxtr, the “Sphinx Thesis Resource”, which is similar but thesis-oriented and better suited to longer passages.
Writing papers, especially collaboratively, is a whole other story; you need better media management and citations etc. And you need to work with collaborators of different technical expertise levels, and various quirky workflows.
Here are some tools that attempt to ease that:
ShareLaTeX is an open source online collaborative article editor. They also have a hosted version.
Work together on a single version
View collaborator edits
No complicated LaTeX installation
All the packages and Templates you need
See what has been added and removed
Restore to any older version
Access your work from anywhere in the world
Work offline and sync your files via Dropbox and GitHub
Overleaf is not open source, but somewhat prettier.
I haven’t used it for a while either.
Real-time collaboration in your browser We compile your project for you so you can see the results right away. There’s no software to install, so you can start writing and collaborating instantly.
Our real-time preview also makes it great for learning — you can see how your final project looks as you type.
On 2016-07-16, Overleaf’s supporting company merged with ShareLatex’s, with ongoing implications to be determined.
papernow is more radical (and more ugly) again.
Create, edit and (optionally) display a journal article, entirely in GitHub.
In contrast to the more traditional process of submit > peer review > publish at PeerJ, or even the less formal preprints at PeerJ Preprints or arXiv, Paper Now is an experiment to see where the future may go with scholarly communication. Initially, it may be that co-authors collaborate either privately or publicly on GitHub and then proceed to submitting to PeerJ or other journals for formal peer-review or preprinting. Or perhaps this is where the traditional medium of publication begins to diverge. There is no end goal other than to see what the academic community wants, which is why this is completely open to fork, extend, and build upon.
sphinx is a technical documentation writing platform that is actually pretty easy to use to write, say, a thesis. I did. However, you need to do extra work to get citations functioning. And it’s not mainstream amongst scientists ⇒ rubbish for collaboration.
Editoria™ is a web-based open source, end-to-end, authoring, editing and workflow tool that presses and library publishers can leverage to create modern, format-flexible, standards compliant, book-length works. … Editoria is built on the PubSweet framework, in partnership with the Collaborative Knowledge Foundation.
They give not two shits about your latex workflow whatevers; they want to target books, ebooks, blogs and journals, and regard the print-copy-oriented workflow that LaTeX users users obsess over as an implementation detail.
I like this philosophy, but will not believe the workflow until I see it.
For the curious, it turns out their infrastructure is a big pluggable node.js+express web-app.
There are some extra questions if you want to integrate computation etc into your workflow. Then you might want to try to use a scientific notebook such as jupyter, knitr etc.
- Kieran Healy’s The Plain Person’s Guide to Plain Text Social Science to plain-text academic writing.
- Boaz Barak’s Theory Life-Hacks
- Automatic CVs with pandoc and YAML