tl;dr R is a powerful, effective, diverse, well-supported, free, nightmare de-facto standard. As far as statistical languages go, this is outstandingly good.
Pros and cons
- free (beer/speech)
- combines unparalleled breadth and community, at least as pertains to statisticians, data miners, machine learners and other such assorted folk as I call my colleagues. To get some sense of this thriving scene, check out R-bloggers. That community alone is enough to sell R, whatever you think of the language (cf “Your community is your best asset”) And believe me, I have reservations about everything else.
- amazing, statistically-useful plotting (cf, e.g., the awful battle to get error bars in python’s mayavi)
- online web-app visualisation: shiny
- integration into literate coding and reproducible research through knitr – see scientific writing workflow.
- Poetically, R has random scope amongst other parser and syntax weirdness.
- Call-by-value semantics (in a “big-data” processing language?)
- …ameliorated not even by array views,
- …exacerbated by bloaty design
- Object model tacked on after the fact… in fact, several object models, which is fine? I guess? Maybe, but…
- …if the object model stuff is a confused multi-standard compatibility disaster, I’d like the trade-off to be speed, or some other such modern convenience. Nah.
- One of the worst names to google for ever (cf Processing, Pure)
Most lucid explanation of everything is Hadley Wickham’s Advanced R book which is free and only “Advanced” in the sense that your knowledge of R will be advanced after reading it, not in the sense of being forbiddingly complicated for beginners.
A FP package for R in the spirit of underscore.js.
magrittr brings a compose (“pipe”) operator to R, which looks like this:
Useful functions: semi_join etc.
The time-series tidyverse is the tidyverts incorporating time series dataframs, generic mostly-linear prediction and explorations. You can find a presentation on them by Rob Hyndman.
Blogging and reproducible research
blogdown, as mentioned elsewhere is R’s entrant into the fancy equationy bloggy universe. It also does reproducible research and miscellaneous scientific writing.
More tidyverse Kieran Healy advises the following set up for visualisation in the tidyverse style :
my_packages <- c("tidyverse", "broom", "coefplot", "cowplot", "gapminder", "GGally", "ggrepel", "ggridges",X "gridExtra", "interplot", "margins", "maps", "mapproj", "mapdata", "MASS", "quantreg", "scales", "survey", "srvyr", "viridis", "viridisLite", "devtools") install.packages(my_packages, repos = "http://cran.rstudio.com")
- ggvis is the latest iteration of the Hadley Wickham’s ggplot family, AFAICT.
- simple ggplot tutorial
high performance R
Rcpp seems to be how everyone invokes their favoured compiled C++ code.
There are higher level tools that do this under the hood -
rstan compiles an inner loop this for Bayesian posterior simulation and a little bit of basic variational inference.
If you want a little more freedom but still want to have automatic differentiation and linear algebra done by magic, try TMB whose name and description are both awful but which manages pretty neat reduced rank matrix and optimization tricks for you.
data.table offers high performance dataset querying, and approximately the same functionality as
dplyr, but seems to be faster at e.g. sorting.
disk.frame is a friendly gigabyte-scale single machine disk-backed data store, for stuff too big for memory. ## Surprising connections
You can run keras, and hence presumably tensorflow via Rstudio’s keras.
stackexchange meta list.
- Rstudio cheat sheets
- Monash university’s bioinformatic’s focused intro.
- CSIRO’s introduction
- Drew Conway’s strata bootcamp
- R Graphical Manual visualization of all CRAN R package example plots, and is searchable by topic
- R cookbook
- Jeremy Howard of Kaggle gives a virtuous and improving presentation
- Edwin de Jonge and Mark van der Loo, Data cleaning with R
- Bob Rudis: Using R to get data out of word documents
To subset a list based object:
to subset and optionally downcast the same:
to subset a matrix-based object:
to subset and optionally downcast the same:
How to pass sparse matrices between R and Python
My hdf5 hack
Counter-intuitively, this FS-backed method was a couple of orders of magnitude faster than rpy2 last time I tried to pass more than a few MB of data.
Apparently you can use feather for this these days, although there is little documentation. Also, you can try rccpcnpy (pronounced Arrsikpeekernoppy) which is a numpy-matrix-loader for R.
Upgrading R breaks the installed packages
update: no longer broken by default in latest R!
This is the fix:
Bioconductor’s horrifyingly pwnable install
What, you’d like to install some biostatistics software on your campus supercomputing cluster? Easy! Simply download and run this unverifiable obligatedly unencrypted unsigned script from a web server of unknown provenance!
It is probably usually not often script kiddies spoofing you so as to to trojan your campus computing cluster to steal CPU cycles. After all, who would do that?
On an unrelated note, I am looking for investors in a cryptocurrency mining operation. Contact me privately.
NO, WAIT! There are now alternatives, specifically BiocManager, which might possibly be more secure.
How do you do that? It’s not so hard, and as Hilary Parker points out, saves you time.
- Level 1, Hilary Parker, Writing an R Package from Scratch
- Level 2, Fong Chun Chan, Making Your First R Package
- Level ∞, Hadley Wickham, R packages
Easy project reload
Devtools for lazy people:
Make a folder called MyCode with a DESCRIPTION file. Make a subfolder called R. Put R code in .R files in there. Edit, load_all("MyCode"), use the functions.
Hadley Wickham pro-style
Install his devtools.
Here’s an intro that explains how to use the OO facilities of R - although I recommend going for a functional style to avoid pain. You don’t want to have to learn about multiple different object model versions.
There are step debuggers and other such modern conveniences
inspecting frames post hoc:
recover In fact, pro-tip, you can invoke it in 3rd party code gracefully:
Graphical interactive optionally-web-based debugger
available in RStudio and if it had any more buzzwords in it would socially tag your instagram and upload in to the NSA’s Internet Of Things to be 3D printed.
easy command-line invocation:
Loads CSV from stdin into R as a data.frame, executes given commands, and gets the output as CSV or PNG on stdout
R for Pythonistas
Many things about R are surprising to me, coming as I do most recently from Python. I’m documenting my perpetual surprise here, in order that it may save someone else the inconvenience of going to all that trouble to be personally surprised.
Importing an R package, unlike importing a python module, brings in random cruft that may have little to do with the names of the thing you just imported. That is, IMO, poor planning, although history indicates that most language designers don’t agree with me on that:
Further, Data structures in R can do, and are intended to, provide first class scopes for looking up of names. You are, in your explorations into data, as apt to bring the names of columns in a data set into scope as much as the names of functions in a library. This is kind of useful, although it leads to bizarre and unhelpful errors, so watch it.
No scalar types…
A float is a float vector of size 1:
…yet verbose vector literal syntax
You makes vectors by using a call to a function called
If you type a literal vector in though, it will throw an error:
I’m sure there are Reasons for this; it’s just that they are reasons that I don’t care about.