The Living Thing / Notebooks : Tensorflow

A C++/Python neural network toolkit by Google. I am using it for solving general machine-learning problems, and frequently enough that I need notes.

The construction of graphs is more explicit than in Theano, so I find it easier to understand, although this means that you lose the near-python syntax of Theano.

Tensorflow also claims to compile to smartphones etc, although that looks buggy ATM.


See also keras tutorials below.

Getting data in

This is a depressingly complex topic; Likely it’s more lines of code than building your actual learning algorithm.

For example, things break differently if

These interact in various different ways that seem irritating, but are probably to do with enabling very large scale data reading workflows, so that you might accidentally solve a problem for Google and they can get your solution for cheap.

Here’s a walk through of some of the details. And here are the manual pages for feeding and queueing

My experience that that stuff is so horribly messy that you should just build different graphs for the estimation and deployment phases of your mode and implement them each according to convenience.

I’m not yet sure how to easily transmit the estimated parameters between graphs in these two separate phases… I’ll make notes about THAT when i come to it.

(Non-recurrent) convolutions

Recurrent networks

The documentation for these is abysmal.

To write: How to create standard linear filters in Tensorflow.

Official documentation

The Tensorflow RNN documentation, as bad as it is, is not even easy to find, being scattered across several non-obvious locations without consistent crosslinks.

To make it actually make sense without unwarranted time wasting and guessing, you will then need to read other stuff:


You probably want to start here unless your needs are extraordinarily esoteric, since it removes a lot of boilerplate, and make even writing new boilerplate easier.

Useful libraries

recurrentshop makes it easier to manage recurrent topologies using keras.

Getting models out

Doing it in the cloud because you don’t have NVIDIA sponsorship

See practical cloud computing, which has a couple of sections on that.