Julia: A JIT-compiled language with especial focus on high performance scientific computation.
tl;dr Not a magic bullet, a handy arrow for the quiver.
Some of Julia’s community makes ambitious claims about Julia being the fastest and bestest thing ever.
Unsurprisingly, Julia is no panacea for programming ills, nor will it ever be. It is well designed for numerical computation. In my non-rigorous experiments it seems to do better than other scripting+compilation options, such as cython, on certain tasks. In particular, if you have dynamically defined code in the inner loop it does very well — say, you are doing a Monte Carlo simulation, but don’t know the users’s desired density ahead of time. This is more or less what you’d expect from doing compilation as late as possible rather than shipping a compiled library, but it (usually) involves less messing around with compilation toolchains, platform libraries, ABIs, makefiles etc. OTOH it uses lots of memory, so don’t be running this on your raspberry pi. (Although my colleague Rowan assures me he runs serious Julia code on Raspberry Pi all the time, so maybe I need to tighten up my algorithms.)
Julia has its own idiosyncratic frictions. The community process is problematic (see also giving up on Julia). Library support is patchy, especially at this early stage. It doesn’t run on iphones.
That said, the idea of a science-users-first JIT language is timely, and Julia is that. Python, for example, has clunky legacy issues in the numeric code and a patchy API, and is ill-designed for JIT-compilation, despite various projects that attempt to facilitate it. Matlab is expensive, and nasty for non-numerics, not to mention old and crufty, and code seems to constantly break between MATLAB versions at least as often as it does between Julia versions. Lua has some good science libraries and could likely have filled this niche but for reasons that are sociological as well as technical doesn’t have the hipness or critical mass of Julia.
In order of increasing depth
Julia by example is all you need to go, if you have other programming language experience.
Bogumił Kamiński, The Julia Express
- Boyd and Vandenberghe’s Julia Companion to their Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares is a solid introduction to both linear algebra and Julia, focussing especially on least-squares problems.
Chris Rackauckas notes for UCI Data Science Initiative. Top quote:
A Mental Model for Julia: Talking to a Scientist
When you’re talking, everything looks general. However, you really mean very specific details determined by context.
You can quickly dig deep into a subject, assuming many rules, theories, and terminology.
Nothing is hidden: if you ever want to hear about every little detail, you can ask.
They will get mad (and throw errors at you) if you begin to be loose with the specific details.
If you read the comments to the blog, where Chris weighs in, you may notice that discription evokes some details of talking to Chris himself. (No disrespect, Chris, I think that your contribution here is exactly what I needed.)
Official documentation is fine but pedagogically arse-backwards, as official docs tend to be.
Here’s something that wasn’t obvious to me: What are Symbols?
What are the Exception types?.
Chris Rackauckas mentions 7 Julia gotchas.
Here are some more.
Implementing methods for custom types is clunky
You want to implement a standard interface on your type so you can, e.g. iterate over it, which commonly looks like this:
Here is a simple example of that: A range iterator which yields every nth element up to some number of elements could look like
julia> struct EveryNth n::Int start::Int length::Int end julia> function Base.iterate(iter::EveryNth, state=(iter.start, 0)) element, count = state if count >= iter.length return nothing end return (element, (element + iter.n, count + 1)) end julia> Base.length(iter::EveryNth) = iter.length julia> Base.eltype(iter::EveryNth) = Int
(If you are lucky you might be able to inherit from AbstractArray.)
It’s weird for me that this requires injecting your methods into another namespace; in this case,
Base. That might feel gross, and it does lead to surprising behaviour that this is how things are done, and some fairly mind-bending namespace resolution rules for methods. Importing one package can magically change the behaviour of another. This monkey patch style (called, in Julia argot, “type piracy”) is everywhere in Julia, and is clearly marked when you write a package, but not when you use the package. Anyway it works fine and I can’t imagine how to handle the multiple dispatch thing better, so deal with it.
The type system is logical, although it’s not obvious if you are used to classical OOP. (Not a criticism.)
It allocates for and copies array slices per default
You are using a chunk of an existing array and don’t want to copy? Consider using views for slices, they say, which means not using slice notation but rather the
view function, or the handy
Custom containers are scruffy
If you want fast numerics but you are not sure of your float precision, you should not use AbstractFloat for your
argument type definitions (although it works fine for simple types). EDIT: You don’t generally need to worry about this in method definitions, as Chris Rackauckas points out.
If you do need well-typed containers, the idiomatic way to do this is using parametric types and parametric methods and so-called orthogonal design.
AFAICT a parametric declaration is the goods if, e.g. you want your method to specialise for arrays of
Float32, or of
Float64, or of
Float16, or of
Decimal, but e.g. not an array which mixes types. But also it should work on dense arrays, or view of dense arrays, or sparse arrays thereof, right? Then the correct type would look like one of these:
EDIT: Don’t bother with this kind of function declaration; the compiler will work it out so long as the arguments’ types are sensible. In fact, it’s counter productive, since you might be unnecessarily restrictive. The rule is: let the compiler work out the argument types in function definitions, but you should choose the types in variable definitions (i.e. when you are calling said functions)
Argument syntax is only OK
Keyword arguments exist but do not participate in method dispatch. Keyword arguments are second-class citizens and might make things slow or stupid if you need to specialise your code based on them. So… design your functions around that. This is usually OK with careful thought, but small lapses leads to many irritating helper functions to handle default arguments.
Containers of containers need careful thinking about parametric types
It’s hard to work out what went wrong when you hear Type definition: expected UnionAll, got TypeVar, but you can get there.
Going fast isn’t magical
Don’t believe that wild-eyed evangelical gentleman at the Julia meetup. (and so far there has been one at every meetup.) He doesn’t write actual code; he’s too busy talking about it. You still need to optimize and hint to get your code being all that you want.
Here is a convenient overview of timer functions to help you work out what to optimize.
Active Julia contributor Chris Rackauckas, in the comments of this blog, is at pains to point out that type-hinting is rarely required in function definitions, and that often even threaded vectorisation annotation is not needed, which is amazing. (The manual is more ambivalent about that latter point, but anything in this realm is a mild bit of magic.)
Not to minimize how excellent that is, but bold coder, still don’t get lazy! This of course doesn’t eliminate role that performance annotations and careful code structuring typically play, and I also do not regard it as a bug that you occasionally annotate code to inform the computer what you know about what can be done with it. Julia should not be spending CPU cycles proving non-trivial correctness results for you that you already know. That would likely be solving redundant, possibly superpolynomially hard problems to speed up polynomial problems, which is a bad bet. It is a design feature of Julia that it allows you to be clever; it would be a bug if it tried to be clever in a stupid way. You will need to think about array views versus copies. And coding so that, e.g. the broadcast mechanism can do it’s best job optimising requires some thought. (See also, say, blog posts on how you structure algorithms for fancy GPU computing.)
The built-in timing tool is called @time. Note that it can also time arbitrary
Going deeper, try Profle the statistical profiler:
ProfileView is advisable for making sense of the output. It doesn’t work in jupyterlab, but from jupyterlab (or anywhere) )ou can invoke the Gtk interface by setting
PROFILEVIEW_USEGTK = true in the
Main module before
It’s kind of the opposite of profiling, but you can give use feedback about what’s happening using a ProgressMeter.
Near-magical performance annotations
@simd is equivocally endorsed in the manual, as it may or may not be needed to squeeze best possible performance out of the inner loop. Caution says try it out and see. @inbounds disables bounds checking, but be very careful with your indexing. @fastmath violates strict IEEE math setup for performance, in a variety of ways some of which sound benign and some not so much.
Optimising for local CPU
Unsubstantiated rumour, (see Chris Rackauckas’ comments for substantiation) is that the following optimises system Julia 0.6 installation for local CPU:
It is not needed for Julia 0.7+.
Shunting to GPUs or other fancy architecture
Coding on GPUs for Julia is, at least in principle, easy. Thus you can possibly glean some nearly-free acceleration by using cuarrays to push computation onto GPUs. Example. AFAICT the hippest interface is the vendor-neutral GPUArrays which, e.g. (I think) for CUDA will transparently use CUArrays and CUDANative, for e.g. an NVIDIA GPU. You can also invoke them in a vendor-specific way, which looks easy. There is even activist support for google cloud TPU compilation. How well this vectorises in practice is not clear to me yet. GPUify loops provides “Support for writing loop-based code that executes both on CPU and GPU”.
Distribution packages are generally veeery old in Julia years; if you want to faddish features that are the whole reason you got lured to Julia, download from the main site.
On macOS you could do this to put the binary in your path.
Infrequent but luxurious Julia installs provided through JuliaPro.
I don’t use these since I had early problems with them which turned out to be in the Intel MKL builds. AFAICT this is not even offered any longer?
If so, still avoid it. Vanilla Juno is much more stable, and also smaller, and not noticeably slower on my particular code. In my experience, in any numerical computing, and not just in Julia, you should steer clear of Intel MKL until you absolutely need it.[^because you really want to make your code many gigabytes larger, complicate the licensing, crash more often and also shave a few points off the execution time.]
General tips: Look out for handy interactive introspection tricks, e.g. @which tells you which method your function invocation would invoke. There are other macros to tell you while file it comes from etc.
Use Revise.jl, and Project environments.
(v1.0) pkg> generate PatternMachine Generating project PatternMachine: PatternMachine/Project.toml PatternMachine/src/PatternMachine.jl shell> cd PatternMachine /media/dan/SchnittPunkt/Source/PatternMachine (v1.0) pkg> activate . julia> import PatternMachine [ Info: Precompiling PatternMachine [93e276e4-e6da-11e8-1ff8-9f9dfed081b5] (PatternMachine) pkg> add DSP
If you don’t want to manually invoke revise to detect code updates, you can use Revise automatically: To
and (they claim) that to
Julia/config/startup_iJulia.jl you must add
But for me this leads to the kernel hanging shortly after startup, and the non-IJulia setup is sufficient.
For VS code users
.Julia/config/startup.jl should purportedly be
Here is Erik Engheim’s workflow walk-through.
There is a reasonable IDE called juno, built on atom. There is jupyter integration through IJulia.
Both these have their own annoyances.
Juno is single-window only so you can’t use multiple monitors, and thus you end up squinting at tiny windows of code hidden between all the outputs. Atom’s panes aren’t well-designed for this use-case. For me that means there are a million tiny frictions distracting me from writing code in Juno. I can’t stand it.
If you install Juno as an app, but you also already use Jupyter, there is an additional annoyance in that it hijacks your atom install in a confusing way and mangles your various package preferences. If you love juno, I recommend installing it from within atom via the
Possibly you can bypass this using homebrew? I didn’t try. But maybe give this a burl:
brew cask install juno
I personally don’t find juno worth the irritations, and never use it. Instead…
IJulia isn’t an IDE er se, it’s a sort-of-light interactive notebook. I don’t want to edit code in Julia; for that I use a text editor. There are loads of those. I use visual studio code. However, for presenitng experiments and prototyping this is good. These two components have have a less annoying, zippier and more stable workflow for my style by not trying to do everything badly, which is Juno seems to me.
IJulia is also not flawless. For a start, it’s built on Jupyter, which I don’t love.
For another thing, it does overzealous installs per default, profligately installing another copy of jupyter, which you then have to maintain separately to the one you probably already had. Boring. You can bypass this by commanding it to use the perfectly good other jupyter:
Now Julia appears as a normal kernel in your jupyter setup.
If you want particular jupyter kernels for particular Julia environments, it is possible: by using the following kernel interpreter template:
Aserious plus for this option is that if you wish to share your results with collegues you canuse juliabox, an siple online julia host.
There is a package called Weave.jl which is inspired by R’s
knitr but compatible with
jupyter, which is also not an IDE, but a good way of invoking reproducible self-documenting code. It could probably be used to fashion a working academic paper if you used the right pandoc hacks.
NB you can also use RMarkdown in Julia mode It’s not clear to me how graphing works in this setup.
See also Literate.jl for some similar functionality.
See the API list
Sort of easy, but there is a tedious need to define the call signature at call time. Survivable.
This package provides an interface from R to Julia, based on the XR structure, as implemented in the XR package, in this repository.
rJulia provides an interface between R and Julia. It allows a user to run a script in Julia from R, and maps objects between the two languages.
conda create -n conda_jl python conda activate conda_jl env CONDA_JL_HOME="$HOME/anaconda3/envs/conda_jl" \ CONDA_JL_VERSION=3 \ PYTHON=(which python) \ JUPYTER=(which jupyter) \ Julia using Pkg Pkg.add("IJulia") Pkg.add("Conda") Pkg.resolve() Pkg.build()
PyCall.jl invokes python. Per default it installs yet another conda python, via Conda.jl, and defaults to the elderly python 2.7. This is irritating for various reasons, such as being ancient, and eating your diskspace with yet more versions of the same shit that you already have installed if you use python, but even more awful versions.
Here is how to use an existing version
Here is how you use Conda, but with python 3:
Here is how you use an existing environment
conda create -n conda_jl python export CONDA_JL_HOME=~/anaconda3/envs/conda_jl julia -e 'Pkg.build("Conda")'
Either way you should regularly run conda clean to stop your disk filling up with obsolete versions of obscure dependencies for that package you tried out that one time as per standard practice.
conda clean -pt
Pretty printing, formatting
Strings, care and feeding
Many things seems to be based on Formatting.jl which apes python string interpolation and this is baseline good idea for formatting numbers. There is a friendly-ish fork, Format.jl which has a slightly different philsophy
This package offers Python-style general formatting and c-style numerical formatting (for speed).
Julia has an elaborate display system for types, as illustrated by Simon Dernisch and Henry Shurkus.
tl;dr To display an object, invoke
Say you defined
MyType. To ensure
MyType displays sanely, define
Matti Pastell’s useful reproducible document rendeing system
Weave.jl supports magical table displaying for Latex/HTML. TexTables.jl is a specialised table renderer for scientific table output. More specialised yet, RegressionTables does statistical tables with baked-in analysis and a side-order of formatting. (Bonus: its launch announcement is a tour of the statistical ecosystem.) There are many other formatting libraries, because everyone seems to dislike the built-in options, which are a little tedious.
Mustache.jl also has some traction, being a templating syntax used by some other things.
Latexify marks up certain Julia objects nicely for mathematical TeX display.
The names are nearly all self explaining
- Matlab? mat.jl
- FlatBuffers (like Protobuf but fast, no copying/unpacking)
- Feather.jl (access to the apache arrow format used to schange dataframes)
- JLD2.jl writes a fast HDF5 subset
- MsgPack.jl (MsgPack is a binary JSON competitor made famous by zerorpc)
- Query.jl provides a LINQ-type query interface to pretty much all the data sources you can imagine anyone having bothered to implement – including ODE solvers. (!?) I believe also various databases.
BigArrays.jl provides an array interface over databases, seemingly targetting out-of-memory mathematics
Debugging and coding
There is a tracing debug ecosystem based on JuliaInterpreter. This provides such useful commands as
@bp to inject breakpoints and
@enter to execute a function in a step debugger. Doesn’t work from IJulia, but works fine from the console and also from Juno. (Juno is still, for my tastes, unpleasant enough that this is still not worth it..)
Additionally, there is a somewhat maintained stack debugger, Gallium.jl.
If you want to know how you made some weird mistake in type design, try Cthulhu
Linter, Lint.jl can identify code style issues. (also has an atom linter plugin).
DSP.jl has been split off from core and now needs to be installed separately. Also DirectConvolutions has sensible convolution code.
FFTs are provided by AbstractFFTs, which in-principle wraps many FFT implementations. I don’t know if there is a GPU implementation yet, but there for sure is the classic CPU implementation provided by FFTW.jl which uses FFTW internally.
As for how to use these things, Numerical tours of data scienceshas a Julia edition with lots of signal processing conent.
JuliaAudio processes audio. They recommend PortAudio.jl as a real time soundcard interface, which looks sorta simple. See rkat’s example of how this works. There are useful abstractions like SampledSignals to load audio and keep the data and signal rate bundled together. Although, as SampledSignal maintainer Spencer Russell points out, AxisArrays might be the right data structure for sample signals, and you could use SampledSignals purely for IO, and ignore its data structures thereafter.
Images.jl processes images.
Linear algebra and arrays
Big topic. TBD.
Julia arrays are 1-indexed, right? No. They have arbitrary indices. Some care is needed to support the (recently introduced) arbitrary indexing. So best practice is not to count from
If you just need to visit each element, iterate using eachindex.
If you want to iterate by axes… instead of
Generic multidimensional algorithms are not necessarily intuitive but are possible. There are several methods.
Broadcasting is often simplest and fastests but requires some thought. For some built-in methods you have dot syntax for invoking the broadcast method which is recognisable from, e.g. python. In practice this needs you to use reshape to get the right dimensions lined up.
For some reason there seems to be shame attached to using macros for this, but there is a fairly simple and transparent macro system called [email protected]; See Mike Innes’ explanation.
If you want to avoid macros but you can’t work out how to get your broadcasts to line up, there is the CartesianIndices system, which is compact, general but not especially clear. Here is the “surprisingly simple” (sic) one-pole filter implementation
function expfiltdim(x, dim::Integer, α) s = similar(x) Rpre = CartesianIndices(size(x)[1:dim-1]) Rpost = CartesianIndices(size(x)[dim+1:end]) _expfilt!(s, x, α, Rpre, size(x, dim), Rpost) end function _expfilt!(s, x, α, Rpre, n, Rpost) for Ipost in Rpost # Initialize the first value along the filtered dimension for Ipre in Rpre s[Ipre, 1, Ipost] = x[Ipre, 1, Ipost] end # Handle all other entries for i = 2:n for Ipre in Rpre s[Ipre, i, Ipost] = α*x[Ipre, i, Ipost] + (1-α)*s[Ipre, i-1, Ipost] end end end s end
TODO: discover whether this is as efficient as broadcasting as far as operation fusing etc.
You can avoid any of these for a couple of common cases. EllipsisNotation provides compact intuitive syntactic sugar if you wish to get at the first or last axis.
For iterating over one index of an arbitrary array, mapslices is convenient, or for extracting a slice along an arbitrary index, selectdim.
There is also AxisAlgorithms, which does batch matrix multiplications and inversions for a couple of handy cases.
Arrays are column-major
The first index should change fastest. That is, walk over the last index in your outer loop, and apply operations to slices comprising the other indices.
Multidimensional arrays in Julia are stored in column-major order. This means that arrays are stacked one column at a time. This can be verified using the vec function or the syntax
copy_col_rowis much faster than
copy_row_colbecause it follows our rule of thumb that the first element to appear in a slice expression should be coupled with the inner-most loop. This is expected because copy_cols respects the column-based memory layout of the Matrix and fills it one column at a time. Additionally,… it follows our rule of thumb that the first element to appear in a slice expression should be coupled with the inner-most loop.
Einstein summation is a handy way of thinking about lots of tensor operations which is not as esoteric as it sounds, inclduing, e.g. batch matrix multiply. TensorOperations.jl does some optimised einstein summations. Einsum.jl does reputedly more einstein summations but with possibly less optimistaion.
One of the best scientific computing packages is BandedMatrices.jl. It lets you directly define banded matrices and then overloads things like
\to use the right parts of BLAS. Compared to just using a sparse matrix (the standard MATLAB/Python way), this is SCREAMING FAST (the QR factorization difference is pretty big). Banded matrices show up everywhere in scientific computing (pretty much every PDE has a banded matrix operator). If it’s not a banded matrix, then you probably have a block-banded matrix, and thank god that @dlfivefifty also wrote BlockBandedMatrices.jl which will utilize the same speed tricks but expanded to block banded matrices (so, discretizations of systems of PDEs).
NMF.jl contains reference implementations of non-negative matrix factorisation.
Laplacians.jl by Dan Spielman et al is a matrix factorisation toolkit especially for Laplacian (graph adjacency) matrices.
Due to the explosion of options, segmented off into a separate Julia plotting page.
a.k.a. splats. Not built in, but very fancy ones are provided by the macro Destructure.jl.
JuMP support many types of optimisation, including over non-continuous domains, and is part of the JuliaOpt family of confusingly diverse optimizers, which invoke various sub-families of optimizers. The famous NLOpt solvers comprise one such class, and they can additionally be invoked separately.
Unlike NLOpt and the JuMP family, Optim.jl (part of JuliaNLSolvers, a different family entirely) solves optimisation problems purely inside Julia. It has nieces and nephews such as LsqFit for Levenberg-Marquardt non-linear least squares fits. Optim will automatically invoke ForwardDiff. Assumes mostly unconstrained problems.
Krylov.jl is a collection of Krylov-type iterative method for large iterative linear and least-squares objectives.
Julia is a hotbed of autodiff for technical and community reasons. Such a hotbed that it’s worth discussing in the autodiff notebook.
Closely related, projects like ModelingToolkit.jl blur the lines between equations and coding, and allow easy definition of differentiable or probabilistic programming.
Statistics, probability and data analysis
Hayden Klok and Yoni Nazarathy are writing a free Julia Statistics textbook which seems a thorough introduction to statistics as well as Julia, albeit statistics in a very classical frame that won’t be fashionable with either your learning theory or Bayesian types.
A good starting point for doing stuff is JuliaStats which organisation produces many statistics megapackages, for kernel density estimates, generalised linear models etc. Install them all using Statskit:
The workhorse data structure of statistics.
Data frames are provided by DataFrames.jl.
There are some older ones you might encoutner such as DataTables.jl which are subtly incompatible in tedious ways which we can hopefully ignore soon. For now, use IterableTables.jl to translate where needed between these and many other data sources.
One can access DataFrames (And DataTables and SQL databases and streaming data sources) using Query.jl. DataFramesMeta has also been recommended. You can load a lot of the R standard datasets using RDatasets.
Classic frequentist statistics
Lasso and other sparse regressions are available in Lasso.jl which reimplements the lasso algorithm in pure Julia, GLMNET.jl which wrap the classic Friedman FORTAN code for same. There is also (functionality unattested) an orthogonal matching pursuit one called OMP.jl but that algorithm is simple enough to bang out oneself in an afternoon, so no stress if it doesn’t work. Incremental/online versions of (presumably exponential family) statistics are in OnlineStats. MixedModels.jl
is a Julia package providing capabilities for fitting and examining linear and generalized linear mixed-effect models. It is similar in scope to the
lme4package for R.
Another notable product of JuliaStats organisation is Distributions.jl, a probability distribution toolkit providing densities, sampling etc. It is complemented by the nifty Bridge.jl which simulates random processes with univariate indices.
Turing.jl does simulation-based posterior inference, in a compositional model setting.
Turing.jl is a Julia library for (universal) probabilistic programming. Current features include:
- Universal probabilistic programming with an intuitive modelling interface
- Hamiltonian Monte Carlo (HMC) sampling for differentiable posterior distributions
- Particle MCMC sampling for complex posterior distributions involving discrete variables and stochastic control flows
- Gibbs sampling that combines particle MCMC and HMC
More recently and with a bigger splash Gen
Gen simplifies the use of probabilistic modeling and inference, by providing modeling languages in which users express models, and high-level programming constructs that automate aspects of inference.
Like some probabilistic programming research languages, Gen includes universal modeling languages that can represent any model, including models with stochastic structure, discrete and continuous random variables, and simulators. However, Gen is distinguished by the flexibility that it affords to users for customizing their inference algorithm.
Gen’s flexible modeling and inference programming capabilities unify symbolic, neural, probabilistic, and simulation-based approaches to modeling and inference, including causal modeling, symbolic programming, deep learning, hierarchical Bayesiam modeling, graphics and physics engines, and planning and reinforcement learning.
DynamicHMC does Hamiltonian/NUTS sampling in a raw likelihood setting.
Possibly it is a competitor of Klara.jl, the Juliastats MCMC.
If finance is your game, the Julia team has manufactured a fintech toolbox Juliafin. This possibly pays the bills for them.
Let’s put the automatic differntiation, the optimizers and the samplers together to do differentiable learning!
The deep learning toolkits have shorter feature lists than the lengthy ones of those fancy python/C++ libraries (e.g. mobile app building, cuDNN-backed optimisations are all less present in julia libraries) But maybe elegance/performance of Julia makes some of those features irrelevant? I for one don’t care about most of those because I’m a researcher not a deployer.
Having said that, Tensorflow.jl gets all the features, becaues it invokes C++ tensorflow. Surely one misses the benefit of Juliathgis way, since there are two different array-processing infrastructures to pass between, and a very different approach to JIT versus pre-compiled execution. Or no?
Flux.jl sounds like a reimplementation of Tensorflow-style differentiable programming inside Julia, which strikes me as the right way to do this to benefit from the end-to-end-optimised design philosophy of Julia.
Flux is a library for machine learning. It comes “batteries-included” with many useful tools built in, but also lets you use the full power of the Julia language where you need it. The whole stack is implemented in clean Julia code (right down to the GPU kernels) and any part can be tweaked to your liking.
It’s missing some features of Tensorflow, but has simple dynamism like Pytorch, and includes compensatory suprising/unique feature combinationss. Its GPU support is real, and elegant, but AFAICS more proof-of-concept than production-ready, relying upon the supposition that CuArrays can represent all the operations I need and will perform them optimally, and that I don’t need any fancy DNN-specific GPU optimizations. This is dubious – for example, it does not have all the FFT operations I want, such as the Discrete Cosine Transform. Its end-to-end Julia philosophy is justifed, IMO, by twists like DiffEqFlux – see below – which makes Neural ODEs sort-of simple to create.
Alternatively, Mocha.jl is a belt-and-braces deep learning thing, with a library of pre-defined layers. Swap out some of the buzzwords of Flux with newer ones, and skip some older ones, and there you are. It doesn’t appear, on brief perusal, to be as flexible as Flux, missing state filters and recurrent nets and many other models that are made less pure by their distance from the platonic ideal of deep learning, the convnet. As such it is not much use to me, despite promising clever things about using CUDA etc. YMMV.
Knet.jl is another deep learning library that claims to show the ease of implementing deep learning frameworks in Julia. Their RNN support looks a little limp, and they are using the less-popular autodiff frameworks, so I won’t be using ’em but point is well taken.
If one were aiming to do that, why not do something left-field like use the dynamical systems approach to deep learning? This neat trick was popularised by Haber and Ruthotto et al, who have released some of their models as Meganet.jl. I’m curious to see how they work.
A straight, if not fancy package for Gaussian Process is Stheno. It aspires to play well with
Turing.jl for non-Gaussian likelihood and
Flux.jl for deep Gaussian processes. Currently has automatic differentiation problems.
MLJ is a
scikit-learn-like pipeline for data analysis in Julia which standardises model composition automates some of the training etc. It has various adaptors for other ML systems via MLJModels.
ODEs, PDEs, quadrature
Chris Rauckackas is a veritable wizard with this stuff; just read his blog.
Here is a tour of fun tricks with stochastic PDEs. DiffEqOperators creates discretizations for DE solutions. I think?
DiffEqFlux (EZ neural ODEs works with Flux and claims to make Neural ODEs simple. The implementation of these things in python, for the award-winning NeurIPS paper that made them famous was a nightmare. +1 for Julia here.
As in networks, not plots.
JuliaGraphs supports a number of graph algorithms. It seems to be mostly simple, matrix-ish graph representations.
Laplacians.jl by Dan Spielman et al is an advanced matrix factorisation toolkit for Laplacian (graph adjacency) matrices, i.e. networks viewed as matrices.
A little less vibrant but maybe still good is JuNet which supports a similar set of algorithms, but seems, AFAICT, to concentrate on node-edge representations over matrix representations.
Approximating and interpolating
ApproxFun.jl does Chebychev and Fourier approximations of given functions This is not, at least primarily, a tool for data analysis, but for solving eigenfunction problems and such like using computational Hilbert space methods for functions which are otherwise difficult. In particular, we must be able to evaluate the target functions at arbitrary points to construct the interpolant, rather than at, say, provided sample points. Useful companion package FastTransforms converts between different basis function representaitons. Interpolations.jl does arbitrary order spline interpolations of mathematical functions, but also data. This enables some very clever tricks, e.g. random sampling of tricky distributions.
Doing heavy calculations? Want to save time? Caching notionally helps, but of course is a quagmire of bikeshedding.
The Anamemsis creator(s) have opinion about this:
Julia is a scientific programming language which was designed from the ground up with computational efficiency as one of its goals. As such, even ad hoc and “naive” functions written in Julia tend to be extremely efficient. Furthermore, any memoization implementation is liable to have some performance pain points: in general they contain type unstable code (even if they have type stable output) and they must include a value cache which is accessible from the same scope as the function itself so that global objects are potentially involved. Because of this, memoization comes with a significant overhead, even with Julia’s highly efficient AbstractDict implementations.
My own opinion of caching is that I’m for it. Despite the popularity of the “memoize” pattern I’m against that as a rule. I usually end up having to think carefully about caching, and magically making functions cache never seems to help with the hard part of that. Caching logic should, for projects I’ve worked on at least, be explicit, opt-in, and that should be an acceptable design overhead because I should only be doing it for a few expensive functions, rather than creating magic cache functions everywhere.
LRUCache is the most commonly recommended cache. It provides simple, explicit, opt-in, in-memory cache structure. I am unclear how this would perform with multiple processes or threads.
Memoize provides an automatic caching macro but primitive in-memory caching using plain old dictionaries. AFAICT it is possibly compatible with LRUCache, but weirdly I could not find discussion of that. However, I don’t like memoisation, so that’s the end of that story.
Anamemsis has advanced cache decorators and lots of levers to pull to alter caching behaviour.
Caching is especially full-featured with macros, disk persistence and configurable evictions etc. It wants type annotations to be used in the function definitions.
Cleverman Simon Byrne notes that you can roll your own fancy caching using wild Julia introspection stunts via Cassette.jl, but I don’t know how battle-tested this is. Possibly this is the correct amount of magic?
People mention that Julia can handle physical units in a syntactically pleasant fashion. It’s not clear how, since all the documentation assumes you already know. From what I can tell, you first need to actually install Unitful, which includes some useful units, and in particular Quantity types.
Then you can use units in the following fashion:
If you want to know how this works, and also how you can invent your own units, read Erik Engheim who explains it in depth.
See SampledSignals for an example of how you do things with units, such as the following method definitions to handle time-to-index conversion.
UIs and servers
(If you really wanted to have lots of control and a proper web framework, you could use the web framework Genie.jl.)
You can plug into classic C++ immediate mode Dear Imgui via CImGui.jl.
Signals, reactive programming
Reactive.jl is widely used to manage signals in UI toolkits. It has a competitor Signals.jl, whose creator describes the differences:
Signals.jl, while offering the same functionality as Reactive, is different on some key factors
Dynamic: Signals are not typed, you can push an integer then float64 and then a string and it blends nicely with Julia’s Multiple Dispatch
Push-Pull: you can either push a value into a Signal and propagate changes along the Signal graph , or you can set a value without any propagation and only pull the necessary changes from any other signal.
Syntax: Syntax is somewhat simplified, square brackets to set or query a value, round brackets to pull or push a value (see documentation for more examples)
The last point is, IMO, a minus, but the second one is a plus.
QML.jl magically plugs into Qt5. Gtk.jl plugs into GTK.