These are rapidly evolving standards. Check the timestamps on any advice.
The other, lighter, hipper alternative to virtual machines, which, AFAICT, attempts to make provisioning more like installing an app than building a computer, because it aims to containerise apps rather than OSes, which emphasis leads to less dicking around, but somehow even more webinars.
There are two ideas I ahve kind of confounded here
- packaging things up neatly
- sandboxing them in their own little environments
I should separate them.
The most common apps this is down for are, AFAICT, Linux-ish, but you cn shorehorn those into Windos/macOS, and probalby equivalents exist for windows/macOs.
The most common way of doing this.
Linux hosts: installing docker
macOS has a confusing profusion of toolchain bits and pieces they can try to install to get the experience, all of which try to install various distinct versions of each other, and give little information about which is the recommended way of doing what.
kitematic provides a GUI for the containers themselves, as opposed to the infrastructure.
docker toolbox bundles some docker infrastructure plus kitematic. It attempts to run docker properly, but seems to fail in weird ways in the default setup, giving, e.g. permission errors and such. If you install Docker for Mac then install this, you get Kitematic but it can’t see your docker images, because of something boring that I can’t be bothered understanding.
Docker for Windows. (i.e. runs Windows clients
GPU-happy docker management
Handling passwords is fiddly – see secrets.
Do you get the following error?
According to thaJeztah, the solution is to use google DNS for docker. (or presumably someothernon-awful DNS) You can set this by providing a JSON configuration in the preference panel (under daemon -> advanced), e.g
Docker for reproducible research
Docker may not be the ultimate tool for reproducible research but it is a start. (TODO: fact-check the linked article.)
…How do you get your data in?
Tiffany Timbers gives a brisk run-through for academics.
Jon Zelner goes in-depth with R in a series culminating in continuous integration for science.
Reproducible research tuts has a docker (plus also VM-backed) tutorial.
Singularity promises potentially useful container infrastructure.
Singularity provides a single universal on-ramp from the laptop, to HPC, to cloud.
USERS OF SINGULARITY CAN BUILD APPLICATIONS ON THEIR DESKTOPS AND RUN HUNDREDS OR THOUSANDS OF INSTANCES—WITHOUT CHANGE—ON ANY PUBLIC CLOUD.
- Support for data-intensive workloads—The elegance of Singularity’s architecture bridges the gap between HPC and AI, deep learning/machine learning, and predictive analytics.
- A secure, single-file-based container format—Cryptographic signatures ensure trusted, reproducible, and validated software environments during runtime and at rest.
- Extreme mobility—Use standard file and object copy tools to transport, share, or distribute a Singularity container. Any endpoint with Singularity installed can run the container.
- Compatibility—Designed to support complex architectures and workflows, Singularity is easily adaptable to almost any environment.
- Simplicity—If you can use Linux®, you can use Singularity.
- Security—Singularity blocks privilege escalation inside containers by using an immutable single-file container format that can be cryptographically signed and verified.
- User groups—Join the knowledgeable communities via GitHub, Google Groups, or in the Slack community channel.
- Enterprise-grade features—Leverage SingularityPRO’s Container Library, Remote Builder, and expanded ecosystem of resources.[…]
Released in 2016, Singularity is an open source-based container platform designed for scientific and high-performance computing (HPC) environments. Used by more than 25,000 top academic, government, and enterprise users, Singularity is installed on more than 3 million cores and trusted to run over a million jobs each day.
In addition to enabling greater control over the IT environment, Singularity also supports Bring Your Own Environment (BYOE)—where entire Singularity environments can be transported between computational resources (e.g., users’ PCs) with reproducibility.
- kitematic, already mentioned, is languishing but works. Windows, macOS
- portainer is a docker gui that runs on docker, and therefor everywhere.
LXC is another containerization standard. Because docker is a de facto default, let’s look at this in terms of docker
archlinux says >Firejail is an easy to use SUID sandbox >program that reduces the risk of security breaches by restricting the running >environment of untrusted applications using Linux namespaces, seccomp-bpf and >Linux capabilities.
nachoparker’s intro is comprehensible.
Abstracted from the app standard Flatpak is bubblewrap.
Many container runtime tools like systemd-nspawn, docker, etc. focus on providing infrastructure for system administrators and orchestration tools (e.g. Kubernetes) to run containers.
These tools are not suitable to give to unprivileged users, because it is trivial to turn such access into to a fully privileged root shell on the host.
Rumour is it is more secure than Firejail because it doesn’t support all the many things firejail does (but this means that eg it doesn’t support audio)