Lorenzo Villani

(╯°□°)╯︵ ┻━┻

A Gentle Introduction To Docker

Jan 6, 2016

Brief History of Containers

In recent times there has been a lot of talk about containers being the definitive solution to simplify application deployments, especially thanks to all the hype around Docker which has become synonym with “container”.

We may be under the impression that containers are a new technology, but we are far from the truth.

We can see containers like the natural evolution of “chroots” and FreeBSD “jails” readily available, respectively, from the ’80s and 2000. On Linux we find that precursors to Docker were already in use since 2005, thanks to commercial solutions like Virtuozzo (which was later open sourced under the OpenVZ brand). An honorable mention goes to LXC which appeared in 2008 and was the foundation for Docker until 2014.

Containers Versus Virtual Machines

What are the main differences between a container and a virtual machine? Why should we choose one or the other?

A “classic” solution like VMware, HyperV or qemu/KVM assumes the creation of a complete virtual machine that simulates a CPU with all its peripherals, disks and network cards. This allows us, for example, to let Windows run on a Linux server. This is clearly heavyweight, even though it can be accelerated by support provided by the most modern CPUs.

Containers instead leverage support offered directly by the operating system’s kernel that allows partitioning and segregation of process groups. Since this mechanism works at the syscall level, without emulation, it has little to no overhead and there are no virtual machines involved. This allows us to greatly increment the “population density” of a server.

Naturally, each solution has up- and downsides. A standard hypervisor gives us maximum flexibility to choose the runtime environment: we can choose the operating system, run custom kernels and manage resources as if we are working on real PC. A container can run applications written for the same operating system and architecture the host is using and doesn’t allow us to run a custom kernel, but in change it gives us finer control over resources.

We also have to consider the security aspects of both solutions: virtual machines offer stronger insulation between each other when compared to containers even though the latter are rapidly improving on this front.

The Rise Of Docker

In 2013 Docker became the synonym with container, thanks to features that made it easier to use than lower-level solutions like LXC and a few technical innovations.

Docker is founded on the concept of “layers”: immutable file-system images that can be stacked on top of one another and can be instanced into containers. The stacking allows developers to compose the final image up of multiple parts and save space, since layers common to many images are shared between them.

Creation of an image happens through a so-called “Dockerfile”, a recipe that contains all steps needed to build a certain image and allows others to reproduce it (other than being a form of “executable documentation”). In the following example you can see how easy it is to create an image to run a simple web application written in Node.JS:

FROM ubuntu:14.04
ENV DEBIAN_FRONTEND noninteractive
RUN sed 's/archive./it.archive./g' -i /etc/apt/sources.list
RUN apt-get update && apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash -
RUN apt-get install -y nodejs
RUN mkdir /var/www
ADD app.js /var/www/app.js
EXPOSE 8080
CMD ["/usr/bin/node", "/var/www/app.js"]

Once we have created our image, we can launch one, ten or a thousand times by running:

docker run -P nodejs-example

But let’s remember that images are immutable: containers will use it as a base and changes persist only as long as the container is still alive. Once we destroy it, its changes will be lost. As we expect, Docker allows us to “mount” a non-volatile directory within a container thanks to “data volumes”.

Once we have created an image, we can publish it to a central registry called Docker Hub to share it with the community, thus simplifying the deployment process, since Docker will automatically try to pull an image from the registry if it can’t find it locally. It is also possible to run a private registry or export and subsequently import images transferred via USB drives, if you are so inclined.

All of this greatly simplifies the process of installing an application. To make an example: a typical installation of Mattermost requires the separate configuration of PostgreSQL and Nginx, in addition to Mattermost itself, a process that can take at least half an hour for an expert system administrator.

Thanks to Docker, though, I can run:

docker run --name mattermost-dev -d --publish 8065:80 mattermost/platform

And then, two minutes later, I can point my browser to port 8065 to complete the configuration process.

The developer didn’t have to make packages for each and every Linux distribution and I didn’t have to become crazy configuring all the moving parts. This is a simple example, though. A complex application might require more than configuring just a database and a web server.

Docker as the alternative to package managers

Since images are really easy to create and they contain all the libraries and dependencies that an application needs, it’s easy to think of Docker as an alternative to a package manager, at least for web applications.

Docker allows us to not bother which version of Ubuntu, Python or Apache the user is running. A developer can start off with the base image of its favorite distribution, bundle all files together and make it available through Docker Hub. Said image can be launched as easily on CentOS, Fedora, SUSE, Arch Linux or whatever Linux distribution is trendy these days.

It’s not a coincidence that the Open Container Initiative was recently formed to standardize a packaging format that is compatible across all containerization solutions.

Summing up

In the end Docker and other containerization technologies bring some advantages that we should not overlook:

  • A better experience for developers that don’t have to target a specific Linux distribution anymore.
  • A better experience for users that can run an application on the Linux distribution of their choice.
  • Reduced overhead when compared to classic virtualization technologies;
  • Reproducibility;
  • Slightly improved security and segregation of applications;

Amazon Fire 5th Gen: Road To A Pure Google Experience

Dec 7, 2015

Update: The method described in this article is no longer valid due to recent bootloader updates. I have since removed my code from GitHub. If you have already updated FireOS to the latest version you won’t be able to root it as easily as before. If you have an older FireOS version, your best bet is to install a Cyanogenmod port.


You know that times have changed when you can get an Android tablet that actually works for just 50€. When Amazon launched the 5th generation Fire Tablet I knew that I had to get one.

Tablets are great consuming devices and this was the right time to get rid of a bulky iPad Air, get my Nexus 7 2013 back that I lent to a family member and replace it with a modest tablet that covers the basics and works well enough for her use case.

Me and a bunch of co-workers took the plunge and bought into Amazon’s crazy “get six tablets for 50€ each” discount pack. There you have it: the perfect recipe for a weekend of tinkering.

When I turned it on I was amazed that, while it was definitely not a premium device, the hardware actually felt solid in the hand.

Design-wise and spec-wise it reminded me of a 2012 Nexus 7 made slightly faster by updated hardware and not penalized by the slow-as-molasses eMMC that plagued Asus’ tablets nearly four years ago. There are some differences made in name of price, such as a lower 1024x600 screen instead of a 1280x800 one, but hey, for 50€ what did I expect anyway?

One huge drawback of all Fire devices in general is that they run Amazon’s fork of Android: Fire OS. Being a fork designed to be incompatible with Android compatibility standards, Amazon doesn’t include the Play Store, Play Services or any Google application. They are contractually forbidden from doing so. This means no Gmail, YouTube, Maps, and no Chromecast support. 1

I was faced with a dilemma: how can I get this incredibly cheap piece of hardware to run the incredibly good Google apps?

After a short trip to XDA-Developers’ forums I had the answer: I had to gain root on the device. Thankfully I didn’t have to find an exploit to escalate privileges.

In fact, the device comes with signed boot and recovery partitions, but the boot loader still allows you to upload a recovery image to the device’s RAM and run it from there. Which means that once you get a compatible recovery image you can then mount the system partition in read-write mode and do as you please.

Most automated scripts on XDA-Developers, however, are targeted at Windows users and come with sketchy ADB drivers and unknown binaries. While I have a Windows machine, I didn’t want to let that stuff touch my computer.

Since I’ve been completely out of the Android scene for the last two years, I had to update my knowledge on custom ROMs, recoveries, etc. Then I started to piece out a simple shell script that downloads everything needed from their original places and applies that to the device.

The end result is available at <https://github.com/lvillani/amazon-fire-tablet-pge> with some instructions to get you started. I've developed and tested the script only on OS X El Capitan but it should work also on the latest Ubuntu LTS and maybe some other Linux distribution.
  1. In fact, Amazon is not a member of the Open Handset Alliance and is contractually forbidden from adding Google stuff to its own distribution. OHA members are forbidden from selling devices with incompatible Android forks if they want to ship the Play Store or any Google application to their customer. It’s an all-or-nothing proposition. One that conditions which hardware manufacturers Amazon can choose to build its tablets. In this case they had to go to Quanta Computer, since they are not an OHA member. 

3-2-1, It's Time to Backup

Nov 4, 2015

November is the right time to define your New Year’s resolutions. There is nothing better than gifting you and your relatives an effective, yet simple, backup strategy. Start now before it’s too late. There’s nothing worse than waking up on the 1st of January and learning with horror that all of your data has vanished into thin air.

The simplest and most effective guideline to keep in mind is the so called “3-2-1 Backup Rule”:

  • At least three copies of every file you care about;
  • In two different formats;
  • With one of them off-site.

Keeping three copies of a files means keeping them in different places. So, mashing CTRL+C and CTRL+V on your computer keyboard does not count as a backup. Likewise, keeping your files in a secondary drive permanently attached to your computer doesn’t qualify as backup, either. You would still loose it all should someone steal your machine. The file you have on your work computer, however, counts as the first copy. What I call the “master copy”.

But why keep different copies? Why three? The answer to the first question is redundancy: you don’t want a single event to wipe out all of your memories and work files. The answer to the latter is: the more copies you have, the merrier. Three strikes a balance between having too few copies and being hard to manage.

You will also want to keep your backup in two different formats so that you are not subject to weaknesses of either of them. For example, magnetic hard drives, leave your data exposed to EMPs and mechanical deficiencies in the device itself. A good alternative is to leverage the SSD in your computer, a magnetic hard drive and a cloud storage service like Amazon Glacier or Google Nearline.

Last but not least, having one off-site backup will keep you safe in the event your house burns down or gets flooded by an hurricane.

Following the 3-2-1 rule will get you nowhere unless you also implement a good backup strategy. While the backup rule described before easily applies to computers, mobile devices and stuff stored in cloud services is entirely another matter.

There is a widespread misconception that data in the cloud is “safe” and that we don’t need to care for it or back it up explicitly, since the provider will usually have a backup system already in place. While that’s true, it’s designed to protect the provider from catastrophes on its own premises (e.g.: loosing a server, lightning strikes, etc). It won’t protect you from accidentally deleting stuff, loosing access keys or being denied access to the service. Having a safety net in this case especially important with companies like Google that have non-existing customer support.

One way you can balance that is to siphon all cloud data back to your computer so that it gets automatically propagated to all your other backup media. I do that with Gmail, Trello, RunKeeper, etc. I also pull my Mom’s emails and pictures to my computer so that they get backup up daily.

Diagram

The other important thing is that all data is encrypted at rest. All destinations only receive opaque blobs of data that can’t be opened without a decryption key. This makes restores a bit more cumbersome, since they involve a specialized tool that may not be readily available, but it greatly simplifies off-site backups since I can just sync the encrypted data without having to worry about third parties peeking into my data.

Further Reading