Running analysis in docker container

There are generally 4 steps to create an analytics environment on your server that will be separate from the rest of the system. By running your analysis within a container, it might reduce the risk of crashing the server because you might have used up all the resouce and cause the server to freeze up.

  1. Install virtualbox (to create a docker machine)

  2. Install docker-machine

  3. Create a docker-machine this will be the machine to run your container

  4. Map your command to run in the container. Following the kaggle/python tutorial.

Install virtualbox-qt

sudo apt-get install virtualbox-qt

Install docker-machine

curl -L`uname -s-uname -m` > /usr/local/bin/docker-machine && \ chmod +x /usr/local/bin/docker-machine

create a docker-machine

This will create a separate docker machine called docker2

docker-machine create -d virtualbox --virtualbox-disk-size "100000" --virtualbox-cpu-count "8" --virtualbox-memory "32092" docker2
docker-machine start docker2

We then need to specify a new destination where docker container will run, i.e. on docker2

eval $(docker-machine env docker2)

See <; for more info

run kaggle/python

You’re now at a point where you can run stuff in the container. Here’s an extra step that will make it super easy: put these lines in your .bashrc file (or the Windows equivalent)

docker run -v $PWD:/tmp/working -w=/tmp/working --rm -it kaggle/python python "$@"
ikpython() {
docker run -v $PWD:/tmp/working -w=/tmp/working --rm -it kaggle/python ipython
kjupyter() {
(sleep 3 && open "http://$(docker-machine ip docker2):8888")&
docker run -v $PWD:/tmp/working -w=/tmp/working -p 8888:8888 --rm -it kaggle/python jupyter notebook --no-browser --ip="*" --notebook-dir=/tmp/working



Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.