Convert Any Server to a Docker Container (Updated)

How and Why?

Let’s say you have a server that has been lovingly hand-crafted that you want to containerize.

Figuring out exactly what software is required on there and what config files need adjustment would be quite a task, but fortunately blueprint exists as a solution to that.

What I’ve done here is automate that process down to a few simple steps. Here’s how it works:

Blueprint_Server

You kick off a ShutIt script (as root) that automates the bash interactions required to get a blueprint copy of your server, then this in turn kicks off another ShutIt script which creates a Docker container that provisions the container with the right stuff, then commits it. Got it? Don’t worry, it’s automated and only a few lines of bash.

There are therefore 3 main steps to getting into your container:

– Install ShutIt on the server

– Run the ‘copyserver’ ShutIt script

– Run your copyserver Docker image as a container

Step 1

Install ShutIt as root:

sudo su -
pip install shutit

The pre-requisites are python-pip, git and docker. The exact names of these in your package manager may vary slightly (eg docker-io or docker.io) depending on your distro.

You may need to make sure the docker server is running too, eg with ‘systemctl start docker’ or ‘service docker start’.

Step 2

Check out the copyserver script:

git clone https://github.com/ianmiell/shutit_copyserver.git

Step 3

Run the copy_server script:

cd shutit_copyserver/bin
./copy_server.sh

There is a prompt to ask what docker base image you want to use. Make sure you use one as close to the original server as possible.

Step 4

Run the built image:

docker run -ti [username]/copyserver /bin/bash

You are now in a practical facsimile of your server within a docker container!

This post is based on material from Docker in Practice, available on Manning’s Early Access Program. Get 39% off with the code: 39miell2

dip

Advertisements

17 thoughts on “Convert Any Server to a Docker Container (Updated)”

  1. I tried it with centos 7 but got this error:

    [root@ip-10-66-118-190 centos]# cd shutit_copyserver/bin
    [root@ip-10-66-118-190 bin]# ./copy_server.sh
    /home/centos/shutit_copyserver /home/centos/shutit_copyserver/bin
    # ShutIt Started…
    # Loading configs…
    2016-08-13 14:34:12,551 INFO: ShutIt modules loaded
    2016-08-13 14:34:12,552 INFO: Configuration loaded
    2016-08-13 14:34:12,618 INFO: Connected to target

    Please input a value for shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver.base_image

    Please input an appropriate docker base image, eg:
    ubuntu
    ubuntu:12.04
    centos

    You can use any base image you want, but be aware that non-apt or yum-based images are less likely to succeed.:
    centos
    2016-08-13 14:34:15,007 INFO: Considering whether to build: shutit.tk.setup
    2016-08-13 14:34:15,087 INFO: Building ShutIt module: shutit.tk.setup with run order: 0
    2016-08-13 14:34:15,090 INFO: Considering whether to build: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver
    2016-08-13 14:34:15,177 INFO: Building ShutIt module: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver with run order: 1161086295
    2016-08-13 14:34:15,177 INFO: Matching output from: “whoami” to one of these regexps:root
    2016-08-13 14:34:15,179 INFO: Installing package: python-pip
    2016-08-13 14:34:15,181 INFO: Sending: yum install -y python-pip
    2016-08-13 14:34:18,776 INFO: Output (squashed): Loaded plugins: fastestmirrorL […]
    2016-08-13 14:34:18,777 INFO: Installing package: git
    2016-08-13 14:34:18,779 INFO: Sending: yum install -y git
    2016-08-13 14:34:22,255 INFO: Output (squashed): Loaded plugins: fastestmirrorL […]
    2016-08-13 14:34:22,256 INFO: Matching output from: “git config -l” to one of these regexps:user.email
    2016-08-13 14:34:22,259 INFO: Matching output from: “git config -l” to one of these regexps:user.name
    2016-08-13 14:34:22,261 INFO: Sending: rm -rf /tmp/shutit_copyserver
    2016-08-13 14:34:22,263 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
    2016-08-13 14:34:22,264 INFO: Sending: pip install blueprint
    2016-08-13 14:34:22,539 INFO: Output (squashed): Requirement already satisfied […]
    2016-08-13 14:34:22,540 INFO: Sending: echo ‘:package:apt/docker’ >> /etc/blueprintignore
    2016-08-13 14:34:22,542 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
    2016-08-13 14:34:22,543 INFO: Sending: echo ‘:package:apt/docker.io’ >> /etc/blueprintignore
    2016-08-13 14:34:22,544 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
    2016-08-13 14:34:22,545 INFO: Sending: echo ‘:package:apt/docker-engine’ >> /etc/blueprintignore
    2016-08-13 14:34:22,547 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
    2016-08-13 14:34:22,548 INFO: Sending: echo ‘:package:apt/virtualbox’ >> /etc/blueprintignore
    2016-08-13 14:34:22,550 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
    2016-08-13 14:34:22,551 INFO: Sending: echo ‘:package:apt/virtualbox-dkms’ >> /etc/blueprintignore
    2016-08-13 14:34:22,553 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
    2016-08-13 14:34:22,554 INFO: Sending: echo ‘:package:apt/virtualbox-qt’ >> /etc/blueprintignore
    2016-08-13 14:34:22,556 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
    Traceback (most recent call last):
    File “/bin/shutit”, line 11, in
    sys.exit(main())
    File “/usr/lib/python2.7/site-packages/shutit_main.py”, line 609, in main
    do_build()
    File “/usr/lib/python2.7/site-packages/shutit_main.py”, line 421, in do_build
    build_module(module)
    File “/usr/lib/python2.7/site-packages/shutit_main.py”, line 362, in build_module
    if not module.build(shutit):
    File “/usr/lib/python2.7/site-packages/shutit_module.py”, line 58, in wrapper
    ret = func(self, shutit)
    File “/home/centos/shutit_copyserver/shutit_copyserver.py”, line 80, in build
    shutit.send(shutit.cfg[‘host’][‘shutit_path’] + ‘/shutit skeleton –output_dir –base_image ‘ + shutit.cfg[self.module_id][‘base_image’] + ‘ –template_branch docker –module_directory /tmp/shutit_copyserver –module_name copied_server –domain shutit.copied_server –delivery docker’)
    KeyError: ‘host’
    /home/centos/shutit_copyserver/bin
    [root@ip-10-66-118-190 bin]#

    1. I pulled the update and retried again, but got this new error, please check:

      [root@ip-10-66-115-25 centos]# cd shutit_copyserver/bin
      [root@ip-10-66-115-25 bin]# ./copy_server.sh
      /home/centos/shutit_copyserver /home/centos/shutit_copyserver/bin
      # ShutIt Started…
      # Loading configs…
      2016-08-13 16:07:36,712 INFO: Correcting insecure file permissions on: configs/build.cnf
      2016-08-13 16:07:36,721 INFO: ShutIt modules loaded
      2016-08-13 16:07:36,722 INFO: Configuration loaded
      2016-08-13 16:07:36,787 INFO: Connected to target

      Please input a value for shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver.base_image

      Please input an appropriate docker base image, eg:
      ubuntu
      ubuntu:12.04
      centos

      You can use any base image you want, but be aware that non-apt or yum-based images are less likely to succeed.:
      centos
      2016-08-13 16:07:39,115 INFO: Considering whether to build: shutit.tk.setup
      2016-08-13 16:07:39,196 INFO: Building ShutIt module: shutit.tk.setup with run order: 0
      2016-08-13 16:07:39,198 INFO: Considering whether to build: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver
      2016-08-13 16:07:39,284 INFO: Building ShutIt module: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver with run order: 1161086295
      2016-08-13 16:07:39,284 INFO: Matching output from: “whoami” to one of these regexps:root
      2016-08-13 16:07:39,288 INFO: Installing package: python-pip
      2016-08-13 16:07:39,289 INFO: Sending: yum install -y python-pip
      2016-08-13 16:07:42,803 INFO: Output (squashed): Loaded plugins: fastestmirrorL […]
      2016-08-13 16:07:42,804 INFO: Installing package: git
      2016-08-13 16:07:42,806 INFO: Sending: yum install -y git
      2016-08-13 16:07:46,258 INFO: Output (squashed): Loaded plugins: fastestmirrorL […]
      2016-08-13 16:07:46,259 INFO: Matching output from: “git config -l” to one of these regexps:user.email
      2016-08-13 16:07:46,261 INFO: Sending: git config –global user.email “shutit@shutit.tk”
      2016-08-13 16:07:46,264 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:46,265 INFO: Matching output from: “git config -l” to one of these regexps:user.name
      2016-08-13 16:07:46,267 INFO: Sending: git config –global user.name “ShutIt”
      2016-08-13 16:07:46,270 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:46,271 INFO: Sending: rm -rf /tmp/shutit_copyserver
      2016-08-13 16:07:46,273 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:46,274 INFO: Sending: pip install blueprint
      2016-08-13 16:07:47,045 INFO: Output (squashed): Collecting blueprint Download […]
      2016-08-13 16:07:47,046 INFO: Sending: echo ‘:package:apt/docker’ >> /etc/blueprintignore
      2016-08-13 16:07:47,048 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:47,049 INFO: Sending: echo ‘:package:apt/docker.io’ >> /etc/blueprintignore
      2016-08-13 16:07:47,050 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:47,051 INFO: Sending: echo ‘:package:apt/docker-engine’ >> /etc/blueprintignore
      2016-08-13 16:07:47,053 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:47,054 INFO: Sending: echo ‘:package:apt/virtualbox’ >> /etc/blueprintignore
      2016-08-13 16:07:47,056 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:47,057 INFO: Sending: echo ‘:package:apt/virtualbox-dkms’ >> /etc/blueprintignore
      2016-08-13 16:07:47,059 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:47,060 INFO: Sending: echo ‘:package:apt/virtualbox-qt’ >> /etc/blueprintignore
      2016-08-13 16:07:47,062 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:47,063 INFO: Sending: shutit skeleton –output_dir –base_image centos –pattern docker –name /tmp/shutit_copyserver –domain shutit.copied_server –delivery docker
      2016-08-13 16:07:47,263 INFO: Output (squashed): # ShutIt Started… # […]
      2016-08-13 16:07:47,264 INFO: Sending: cd /tmp/shutit_copyserver
      2016-08-13 16:07:47,266 INFO: Output (squashed): shutit.shutit_copyserver.shuti […]
      2016-08-13 16:07:47,267 INFO: Sending: blueprint create shutit_copyserver
      2016-08-13 16:08:06,971 INFO: Output (squashed): # [blueprint] searching for AP […]
      2016-08-13 16:08:06,974 INFO: Fixing up your terminal, please wait…

      ================================================================================

      WARNING: command:
      blueprint create shutit_copyserver
      returned unaccepted exit code: 1
      If this is expected, pass in check_exit=False or an exit_values array into the send function call.

      Interactive, so not retrying.
      Pause point on exit_code != 0 (1). CTRL-C to quit
      ================================================================================

      You now have a standard shell. Hit CTRL and then ] at the same time to continue ShutIt run.
      /tmp/shutit_fixterm
      shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver#cHAQmuDJ>

  2. Hi! I have tried to create the copy of my server (Ubuntu 16.04.2) but the script fails with the following output. Can you please help me?

    root@sputnik:~/shutit_copyserver/bin# ./copy_server.sh
    ~/shutit_copyserver ~/shutit_copyserver/bin
    # ShutIt Started…
    # Loading configs…
    2017-05-03 16:38:59,270 INFO: Correcting insecure file permissions on: configs/build.cnf
    2017-05-03 16:38:59,282 INFO: ShutIt modules loaded
    2017-05-03 16:38:59,283 INFO: Configuration loaded
    2017-05-03 16:39:01,530 INFO: Connected to target

    Please input a value for shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver.base_image

    Please input an appropriate docker base image, eg:
    ubuntu
    ubuntu:12.04
    centos

    You can use any base image you want, but be aware that non-apt or yum-based images are less likely to succeed.:
    ubuntu
    2017-05-03 16:39:16,880 INFO: Considering whether to build: shutit.tk.setup
    2017-05-03 16:39:18,218 INFO: Building ShutIt module: shutit.tk.setup with run order: 0
    2017-05-03 16:39:18,372 INFO: Considering whether to build: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver
    2017-05-03 16:39:19,740 INFO: Building ShutIt module: shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver with run order: 1161086295
    2017-05-03 16:39:19,740 INFO: Matching output from: “whoami” to one of these regexps:root
    2017-05-03 16:39:19,793 INFO: Installing package: python-pip
    2017-05-03 16:39:19,896 INFO: python-pip already installed.
    2017-05-03 16:39:19,896 INFO: Installing package: git
    2017-05-03 16:39:19,995 INFO: git already installed.
    2017-05-03 16:39:19,996 INFO: Matching output from: “git config -l” to one of these regexps:user.email
    2017-05-03 16:39:20,049 INFO: Matching output from: “git config -l” to one of these regexps:user.name
    2017-05-03 16:39:20,103 INFO: Sending: rm -rf /tmp/shutit_copyserver
    2017-05-03 16:39:20,186 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:39:20,238 INFO: Sending: pip install blueprint
    2017-05-03 16:39:20,784 INFO: Output (squashed): Requirement already satisfied: blueprint in /usr/local/lib/python2.7/dist-packagessputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K# […]
    2017-05-03 16:39:20,836 INFO: Sending: echo ‘:package:apt/docker’ >> /etc/blueprintignore
    2017-05-03 16:39:20,890 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:39:20,942 INFO: Sending: echo ‘:package:apt/docker.io’ >> /etc/blueprintignore
    2017-05-03 16:39:20,996 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:39:21,048 INFO: Sending: echo ‘:package:apt/docker-engine’ >> /etc/blueprintignore
    2017-05-03 16:39:21,103 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:39:21,155 INFO: Sending: echo ‘:package:apt/virtualbox’ >> /etc/blueprintignore
    2017-05-03 16:39:21,209 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:39:21,261 INFO: Sending: echo ‘:package:apt/virtualbox-dkms’ >> /etc/blueprintignore
    2017-05-03 16:39:21,316 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:39:21,368 INFO: Sending: echo ‘:package:apt/virtualbox-qt’ >> /etc/blueprintignore
    2017-05-03 16:39:21,422 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:39:21,474 INFO: Sending: shutit skeleton –output_dir –base_image ubuntu –pattern docker –name /tmp/shutit_copyserver –domain shutit.copied_server –delivery docker
    2017-05-03 16:39:21,768 INFO: Output (squashed): # ShutIt Started… # Loading configs…# Run:cd /tmp/shutit_copyserver && ./build.sh# to build.# And then:./run.sh# to run.sputnik:shutit.shutit_copyserver.shu […]
    2017-05-03 16:39:21,820 INFO: Sending: cd /tmp/shutit_copyserver
    2017-05-03 16:39:21,872 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:39:21,924 INFO: Sending: blueprint create shutit_copyserver
    2017-05-03 16:41:17,936 INFO: Output (squashed): # [blueprint] parsing blueprintignore(5) rulesdpkg-query: Kein Paket gefunden, das auf python3:any passtdpkg-query: Kein Paket gefunden, das auf docker-engine p […]
    2017-05-03 16:41:17,987 INFO: Sending: blueprint show -S shutit_copyserver
    2017-05-03 16:41:19,067 INFO: Output (squashed): shutit_copyserver/bootstrap.shsputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:41:19,118 INFO: Sending: sed -i ‘s/^…n “$SERVICE_.*$//g’ shutit_copyserver/bootstrap.sh
    2017-05-03 16:41:19,212 INFO: Output (squashed): sputnik:shutit.shutit_copyserver.shutit_copyserver.shutit_copyserver:Koy9EA0K#
    2017-05-03 16:41:26,349 INFO: ShutIt run finished

  3. Just tried to run through this on one of our ec2 instances and I ran into the following problem. Any ideas?

    Thanks in advance!

    [root@ip-172-31-8-239 bin]# ./copy_server.sh
    ~/shutit_copyserver ~/shutit_copyserver/bin
    /usr/lib/python2.6/dist-packages/shutit_pexpect.py:46: DeprecationWarning: the md5 module is deprecated; use hashlib instead
    from md5 import md5
    Traceback (most recent call last):
    File “/usr/bin/shutit”, line 11, in
    sys.exit(main())
    File “/usr/lib/python2.6/dist-packages/shutit.py”, line 50, in main
    if sys.version_info.major == 2:
    AttributeError: ‘tuple’ object has no attribute ‘major’
    ~/shutit_copyserver/bin

  4. Hi zwischenzugs,

    I am trying to run ./copy_server.sh. But getting the below error. I am running it in an amazonlinux system (yum). Is there an issue due to this OS? Please help.

    ShutIt version: 1.0.27
    Linux ip-10-0-2-39 3.14.42-31.38.amzn1.x86_64 #1 SMP Wed May 13 20:33:05 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

    2017-10-04 11:50:18,809 INFO: Installing package: python-pip
    Traceback (most recent call last):
    File “/usr/local/bin/shutit”, line 11, in
    sys.exit(main())
    File “/usr/local/lib/python2.7/site-packages/shutit.py”, line 67, in main
    shutit.setup_shutit_obj()
    File “/usr/local/lib/python2.7/site-packages/shutit_class.py”, line 3942, in setup_shutit_obj
    self.do_build()
    File “/usr/local/lib/python2.7/site-packages/shutit_class.py”, line 4181, in do_build
    self.build_module(module)
    File “/usr/local/lib/python2.7/site-packages/shutit_class.py”, line 4123, in build_module
    if not module.build(self):
    File “/usr/local/lib/python2.7/site-packages/shutit_module.py”, line 59, in wrapper
    ret = func(self, shutit)
    File “/opt/shutit_copyserver/shutit_copyserver.py”, line 66, in build
    shutit.install(‘python-pip’)
    File “/usr/local/lib/python2.7/site-packages/shutit_class.py”, line 1798, in install
    loglevel=loglevel)
    File “/usr/local/lib/python2.7/site-packages/shutit_pexpect.py”, line 1393, in install
    package = package_map.map_packages(package, self.current_environment.install_type)
    File “/usr/local/lib/python2.7/site-packages/package_map.py”, line 108, in map_packages
    res = res + ‘ ‘ + map_package(package,install_type)
    TypeError: cannot concatenate ‘str’ and ‘function’ objects

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s