Container exercises

Create and start a new CentOS 7 container running ping to 8.8.8.8. Docker will downlaod the CentOS 7 image since you do not have it available locally.

Press CTRL+C after a few pings. This stops and exits the container.

This first container sent its STDOUT to your terminal. Create a second container, this time in detached mode:

Instead of seeing the executed command (ping 8.8.4.4), Docker engine displays a long hexidecimal number, which is the full container ID of your new container. The container is running detached, which means the container is running as a background process, rather than printing its STDOUT to your terminal.

List the running Docker containers using the docker container ls container command. You will see only one container running.

Now you know that the docker container ls command only shows running containers. You can show all containers that exist (running or stopped) by using docker container ls --all. Your container ID and name will vary. Note that you will see two containers: a stopped container and a running container.

Where did those names come from? All containers have names, which in most Docker CLI commands can be substituted for the container ID as we’ll see in later exercises. By default, containers get a randomly generated name of the form <adjective>_<scientist / technologist>, but you can choose a name explicitly with the --name flag in docker container run.

Start up another detached container, this time giving it a name "opendnsping".

List all your containers again. You can see all of the containers, including your new one with your customized name.

Next, remove the exited container. To do this, use docker container rm <container ID>. In the example above, the Docker container ID is 00f763b9308d. You only need as many characters as to uniquely identify this container from all the others.

Now try to remove one of the other Docker containers using the same command. It does not work. Why?

You can see that running containers are not removed. You’ll have to look for an option to remove a running container. In order to find out the option you need to do a force remove, check the command line help. To do this with the docker container rm command, use the --help option:

Now, run a force remove on the running container you tried to remove in the two previous steps. This time it works.

Start another detached container pinging 8.8.8.8, with the name pinggoogledns.

Now that you’ve finished your testing, you need to remove your containers. In order to remove all of them at once, you want to get only the container IDs. Look at docker container ls --help to get the information you need:

To get only the container IDs, use the --quiet option. If you want to use only the container IDs of all existing containers to perform an action on, you can use --quiet with the --all option.

Since we are done running pings on the public DNS servers, kill the containers. To do this, use the syntax docker container rm --force <containerID>. However, this only kills one container at a time. We want to kill all the containers, no matter what state the containers are in. To get this information, you will need to use the output from docker container ls --quiet --all. To capture this output within the command, use $(…​) to nest the listing command inside the docker container rm command.

This exercise taught you how to start, list, and kill containers. In this exercise you ran your first containers using docker container run, and how they are running commands inside the containers. You also learned to how to list your containers, and how to kill the containers using the command docker container rm. If you run into trouble, you’ve learned that the --help option can provide you with information that could help get you answers.

Create a container using the centos:7 image. Connect to its bash shell in interactive mode using the -i flag and request a TTY connection using the -t flag:

Explore your container’s filesystem with ls, and then create a new file. Use ls again to confirm you have successfully created your file. Use the -l option with ls to list the files and directories in long list format.

Exit the connection to the container:

Run the same command as before to start a container using the centos:7 image:

Use ls to explore your container. You will see that your previously created test.txt is nowhere to be found in your new container. Exit this container in the same way you did above.

We wish to recover test.txt written to our container in the first example, but starting a new container didn’t get us there. We need to restart and reconnect to our original container. List all your stopped containers:

We can restart a container via the container ID listed in the first column. Use the container ID for the first centos:7 container you created with bash as its command (see the CREATED column above to make sure you’re choosing the first bash container you ran):

Your container status has changed from Exited to Up, via docker container start.

Run ps -ef inside the container you just restarted using Docker’s exec command (exec runs the specified process as a child of the PID 1 process inside the container):

What process is PID 1 inside the container? Find the PID of that process on the host machine by using:

Launch a bash shell in your running container with docker container exec:

List the contents of the container’s filesystem with ls -l; your test.txt should be where you left it. Exit the container again by typing exit.

In the last step, we saw how to get the short container ID of all our containers using docker container ls -a. Try adding the --no-trunc flag to see the entire container ID:

This long ID is the same as the string that is returned after starting a container with docker container run.

List only the container ID using the -q flag:

List the last container to have been created using the -l flag:

Finally, you can also filter results with the --filter flag; for example, try filtering by exit code:

[user@node ~]$ docker container ls -a --filter "exited=0" The output of this command will list the containers that have exited successfully.

Clean up with:

In this demo, you saw that files added to a container’s filesystem do not get added to all containers created from the same image. Changes to a container’s filesystem are local to itself, and exist only in that particular container. You also learned how to restart a stopped Docker container using docker container start, how to run a command in a running container using docker container exec, and also saw some more options for listing containers via docker container ls.

First try running a container as usual; the STDOUT and STDERR streams from whatever is PID 1 inside the container are directed to the terminal:

The same process can be run in the background with the -d flag:

This time, we only see the container’s ID; its STDOUT isn’t being sent to the terminal.

Use this second container’s ID to inspect the logs it generated:

These logs correspond to STDOUT and STDERR from the container’s PID 1. Also note when using container IDs: you don’t need to specify the entire ID. Just enough characters from the start of the ID to uniquely identify it, often just 2 or 3, is sufficient.

We can attach a terminal to a container’s PID 1 output with the attach command; try it with the last container you made in the previous step:

We can leave attached mode by then pressing CTRL+C. After doing so, list your running containers; you should see that the container you attached to has been killed, since the CTRL+C issued killed PID 1 in the container, and therefore the container itself.

Try running the same thing in detached interactive mode:

Attach to this container like you did the first one, but this time detach with CTRL+P CTRL+Q (sequential, not simultaneous), and list your running containers. In this case, the container should still be happily running in the background after detaching from it.

We saw previously how to read the entire log of a container’s PID 1; we can also use a couple of flags to control what logs are displayed. --tail n limits the display to the last n lines; try it with the container that should be running from the last step:

You should see the last 5 pings from this container.

We can also follow the logs as they are generated with -f:

The container’s logs get piped in real time to the terminal (CTRL+C to break out of following mode - note this doesn’t kill the process like when we attached to it, since now we’re tailing the logs, not attaching to the process).

Finally, try combining the tail and follow flags to begin following the logs from 10 lines back in history.

In this exercise, we saw our first detached containers. Almost all containers you ever run will be running in detached mode; you can use container attach to interact with their PID 1 processes, as well as container logs to fetch their logs. Note that both attach and logs interact with the PID 1 process only - if you launch child processes inside a container, it’s up to you to manage their STDOUT and STDERR streams. Also, be careful when killing processes after attaching to a container; as we saw, it’s easy to attach to a container and then kill it, by issuing a CTRL+C to the PID 1 process you’ve attached to.

Start by running a container in the background, and check that it’s really running:

Stop the container using docker container stop, and check that the container is indeed stopped:

Start the container again with docker container start, and attach to it at the same time with the -a flag:

As you saw previously, this brings the container from the Exited to the Up state; in this case, we’re also attaching to the PID 1 process.

Detach and stop the container with CTRL+C, then restart the container without attaching and follow the logs starting from 10 lines previous.

Finally, stop the container with docker container kill:

Unlike docker container stop, container kill just sends the SIGKILL right away - no grace period.

Start your ping container again, then inspect the container details using docker container inspect:

You get a JSON object describing the container’s config, metadata and state.

Find the container’s IP and long ID in the JSON output of inspect. If you know the key name of the property you’re looking for, try piping to grep:

The output should look similar to this:

Now try grepping for Cmd, the PID 1 command being run by this container. grep’s simple text search doesn’t always return helpful results:

A more powerful way to filter this JSON is with the --format flag. Syntax follows Go’s text/template package: link::http://golang.org/pkg/text/template/[golang text template]. For example, to find the Cmd value we tried to grep for above, instead try:

This time, we get a the value of the Config.Cmd key from the inspect JSON.

Keys nested in the JSON returned by docker container inspect can be chained together in this fashion. Try modifying this example to return the IP address you grepped for previously.

Finally, we can extract all the key/value pairs for a given object using the json function:

Try adding | jq to this command to get the same output a little bit easier to read.

Start three containers in background mode, then stop the first one.

List only exited containers using the --filter flag we learned earlier, and the option status=exited.

Delete the container you stopped above with docker container rm, and do the same listing operation as above to confirm that it has been removed:

Now do the same to one of the containers that’s still running; notice docker container rm won’t delete a container that’s still running, unless we pass it the force flag -f. Delete the second container you started above:

Try using the docker container ls flags we learned previously to remove the last container that was run, or all stopped containers. Recall that you can pass the output of one shell command cmd-A into a variable of another command cmd-B with syntax like cmd-B $(cmd-A).

When done, clean up any containers you may still have:

In this exercise, you explored the lifecycle of a container, particularly in terms of stopping and restarting containers. Keep in mind the behavior of docker container stop, which sends a SIGTERM, waits a grace period, and then sends a SIGKILL before forcing a container to stop; this two step process is designed to give your containers a chance to shut down 'nicely': dump their state to a log, finish a database transaction, or do whatever your application needs them to do in order to exit without causing additional problems. Make sure you bear this in mind when designing containerized software.

Also keep in mind the docker container inspect command we saw, for examining container metadata, state and config; this is often the first place to look when trying to troubleshoot a failed container.

Start a bash terminal in a CentOS container:

Install a couple pieces of software in this container - there’s nothing special about wget, any changes to the filesystem will do. Afterwards, exit the container:

Finally, try docker container diff to see what’s changed about a container relative to its image; you’ll need to get the container ID via docker container ls -a first:

Those C 's at the beginning of each line stand for files Changed, and A for Added; lines that start with D indicate Deletions.

Installing which and wget in the last step wrote information to the container’s read/write layer; now let’s save that read/write layer as a new read-only image layer in order to create a new image that reflects our additions, via the docker container commit:

Check that you can see your new image by listing all your images:

Create a container running bash using your new image, and check that which and wget are installed:

The which commands should show the path to the specified executable, indicating they have been installed in the image. Exit your container when done by typing exit.

In this exercise, you saw how to inspect the contents of a container’s read / write layer with docker container diff, and commit those changes to a new image layer with docker container commit. Committing a container as an image in this fashion can be useful when developing an environment inside a container, when you want to capture that environment for reproduction elsewhere.

Create a folder called myimage, and a text file called Dockerfile within that folder. In Dockerfile, include the following instructions:

This serves as a recipe for an image based on centos:7, that has all its default packages updated and wget installed on top.

Build your image with the build command. Don’t miss the . at the end; that’s the path to your Dockerfile. Since we’re currently in the directory myimage which contains it, the path is just . (here).

You’ll see a long build output - we’ll go through the meaning of this output in a demo later. For now, your image creation was successful if the output ends with Successfully tagged myimage:latest.

Verify that your new image exists with docker image ls, then use your new image to run a container and wget something from within that container, just to confirm that everything worked as expected:

You should see the HTML from example.com, downloaded by wget from within your container.

It’s also possible to pipe a Dockerfile in from STDIN; try rebuilding your image with the following:

In the previous step, the second time you built your image should have completed immediately, with each step except the first reporting using cache. Cached build steps will be used until a change in the Dockerfile is found by the builder.

Open your Dockerfile and add another RUN step at the end to install vim:

Build the image again as before; which steps is the cache used for?

Build the image again; which steps use the cache this time?

Swap the order of the two RUN commands for installing wget and vim in the Dockerfile:

Build one last time. Which steps are cached this time?

The docker image history command allows us to inspect the build cache history of an image. Try it with your new image:

Replace the two RUN commands that installed wget and vim with a single command:

Build the image again, and run docker image history on this new image. How has the history changed?

In this exercise, we’ve seen how to write a basic Dockerfile using FROM and RUN commands, some basics of how image caching works, and seen the docker image history command. Using the build cache effectively is crucial for images that involve lengthy compile or download steps. In general, moving commands that change frequently as late as possible in the Dockerfile will minimize build times. We’ll see some more specific advice on this later in this lesson.

Add the following line to the bottom of your Dockerfile from the last exercise:

This sets ping as the default command to run in a container created from this image, and also sets some parameters for that command.

Rebuild your image:

Run a container from your new image with no command provided:

You should see the command provided by the CMD parameter in the Dockerfile running.

Try explicitly providing a command when running a container:

Providing a command in docker container run overrides the command defined by CMD.

Replace the CMD instruction in your Dockerfile with an ENTRYPOINT:

Build the image and use it to run a container with no process arguments:

You’ll get an error. What went wrong?

Try running with an argument after the image name:

You should see a successful ping output. Tokens provided after an image name are sent as arguments to the command specified by ENTRYPOINT.

Open your Dockerfile and modify the ENTRYPOINT instruction to include 2 arguments for the ping command:

If CMD and ENTRYPOINT are both specified in a Dockerfile, tokens listed in CMD are used as default parameters for the ENTRYPOINT command. Add a CMD with a default IP to ping:

Build the image and run a container with the defaults:

You should see it pinging the default IP, 127.0.0.1.

Run another container with a custom IP argument:

This time, you should see a ping to 8.8.8.8. Explain the difference in behavior between these two last containers.

In this exercise, we encountered the Dockerfile commands CMD and ENTRYPOINT. These are useful for defining the default process to run as PID 1 inside the container right in the Dockerfile, making our containers more like executables and adding clarity to exactly what process was meant to run in a given image’s containers.

Make a new folder named 'multi' to do this exercise in, and cd into it.

Add a file hello.c to the multi folder containing Hello World in C:

Try compiling and running this right on the host OS:

Now let’s Dockerize our hello world application. Add a Dockerfile to the multi folder with this content:

Build the image and note its size:

Test the image to confirm it was built successfully:

It should print`hello world`" in the console.

Update your Dockerfile to use an AS clause on the first line, and add a second stanza describing a second build stage:

Build the image again and compare the size with the previous version:

As expected, the size of the multi-stage build is much smaller than the large one since it does not contain the Alpine SDK.

Finally, make sure the app works:

You should get the expected Hello, World! output from the container with just the required executable.

In the previous step, we took our compiled executable from the first build stage, but that image wasn’t tagged as a regular image we can use to start containers with; only the final FROM statement generated a tagged image. In this step, we’ll see how to persist whichever build stage we like.

Build an image from the build stage in your Dockerfile using the --target flag:

Notice all its layers are pulled from the cache; even though the build stage wasn’t tagged originally, its layers are nevertheless persisted in the cache.

Run a container from this image and make sure it yields the expected result:

List your images again to see the size of my-build-stage compared to the small version of the app.

So far, every image we’ve built has been based on a pre-existing image, referenced in the FROM command. But what if we want to start from nothing, and build a completely original image? For this, we can build FROM scratch.

In a new directory ~/scratch, create a file named sleep.c that just launches a sleeping process for an hour:

Create a file named Dockerfile to build this sleep program in a build stage, and then copy it to a scratch-based image:

This image will contain nothing but our executable and the bare minimum file structure Docker needs to stand up a container filesystem.

Build your image:

List your images, and search for the one you just built:

This image is only 128 kB, as tiny as possible.

Run your image, and check out its filesystem; we can’t list directly inside the container, since ls isn’t installed in this ultra-minimal image, so we have to find where this container’s filesystem is mounted on the host. Start by finding the PID of your sleep process after its running:

In this example, the PID for sleep is 1190.

List your container’s filesystem from the host using this PID:

We see not only our binary sleep but a bunch of other folders and files. Where does these come from? runC, the tool for spawning and running containers, requires a json config of the container and a root file system. At runtime, Docker Engine adds these minimum requirements to form the most minimal container filesystem possible.

Clean up by deleting your container:

In addition to the default builder, BuildKit can be enabled to take advantages of some optimizations of the build process.

Back in the ~/multi directory, turn on BuildKit:

Add an AS label to the final stage of your Dockerfile (this is not strictly necessary, but will make the output in the next step easier to understand):

Re-build my-app-small, without the cache:

Notice the lines marked like [prod 2/3] and [build 4/6]: prod and build in this context are the AS labels you applied to the FROM lines in each stage of your build in the Dockerfile; from the above output, you can see that the build stages were built in parallel. Every step of the final image was completed while the build environment image was being created; the prod environment image creation was only blocked at the COPY instruction since it required a file from the completed build image.

Comment out the COPY instruction in the prod image definition in your Dockerfile, and rebuild; the build image is skipped. BuildKit recognized that the build stage was not necessary for the image being built, and skipped it.

Turn off BuildKit:

In this exercise, you created a Dockerfile defining multiple build stages. Being able to take artifacts like compiled binaries from one image and insert them into another allows you to create very lightweight images that do not include developer tools or other unnecessary components in your production-ready images, just like how you currently probably have separate build and run environments for your software. This will result in containers that start faster, and are less vulnerable to attack.

If you don’t have one already, head over to link::https://hub.docker.com[https://hub.docker.com] and make an account.

For the rest of this workshop, <Docker ID> refers to the username you chose for this account.

Download the centos:7 image from Docker Hub:

Make a new tag of this image:

List your images:

You should have centos:7 and my-centos:dev both listed, but they ought to have the same hash under image ID, since they’re actually the same image.

Push your image to Docker Hub:

You should get a denied: requested access to the resource is denied error.

Login by doing docker login, and try pushing again. The push fails again because we haven’t namespaced our image correctly for distribution on Docker Hub; all images you want to share on Docker Hub must be named like <Docker ID>/<repo name>[:<optional tag>].

Retag your image to be namespaced properly, and push again:

Search Docker Hub for your new <Docker ID>/my-centos repo, and confirm that you can see the :dev tag therein.

Next, write a Dockerfile that uses <Docker ID>/my-centos:dev as its base image, and installs any application you like on top of that. Build the image, and simultaneously tag it as :1.0:

Push your :1.0 tag to Docker Hub, and confirm you can see it in the appropriate repository.

Finally, list the images currently on your node with docker image ls. You should still have the version of your image that wasn’t namespaced with your Docker Hub user name; delete this using docker image rm:

Only the tag gets deleted, not the actual image. The image layers are still referenced by another tag.

In this exercise, we praciced tagging images and exchanging them on the public registry. The namespacing rules for images on registries are mandatory: user-generated images to be exchanged on the public registry must be named like <Docker ID>/<repo name>[:<optional tag>]; official images in the Docker registry just have the repo name and tag.

Download a postgres image, and look at its history to determine its default volume usage:

You should see a Volumes block like the above, indicating that those paths in the container filesystem will get volumes automatically mounted to them when a container is started based on this image.

Set up a running instance of this postgres container:

Notice the explicit volume mount, -v db_backing:/var/lib/postgresql/data; if we hadn’t done this, a randomly named volume would have been mounted to the container’s /var/lib/postgresql/data. Naming the volume explicitly is a best practice that will become useful when we start mounting this volume in multiple containers.

The psql command line interface to postgres comes packaged with the postgres image; spawn it as a child process in your postgres container interactively, to create a postgres terminal:

[user@node ~]$ docker container exec \ -it some-postgres psql -U postgres Create an arbitrary table in the database:

Double check you created the table you expected, and then quit this container:

Delete the postgres container:

Create a new postgres container, mounting the db_backing volume just like last time:

Reconnect a psql interface to your database, also like before:

List the contents of the PRODUCTS table:

The contents of the database have survived the deletion and recreation of the database container; this would not have been true if the database was keeping its data in the writable container layer. As above, use \q to quit from the postgres prompt.

Whenever data needs to live longer than the lifecycle of a container, it should be pushed out to a volume outside the container’s filesystem; numerous popular databases are containerized using this pattern.

In the dropdown menu at the top of the Strigo webpage, click into node-1. See what networks are present on your host:

You should have entries for host, none, and bridge.

Find some metadata about the default bridge network:

See similar info from common networking tools:

Use brctl to see connections to the docker0 bridge:

At the moment, there are no connections to docker0.

Start a container and reexamine the network; the container is listed as connected to the network, with an IP assigned to it from the bridge network’s subnet:

Inspect the network interfaces with ip and brctl again, now that you have a container running:

ip addr indicates a veth endpoint has been created and plugged into the docker0 bridge, as indicated by master docker0, and that it is connected to device index 4 in this case (indicated by the @if4 suffix to the veth device name above). Similarly, brctl now shows this veth connection on docker0 (notice that the ID for the veth connection matches in both utilities).

Launch a bash shell in your container, and look for the eth0 device therein:

We see that the eth0 device in this namespace is in fact the device that the veth connection in the host namespace indicated it was attached to, and vice versa - eth0@if5 indicates it is plugged into networking interface number 5, which we saw above was the other end of the veth connection. Docker has created a veth connection with one end in the host’s docker0 bridge, and the other providing the eth0 device in the container.

In the last step, we investigated the default bridge network; now let’s try making our own. User defined bridge networks work exactly the same as the default one, but provide DNS lookup by container name, and are firewalled from other networks by default.

Create a bridge network by using the bridge driver with docker network create:

Launch a container connected to your new network via the --network flag:

Use the inspect command to investigate the network settings of this container:

my_bridge should be listed under the Networks key.

Launch another container, this time interactively:

From inside container u3, ping u2 by name: ping u2. The ping succeeds, since Docker is able to resolve container names when they are attached to a custom network.

Try starting a container on the default network, and pinging u1 by name:

The ping fails; even though the containers are both attached to the bridge network, Docker does not provide name lookup on this default network. Try the same command again, but using u1 's IP instead of name, and you should be successful.

Finally, try pinging u1 by IP, this time from container u2:

The ping fails, since the containers reside on different networks; all Docker networks are firewalled from each other by default.

Clean up your containers and networks:

In this exercise, you explored the fundamentals of container networking. The key take away is that containers on separate networks are firewalled from each other by default. This should be leveraged as much as possible to harden your applications; if two containers don’t need to talk to each other, put them on separate networks.

You also explored a number of API objects:

Run an nginx container with no special port mappings:

nginx stands up a landing page at <ip>:80. If you try to visit this at your host or container’s IP it won’t be visible; no external traffic can make it past the linux bridge’s firewall to the nginx container.

Now run an nginx container and map port 80 on the container to port 5000 on your host using the -p flag:

Verify the port mappings with the docker container port command

Visit your nginx landing page at <host ip>:5000, e.g. using curl -4 localhost:5000, just to confirm it’s working as expected.

In addition to manual port mapping, we can expose some ports for automatic port mapping on container startup using a Dockerfile. In a new directory ~/port, create a Dockerfile:

Build your image as my_nginx:

Use the -P flag when running to map all ports mentioned in the EXPOSE directive:

Use docker container ls or docker container port to find out which host ports were used, then visit your nginx landing page in a browser at <node-1 public IP>:<port>.

Clean up your containers:

In this exercise, we saw how to explicitly map ports from our container’s network stack onto ports of our host at runtime with the -p option to docker container run, or more flexibly in our Dockerfile with EXPOSE, which will result in the listed ports inside our container being mapped to random available ports on our host. In both cases, Docker is writing iptables rules to forward traffic from the host to the appropriate port in the container’s network namespace.

Download the Dockercoins app from github:

This app consists of 5 services: a random number generator rng, a hasher, a backend worker, a redis queue, and a web frontend; the code you just downloaded has the source code for each process and a Dockerfile to containerize each of them.

Have a brief look at the source for each component of your application. Each folder under ~/orchestration-workshop/ dockercoins contains the application logic for the component, and a Dockerfile for building that logic into a Docker image. We’ve pre-built these images as training/dockercoins-rng:1.0, training/dockercoins-worker:1.0 et cetera, so no need to build them yourself.

Have a look in docker-compose.yml; especially notice the services section. Each block here defines a different Docker service. They each have exactly one image which containers for this service will be started from, as well as other configuration details like network connections and port exposures. Full syntax for Docker Compose files can be found here: link::https://dockr.ly/2iHUpeX[Docker Compose Specification].

Stand up the app:

After a moment, your app should be running; visit <node 0 public IP>:8000 to see the web frontend visualizing your rate of Dockercoin mining.

Logs from all the running services are sent to STDOUT. Let’s send this to the background instead; kill the app with CTRL+C, sending a SIGTERM to all running processes; some exit immediately, while others wait for a 10s timeout before being killed by a subsequent SIGKILL. Start the app again in the background:

Check out which containers are running thanks to Compose:

Compare this to the usual docker container ls; do you notice any differences? If not, start a couple of extra containers using docker container run…​, and check again.

See logs from a Compose-managed app via:

The logging API in Compose follows the main Docker logging API closely. For example, try following the tail of the logs just like you would for regular container logs:

Note that when following a log, CTRL+S and CTRL+Q pauses and resumes live following; CTRL+C exits follow mode as usual.

In this exercise, you saw how to start a pre-defined Compose app, and how to inspect its logs. Application logic was defined in each of the five images we used to create containers for the app, but the manner in which those containers were created was defined in the docker-compose.yml file; all runtime configuration for each container is captured in this manifest. Finally, the different elements of Dockercoins communicated with each other via service name; the Docker daemon’s internal DNS was able to resolve traffic destined for a service, into the IP or MAC address of the corresponding container.

Any service defined in our docker-compose.yml can be scaled up from the Compose API; in this context, 'scaling' means launching multiple containers for the same service, which Docker Compose can route requests to and from.

Scale up the worker service in our Dockercoins app to have two workers generating coin candidates by redeploying the app with the --scale flag, while checking the list of running containers before and after:

A new worker container has appeared in your list of containers.

Look at the performance graph provided by the web frontend; the coin mining rate should have doubled. Also check the logs using the logging API we learned in the last exercise; you should see a second worker instance reporting.

Try running top to inspect the system resource usage; it should still be fairly negligible. So, keep scaling up your workers:

Check your web frontend again; has going from 2 to 10 workers provided a 5x performance increase? It seems that something else is bottlenecking our application; any distributed application such as Dockercoins needs tooling to understand where the bottlenecks are, so that the application can be scaled intelligently.

Look in docker-compose.yml at the rng and hasher services; they’re exposed on host ports 8001 and 8002, so we can use httping to probe their latency.

rng on port 8001 has the much higher latency, suggesting that it might be our bottleneck. A random number generator based on entropy won’t get any better by starting more instances on the same machine; we’ll need a way to bring more nodes into our application to scale past this, which we’ll explore in the next unit on Docker Swarm.

For now, shut your app down:

In this exercise, we saw how to scale up a service defined in our Compose app using the --scale flag. Also, we saw how crucial it is to have detailed monitoring and tooling in a microservices-oriented application, in order to correctly identify bottlenecks and take advantage of the simplicity of scaling with Docker.

In this exercise, we saw some very basic docker prune usage - most of the top-level docker objects have a prune command (docker container prune, docker volume prune etc). Most docker objects leave something on disk even after being shut down; consider using these cleanup commands as part of your cluster maintenance and garbage collection plan, to avoid accidentally running out of disk on your Docker hosts.

We can find the info command under system. Execute:

This provides some high-level information about the docker deployment on the current node, and the node itself. From this output, identify:

There is another powerful system command that allows us to monitor what’s happening on the Docker host. Execute the following command:

Please note that it looks like the system is hanging, but that is not the case. The system is just waiting for some events to happen.

Open a second connection to node-3 and execute the following command:

Granular information about every action taken by the Docker engine is presented in the events stream.

If you don’t like the format of the output then we can use the --format parameter to define our own format in the form of a link::https://golang.org/pkg/text/template/[Go template]. Stop the events watch on your first terminal with CTRL+C, and try this:

now the output looks a little bit less cluttered when we run our alpine container on the second terminal as above.

Finally we can find out what the event structure looks like by outputting the events in json format (once again after killing the events watcher on the first terminal and restarting it with):

which should give us for the first event in the series after re-running our alpine container on the other connection to node-3 something like this (note, the output has been prettyfied for readability):

In this exercise we have learned how to inspect system wide properties of our Docker host by using the docker system info command; this is one of the first places to look for general config information to include in a bug report. We also saw a simple example of docker system events; the events stream is one of the primary sources of information that should be logged and monitored when running Docker in production. Many commercial as well as open source products (such as Elastic Stack) exist to facilitate aggregating and mining these streams at scale.

If you’re not familiar with the common Linux commands (cd, ls, ps, sudo, etc.), this exercise is designed to lead you through the common linux commands that we use in the Docker training.

By the end of this exercise, you should be able to: * Manage files and folders using Linux commands * Inspect processes running on your machine and use some discovery tools.

Pre-requisites:

In this first part we will walk through common commands for creating and manipulating files and folders from the command line.

Open a terminal.

Terminals work by accepting text-based commands, and retuning text-based responses. Ask the terminal what directory you are currently in; in all our instructions, $ represents the terminal prompt, and you should enter what you see after the prompt. Yours might look a bit different; in any case, type pwd:

pwd stands for Print Working Directory. You should see something like:

This means your terminal is currently in the ckaserer folder, which itself is in the /Users folder (yours will have different folder names, but the logic is the same).

List the files and folders in your current directory:

Here we show some typical output below the command; again, you only need to enter what comes after the command prompt, ls in this case. ls lists all the contents of your current directory.

Change directory with cd:

Again, your directories might be named differently - that’s ok. Try to navigate to a directory you recognize the names of; Desktop or Downloads are common.

Use ls again to see what’s in this directory. Compare what you see in your Desktop directory to what you actually see on your machine’s desktop; the contents are identical. The terminal is just another way to browse the contents of your machine, based on programmatic text rather than visual analogies.

Make a new folder on your desktop (or wherever you currently are):

If you do ls again, you’ll see a new directory demo-dir has been made.

Change directory to demo-dir, and open a new plain text file called demo.txt using the simple text editor nano:

You’re taken to nano 's text-based interface for creating a plain-text file. There are many other plain-text editors available, the most popular being vim, but nano is probably the simplest if this is your first time using a text editor from the command line.

Add any text you want to your file, just by typing something.

Save your file by pressing CMD+O, and pressing return when asked for a filename. Exit by typing CMD+X.

Dump the contents of your file to the screen:

Make a copy of your file with cp:

Move and rename your file with mv:

Delete a file:

Check what directory you’re in, then back up one level with the special .. path, and check again:

cd .. took us one level up in our directory tree, from demo-dir back out to the directory that contains it, Desktop in my example.

Delete your demo-dir, again with remove, but this time using the -r flag, which stands for recursive in order to delete the folder and all its contents: