How to run Containers as a Service – Secure Docker communications
Last March, Dimitri did an excellent series on ‘How to run Containers as a Service’. In part 1, Dimitri describes containers, the
problems of running containers and possible solutions. In part 2, he published a very elaborate guide on how to configure Containers as a Service using vRealize Automation. But when upgrading vRealize Automation to the new 7.3 release this breaks. Investigating this revealed that vRealize Automation 7.3 requires secure Docker communications and the unsecured http option used in part 2 no longer works.
By default, Docker runs via a non-networked Unix socket. If you need Docker to be reachable via the network in a safe manner, you can enable TLS by specifying the
tlsverify flag and pointing Docker’s
tlscacert flag to a trusted CA certificate. In the daemon mode, it will only allow connections from clients authenticated by a certificate signed by that CA. In the client mode, it will only connect to servers with a certificate signed by that CA.
So how do we secure Docker communications?
Generate Certificates on the Photon OS Template
- Connect to the Photon OS template via SSH
- Start and enable docker using the following commands
$ systemctl start docker
$ systemctl enable docker
- Generate self-signed Certificates using the following commands:
$ openssl genrsa -aes256 -out ca-key.pem 4096
$ openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem
Provide the certificate details and Photon template FQDN, when prompted for Common Name input.
- Now that we have a CA, you can create a server key and certificate signing request (CSR). Make sure that “Common Name” (i.e., server FQDN or YOUR name) matches the hostname you will use to connect to Docker.
$ openssl genrsa -out server-key.pem 4096
$ openssl req -subj "/CN=[FQDN Docker host]" -sha256 -new -key server-key.pem -out server.csr
$ echo subjectAltName = DNS:[FQDN Docker host],IP:[IP ranges] > extfile.cnf
$ openssl x509 -req -days 365 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem \ -CAcreateserial -out server-cert.pem -extfile extfile.cnf
- For client authentication, create a client key, a certificate signing request, an extensions config file and sign the private key.
$ openssl genrsa -out key.pem 4096
$ openssl req -subj '/CN=client' -new -key key.pem -out client.csr
$ echo extendedKeyUsage = clientAuth > extfile.cnf
$ openssl x509 -req -days 365 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem \ -CAcreateserial -out cert.pem -extfile extfile.cnf
- Next, make the Docker daemon only accept connections from clients providing a certificate trusted by the CA and enable certificate validation. Copy your CA certificate, your server certificate, and your client certificate to your Docker client machine.
$ openssl genrsa -out key.pem 4096
$ dockerd --tlsverify --tlscacert=ca.pem --tlscert=server-cert.pem --tlskey=server-key.pem \ -H=0.0.0.0:2376
$ docker --tlsverify --tlscacert=ca.pem --tlscert=cert.pem --tlskey=key.pem \ -H=[FQDN Docker host]:2376 version
- Open docker file using vi and add the following line to this file.
$ vi /etc/default/docker
DOCKER_OPTS="--tlsverify --tlscacert=/root/ca.pem --tlscert=/root/server-cert.pem --tlskey=/root/server-key.pem -H=0.0.0.0:2376"
- Incoming traffic on port 2376 is blocked by default so it must be allowed through the firewall. To do this, add the firewall rule and restart the service to make the changes effective.
$ echo "iptables -I INPUT 1 -p tcp --dport 2376 -j ACCEPT" >> /etc/systemd/scripts/iptables
$ systemctl restart iptables
$ systemctl restart docker
- We’re almost done. The final step is to display and save the contents of the public and private certificate keys.
$ cat /root/server-cert.pem
$ cat /root/server-key.pem
Copy and paste the certificates into a text file which you will need to use in a later stage.
Verify secure Docker communications
Now the secure Docker communications is setup we need to check if we performed the steps above in a correct manner, we need to verify if the certificate validation if done correctly. We can do this by adding the template to the Containers Tab within vRealize Automation. To do this do the following:
- Open your vRA console and go to the Containers tab.
- Select [Add a Host].
- Provide the Photon OS template URL in the address field (syntax = https://[IP address]:2376).
- Enter new credentials using the [New Credentials] option.
- Enter the correct credentials and select the [Certificate] option.
- Paste the Docker certificates we saved to notepad earlier in the appropriate fields.
- Confirm and save the credentials by using the [Verify] button and select [Yes] to confirm the certificate.
If you performed the actions correctly you should see a green message ribbon saying: “Verified successfully!”
- Now that secure Docker communications is verified, shutdown the Photon VM and convert to a vSphere template
You can now continue with ‘Create a container host Blueprint and configure Container Placement‘ in part 2 of Dimitri’s series on Containers as a Service.
Other articles in the series vRealize Automation:
- Infoblox & vRealize Automation, Setup IPAM endpoint
- Infoblox & vRealize Automation, Install Infoblox IPAM plugin
- Setup provisioning of vCloud Air workloads
- Setup a vCloud Air Endpoint in vRealize Automation
- Setup Postgres database high availability for vRealize Automation
- Setup vRealize Automation Cost profiles & currency
- Speed up vRealize Automation life cycle updates
- Configure vRealize Automation not to delete virtual machines
- Track infrastructure changes with vRealize Automation Data Collection
- MS SQL high availability support for vRealize Automation
- Setup the vRealize Automation prerequisites for IaaS
- vRealize Automation: How to …
- Downgrade the vRealize Automation license
- vRealize Automation 7 – New Architecture & Installation
- vRealize Automation 7 – Improved Authentication
- vRealize Automation 7 – Converged Blueprints
- vRealize Automation 7 – LifeCycle Extensibility
- How to transfer vRealize Automation payload to Orchestrator
- Automation is not difficult with vRealize Automation 7
- vRA custom host name using Orchestrator and Event Broker
- How to deliver Database-as-a-Service with vRealize Automation
- Prepare Windows for vRA software deployment
- Missing endpoints after upgrading to vRealize Automation 7.3
- How to run Containers as a Service – Part 1
- How to run Containers as a Service – Part 2
- Infoblox & vRealize Automation, IP Address Management (IPAM) made easy
- How to run Containers as a Service – Secure Docker communications
- Create a new database user using vRealize Automation XaaS
- Infoblox & vRealize Automation, Infoblox NIOS setup
- Infoblox & vRealize Automation, vRA setup
- vRealize Automation failed requests monitoring
- vRealize Automation 7.2 released
- How to specify a deployment target when using vRealize Automation
- How to run Containers as a Service – Part 2 by Dimitri De Swart
- How to run Containers as a Service – Part 1 by Dimitri De Swart
- PowerShell Friday: Connecting to vCenter by Anne Jan Elsinga
- vRealize Orchestrator Puppet plugin is available now by Erik Scholten
- Secure Cloud Native Applications with Lightwave by Erik Scholten