Consul notes

Russell Bateman
March 2019
last update:

Links

Consul is a distributed (clusterable) service mesh to connect, secure and configure services across any runtime platform and public or private cloud. Think of it as accurate configuration files on clusterable, shared, steroids. It is more or less a performant key-value store with lots of adaptable technology in the role it fulfills.

Installation and getting started

Consul is available on Linux Mint from the Software Manager, but like most ready software from Ubuntu and other distributions, it's very much out of date, so you'll want to get the software from HashiCorp's website.

russ@tirion:~/Downloads$ ll consul_1.4.3_linux_amd64.zip
-rw-rw-r-- 1 russ russ 34777003 Mar 19 09:19 consul_1.4.3_linux_amd64.zip
russ@tirion:~/Downloads$ sudo mv consul /usr/local/bin
russ@tirion:~/Downloads$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Verify that Consul is working

russ@tirion:~/Downloads$ consul --version
Consul v1.4.3
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking\
    to compatible agents)

Starting and stopping the Consul agent

russ@tirion:~/Downloads$ consul agent -dev
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.4.3'
           Node ID: 'a3007bbc-cff7-ae19-2c86-2fd22e8ad6f0'
         Node name: 'tirion'
        Datacenter: 'dc1' (Segment: '<all>')
            Server: true (Bootstrap: false)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
      Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2019/03/19 09:31:07 [DEBUG] agent: Using random ID "a3007bbc-cff7-ae19-2c86-2fd22e8ad6f0" as node ID
    2019/03/19 09:31:07 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:a3007bbc-cff7-ae19-2c86-2fd22e8ad6f0 Address:127.0.0.1:8300}]
    2019/03/19 09:31:07 [INFO] raft: Node at 127.0.0.1:8300 [Follower] entering Follower state (Leader: "")
    2019/03/19 09:31:07 [INFO] serf: EventMemberJoin: tirion.dc1 127.0.0.1
    2019/03/19 09:31:07 [INFO] serf: EventMemberJoin: tirion 127.0.0.1
    2019/03/19 09:31:07 [INFO] consul: Handled member-join event for server "tirion.dc1" in area "wan"
    2019/03/19 09:31:07 [INFO] consul: Adding LAN server tirion (Addr: tcp/127.0.0.1:8300) (DC: dc1)
    2019/03/19 09:31:07 [DEBUG] agent/proxy: managed Connect proxy manager started
    2019/03/19 09:31:07 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2019/03/19 09:31:07 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2019/03/19 09:31:07 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2019/03/19 09:31:07 [INFO] agent: started state syncer
    2019/03/19 09:31:07 [INFO] agent: Started gRPC server on 127.0.0.1:8502 (tcp)
    2019/03/19 09:31:08 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2019/03/19 09:31:08 [INFO] raft: Node at 127.0.0.1:8300 [Candidate] entering Candidate state in term 2
    2019/03/19 09:31:08 [DEBUG] raft: Votes needed: 1
    2019/03/19 09:31:08 [DEBUG] raft: Vote granted from a3007bbc-cff7-ae19-2c86-2fd22e8ad6f0 in term 2. Tally: 1
    2019/03/19 09:31:08 [INFO] raft: Election won. Tally: 1
    2019/03/19 09:31:08 [INFO] raft: Node at 127.0.0.1:8300 [Leader] entering Leader state
    2019/03/19 09:31:08 [INFO] consul: cluster leadership acquired
    2019/03/19 09:31:08 [INFO] consul: New leader elected: tirion
    2019/03/19 09:31:08 [INFO] connect: initialized primary datacenter CA with provider "consul"
    2019/03/19 09:31:08 [DEBUG] consul: Skipping self join check for "tirion" since the cluster is too small
    2019/03/19 09:31:08 [INFO] consul: member 'tirion' joined, marking health alive
    2019/03/19 09:31:08 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/19 09:31:08 [INFO] agent: Synced node info
    2019/03/19 09:31:08 [DEBUG] agent: Node info in sync
    2019/03/19 09:31:10 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/19 09:31:10 [DEBUG] agent: Node info in sync

Examine all the nodes in the cluster. There is just one node, obviously. Consul offers an HTTP-based API (port 8500) and also a DNS interface (port 8600).

russ@tirion:~/Downloads$ consul members
Node    Address         Status  Type    Build  Protocol  DC   Segment
tirion  127.0.0.1:8301  alive   server  1.4.3  2         dc1  
russ@tirion:~/Downloads$ curl localhost:8500/v1/catalog/nodes
[
    {
        "ID": "a3007bbc-cff7-ae19-2c86-2fd22e8ad6f0",
        "Node": "tirion",
        "Address": "127.0.0.1",
        "Datacenter": "dc1",
        "TaggedAddresses": {
            "lan": "127.0.0.1",
            "wan": "127.0.0.1"
        },
        "Meta": {
            "consul-network-segment": ""
        },
        "CreateIndex": 9,
        "ModifyIndex": 10
    }
]
russ@tirion:~/Downloads$ dig @127.0.0.1 -p 8600 tirion.node.consul

; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> @127.0.0.1 -p 8600 tirion.node.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58074
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;tirion.node.consul.		IN	A

;; ANSWER SECTION:
tirion.node.consul.	0	IN	A	127.0.0.1

;; ADDITIONAL SECTION:
tirion.node.consul.	0	IN	TXT	"consul-network-segment="

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Mar 19 09:36:09 MDT 2019
;; MSG SIZE  rcvd: 99

Stopping the agent is done gracefully using Ctrl-C:

    .
    .
    .
    2019/03/19 09:38:08 [DEBUG] consul: Skipping self join check for "tirion" since the cluster is too small
    2019/03/19 09:38:08 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/19 09:38:08 [DEBUG] agent: Node info in sync
^C    2019/03/19 09:39:06 [INFO] agent: Caught signal:  interrupt
    2019/03/19 09:39:06 [INFO] agent: Graceful shutdown disabled. Exiting
    2019/03/19 09:39:06 [INFO] agent: Requesting shutdown
    2019/03/19 09:39:06 [WARN] agent: dev mode disabled persistence, killing all proxies since we can't recover them
    2019/03/19 09:39:06 [DEBUG] agent/proxy: Stopping managed Connect proxy manager
    2019/03/19 09:39:06 [INFO] consul: shutting down server
    2019/03/19 09:39:06 [WARN] serf: Shutdown without a Leave
    2019/03/19 09:39:06 [WARN] serf: Shutdown without a Leave
    2019/03/19 09:39:06 [INFO] manager: shutting down
    2019/03/19 09:39:06 [INFO] agent: consul server down
    2019/03/19 09:39:06 [INFO] agent: shutdown complete
    2019/03/19 09:39:06 [INFO] agent: Stopping DNS server 127.0.0.1:8600 (tcp)
    2019/03/19 09:39:06 [INFO] agent: Stopping DNS server 127.0.0.1:8600 (udp)
    2019/03/19 09:39:06 [INFO] agent: Stopping HTTP server 127.0.0.1:8500 (tcp)
    2019/03/19 09:39:06 [INFO] agent: Waiting for endpoints to shut down
    2019/03/19 09:39:06 [INFO] agent: Endpoints down
    2019/03/19 09:39:06 [INFO] agent: Exit code: 1

Registering services

Now we're getting somewhere useful. We're still following the getting-started instructions here...

russ@tirion:~/Downloads$ sudo ls -alg -d /etc/consul*
ls: cannot access '/etc/consul*': No such file or directory
russ@tirion:~/Downloads$ sudo mkdir /etc/consul.d
russ@tirion:~/Downloads$ sudo bash
root@tirion:~/Downloads# cd /etc/consul.d/
root@tirion:/etc/consul.d# echo '{"service": {"name": "web", "tags": ["rails"], "port": 80}}' > /etc/consul.d/web.json
root@tirion:/etc/consul.d# ll
total 20
drwxr-xr-x   2 root root  4096 Mar 19 09:53 ./
drwxr-xr-x 144 root root 12288 Mar 19 09:42 ../
-rw-r--r--   1 root root    60 Mar 19 09:53 web.json
root@tirion:/etc/consul.d# consul agent -dev -config-dir=/etc/consul.d
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v1.4.3'
           Node ID: '07f0fffe-08c5-7ca5-9de0-55729bd459b6'
         Node name: 'tirion'
        Datacenter: 'dc1' (Segment: '')
            Server: true (Bootstrap: false)
       Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
      Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2019/03/19 09:56:13 [DEBUG] agent: Using random ID "07f0fffe-08c5-7ca5-9de0-55729bd459b6" as node ID
    2019/03/19 09:56:13 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:07f0fffe-08c5-7ca5-9de0-55729bd459b6 Address:127.0.0.1:8300}]
    2019/03/19 09:56:13 [INFO] raft: Node at 127.0.0.1:8300 [Follower] entering Follower state (Leader: "")
    2019/03/19 09:56:13 [INFO] serf: EventMemberJoin: tirion.dc1 127.0.0.1
    2019/03/19 09:56:13 [INFO] serf: EventMemberJoin: tirion 127.0.0.1
    2019/03/19 09:56:13 [INFO] consul: Adding LAN server tirion (Addr: tcp/127.0.0.1:8300) (DC: dc1)
    2019/03/19 09:56:13 [DEBUG] agent/proxy: managed Connect proxy manager started
    2019/03/19 09:56:13 [WARN] agent/proxy: running as root, will not start managed proxies
    2019/03/19 09:56:13 [INFO] consul: Handled member-join event for server "tirion.dc1" in area "wan"
    2019/03/19 09:56:13 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2019/03/19 09:56:13 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2019/03/19 09:56:13 [INFO] agent: Started HTTP server on 127.0.0.1:8500 (tcp)
    2019/03/19 09:56:13 [INFO] agent: started state syncer
    2019/03/19 09:56:13 [INFO] agent: Started gRPC server on 127.0.0.1:8502 (tcp)
    2019/03/19 09:56:13 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2019/03/19 09:56:13 [INFO] raft: Node at 127.0.0.1:8300 [Candidate] entering Candidate state in term 2
    2019/03/19 09:56:13 [DEBUG] raft: Votes needed: 1
    2019/03/19 09:56:13 [DEBUG] raft: Vote granted from 07f0fffe-08c5-7ca5-9de0-55729bd459b6 in term 2. Tally: 1
    2019/03/19 09:56:13 [INFO] raft: Election won. Tally: 1
    2019/03/19 09:56:13 [INFO] raft: Node at 127.0.0.1:8300 [Leader] entering Leader state
    2019/03/19 09:56:13 [INFO] consul: cluster leadership acquired
    2019/03/19 09:56:13 [INFO] consul: New leader elected: tirion
    2019/03/19 09:56:13 [INFO] connect: initialized primary datacenter CA with provider "consul"
    2019/03/19 09:56:13 [DEBUG] consul: Skipping self join check for "tirion" since the cluster is too small
    2019/03/19 09:56:13 [INFO] consul: member 'tirion' joined, marking health alive
    2019/03/19 09:56:14 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/19 09:56:14 [INFO] agent: Synced service "web"
    2019/03/19 09:56:14 [DEBUG] agent: Node info in sync
    2019/03/19 09:56:14 [DEBUG] agent: Service "web" in sync
    2019/03/19 09:56:14 [DEBUG] agent: Node info in sync
    2019/03/19 09:56:16 [DEBUG] agent: Skipping remote check "serfHealth" since it is managed automatically
    2019/03/19 09:56:16 [DEBUG] agent: Service "web" in sync
    2019/03/19 09:56:16 [DEBUG] agent: Node info in sync

Querying services using the DNS API. As may have been guessed from what's gone before, the DNS name for a node is node-name.node.consul. The DNS name for a service is service-name.service.consul.

root@tirion:/etc/consul.d# dig @127.0.0.1 -p 8600 web.service.consul

; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> @127.0.0.1 -p 8600 web.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32369
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;web.service.consul.		IN	A

;; ANSWER SECTION:
web.service.consul.	0	IN	A	127.0.0.1

;; ADDITIONAL SECTION:
web.service.consul.	0	IN	TXT	"consul-network-segment="

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Mar 19 10:43:52 MDT 2019
;; MSG SIZE  rcvd: 99

An A record was returned with the IP address of the node making the service available. (A records only hold IP addresses.) You can use the DNS API to give this information as an SRV record...

root@tirion:/etc/consul.d# dig @127.0.0.1 -p 8600 web.service.consul SRV

; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> @127.0.0.1 -p 8600 web.service.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19275
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;web.service.consul.		IN	SRV

;; ANSWER SECTION:
web.service.consul.	0	IN	SRV	1 1 80 tirion.node.dc1.consul.

;; ADDITIONAL SECTION:
tirion.node.dc1.consul.	0	IN	A	127.0.0.1
tirion.node.dc1.consul.	0	IN	TXT	"consul-network-segment="

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Mar 19 10:46:34 MDT 2019
;; MSG SIZE  rcvd: 141

...and another to filter services (we've only one here) by their tags. Our service's (only) tag is rails, specified by /etc/consul.d/web.jason:

root@tirion:/etc/consul.d# dig @127.0.0.1 -p 8600 rails.web.service.consul

; <<>> DiG 9.11.3-1ubuntu1.5-Ubuntu <<>> @127.0.0.1 -p 8600 rails.web.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44674
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;rails.web.service.consul.	IN	A

;; ANSWER SECTION:
rails.web.service.consul. 0	IN	A	127.0.0.1

;; ADDITIONAL SECTION:
rails.web.service.consul. 0	IN	TXT	"consul-network-segment="

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Tue Mar 19 10:50:50 MDT 2019
;; MSG SIZE  rcvd: 105

Back to the HTTP API, we can query services too. And we can limit our query to return only "healthy" (passing) services:


root@tirion:/etc/consul.d# curl http://localhost:8500/v1/catalog/service/web
[
    {
        "ID": "07f0fffe-08c5-7ca5-9de0-55729bd459b6",
        "Node": "tirion",
        "Address": "127.0.0.1",
        "Datacenter": "dc1",
        "TaggedAddresses": {
            "lan": "127.0.0.1",
            "wan": "127.0.0.1"
        },
        "NodeMeta": {
            "consul-network-segment": ""
        },
        "ServiceKind": "",
        "ServiceID": "web",
        "ServiceName": "web",
        "ServiceTags": [
            "rails"
        ],
        "ServiceAddress": "",
        "ServiceWeights": {
            "Passing": 1,
            "Warning": 1
        },
        "ServiceMeta": {},
        "ServicePort": 80,
        "ServiceEnableTagOverride": false,
        "ServiceProxyDestination": "",
        "ServiceProxy": {},
        "ServiceConnect": {},
        "CreateIndex": 10,
        "ModifyIndex": 10
    }
]
root@tirion:/etc/consul.d# curl http://localhost:8500/v1/catalog/service/web?passing
(ibid)

To change or update a service, you can modify its configuration file, then send SIGHUP to Consul:

root@tirion:/etc/consul.d# ps -ef | grep [c]onsul
root     24034 24003  0 09:56 pts/0    00:00:24 consul agent -dev -config-dir=/etc/consul.d
root@tirion:/etc/consul.d# kill -l | grep SIGHUP
 1) SIGHUP	 2) SIGINT	 3) SIGQUIT	 4) SIGILL	 5) SIGTRAP
root@tirion:/etc/consul.d# kill -1 24034

    .
    .
    .
    2019/03/19 10:58:30 [DEBUG] agent: Node info in sync
    2019/03/19 10:59:13 [DEBUG] consul: Skipping self join check for "tirion" since the cluster is too small
    2019/03/19 10:59:27 [DEBUG] manager: Rebalanced 1 servers, next active server is tirion.dc1 (Addr: tcp/127.0.0.1:8300) (DC: dc1)
    2019/03/19 10:59:37 [INFO] agent: Caught signal:  hangup
    2019/03/19 10:59:37 [INFO] agent: Reloading configuration...
    2019/03/19 10:59:37 [DEBUG] agent: removed service "web"
    2019/03/19 10:59:37 [INFO] agent: Synced service "web"
    2019/03/19 10:59:37 [DEBUG] agent: Node info in sync
    2019/03/19 10:59:37 [DEBUG] agent: Service "web" in sync
    2019/03/19 10:59:37 [DEBUG] agent: Node info in sync

Connections

To do this demonstration, you'll need to install Linux socat. We're going to use it as a service to demonstrate communications with Consul.

russ@tirion:~$ which socat
russ@tirion:~$ sudo apt-get install socat
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  socat
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 342 kB of archives.
After this operation, 1,034 kB of additional disk space will be used.
Get:1 http://mirrors.xmission.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB]
Fetched 342 kB in 1s (651 kB/s)
Selecting previously unselected package socat.
(Reading database ... 308541 files and directories currently installed.)
Preparing to unpack .../socat_1.7.3.2-2ubuntu2_amd64.deb ...
Unpacking socat (1.7.3.2-2ubuntu2) ...
Setting up socat (1.7.3.2-2ubuntu2) ...
Processing triggers for doc-base (0.10.8) ...
Processing 1 added doc-base file...
Registering documents with scrollkeeper...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

socat will accept TCP connections and echo back any data sent on them. Here's what happens when we run socat and talk to it using UNIX nc, a utility that makes arbitrary TCP and UDP connections, then listens to them. Using nc, we type a greeting to socat which is echoed back to us. In socat's console, we see it receiving and echoing the greeting:

Console on which we launch socat:
russ@tirion:~$ socat -v tcp-l:8181,fork exec:"/bin/cat"
> 2019/03/19 11:08:02.857124  length=13 from=0 to=12
Hello socat!
< 2019/03/19 11:08:02.857313  length=13 from=0 to=12
Hello socat!
Second console using nc to send greeting:
russ@tirion:~$ nc 127.0.0.1 8181
Hello socat!
Hello socat!

Registering the service with Consul and connecting. First, let's create a configuration file, socat.json, from the command line:

root@tirion:/etc/consul.d# cat <<EOF | tee /etc/consul.d/socat.json
> {
>   "service" :
>   {
>     "name" : "socat",
>     "port" : 8181,
>     "connect" : { "sidecar_service" : {} }
>   }
> }
> EOF
{
  "service" :
  {
    "name" : "socat",
    "port" : 8181,
    "connect" : { "sidecar_service" : {} }
  }
}

Then, with this new (could have been a changed) service, we reload Consul either by sending SIGHUP as we did before or by using a Consul command forcing it to re-read configuration:

root@tirion:/etc/consul.d# consul reload
Configuration reload triggered

(Be sure to kill nc in that other console because if you do not, you'll get this in Consul's output:

    2019/03/19 11:22:50 [WARN] agent: Check "service:socat-sidecar-proxy:1" socket connection failed: dial tcp 127.0.0.1:21000: \
    connect: connection refused

As noted in the getting-started guide, the sidecar service registration is just telling Consol that a proxy is/will be running, but Consol doesn't run it for you. Here's the command which uses the socat configuration we just set up. After doing this, you'll see happy messages in Consul's output.

russ@tirion:~$ consul connect proxy -sidecar-for socat
==> Consul Connect proxy starting...
    Configuration mode: Agent API
        Sidecar for ID: socat
              Proxy ID: socat-sidecar-proxy

==> Log data will now stream in as it occurs:

    2019/03/19 11:25:14 [INFO] Proxy loaded config and ready to serve
    2019/03/19 11:25:14 [INFO] TLS Identity: spiffe://88f508f8-3a3b-ee7d-8c32-3c4942baa537.consul/ns/default/dc/dc1/svc/socat
    2019/03/19 11:25:14 [INFO] TLS Roots   : [Consul CA 7]
    2019/03/19 11:25:14 [INFO] public listener starting on 0.0.0.0:21000

Now, let's connect the web service (we first set up) with socat (set up second):

russ@tirion:~$ consul connect proxy -service web -upstream socat:9191
==> Consul Connect proxy starting...
    Configuration mode: Flags
               Service: web
              Upstream: socat => :9191
       Public listener: Disabled

==> Log data will now stream in as it occurs:

    2019/03/19 11:46:36 [INFO] 127.0.0.1:9191->service:default/socat starting on 127.0.0.1:9191
    2019/03/19 11:46:36 [INFO] Proxy loaded config and ready to serve
    2019/03/19 11:46:36 [INFO] TLS Identity: spiffe://88f508f8-3a3b-ee7d-8c32-3c4942baa537.consul/ns/default/dc/dc1/svc/web
    2019/03/19 11:46:36 [INFO] TLS Roots   : [Consul CA 7]

This part of the demonstration is very unclear. socat is echoing, but it's uncertain what port number, 8181 or 9191, this happens on. This whole connection section is very muddy because it doesn't say what executables to terminate before proceeding to the next demonstration and which ones to leave up and running. I end up with plenty of pending (running) commands to socat, nc and consul, errors coming out of the original consul instance and others as well, perhaps because this or that new console should have been terminated before moving on.