Software
Etcd + Fleet 3-Node Cluster
We're going to extend the my previous post and combine it with CoreOS example deployment.
Here's my example cloud-config, but you'll need to customise it for you're own purpose.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #cloud-config | |
| coreos: | |
| etcd: | |
| #generate a new token for each unique cluster from https://discovery.etcd.io/new | |
| discovery: https://discovery.etcd.io/<token> | |
| addr: $public_ipv4:4001 | |
| peer-addr: $private_ipv4:7001 | |
| peer-election-timeout: 500 | |
| peer-heartbeat-interval: 100 | |
| update: | |
| group: stable | |
| reboot-strategy: best-effort | |
| units: | |
| - name: etcd.service | |
| command: start | |
| - name: fleet.service | |
| command: start | |
| enable: true | |
| content: | | |
| [Unit] | |
| Description=fleet | |
| [Service] | |
| Environment="FLEET_PUBLIC_IP={{ public_ip }}" | |
| ExecStart=/usr/bin/fleet | |
| - name: docker-tcp.socket | |
| command: start | |
| enable: true | |
| content: | | |
| [Unit] | |
| Description=Docker Socket for the API | |
| [Socket] | |
| ListenStream=2375 | |
| Service=docker.service | |
| BindIPv6Only=both | |
| [Install] | |
| WantedBy=sockets.target | |
| - name: project.backend.@.service | |
| command: start | |
| content: | | |
| [Unit] | |
| Description=Project backend | |
| After=docker.service | |
| Requires=docker.service | |
| [Service] | |
| TimeoutStartSec=0 | |
| ExecStartPre=-/usr/bin/docker kill project | |
| ExecStartPre=-/usr/bin/docker rm project | |
| ExecStartPre=/usr/bin/docker pull hackzilla/project:latest | |
| ExecStart=/usr/bin/docker run --rm --name project -p 81:80 -e PROJECT_PASSWORD=letmein hackzilla/project:latest | |
| ExecStop=/usr/bin/docker stop project | |
| [X-Fleet] | |
| X-Conflicts=project.backend.*.service | |
| - name: project-discovery.backend.@.service | |
| command: start | |
| content: | | |
| [Unit] | |
| Description=Announce Project backend | |
| BindTo=project.backend.@.service | |
| [Service] | |
| ExecStart=/bin/sh -c "while true; do etcdctl set /varnish/backends/project/%H:81 '{ \"host\": \"%H\", \"port\": 81, \"version\": \"ooh\" }' --ttl 60;sleep 45;done" | |
| ExecStop=/usr/bin/etcdctl rm /varnish/project/kohana/%H:81 | |
| [X-Fleet] | |
| X-ConditionMachineOf=project.backend.@.service | |
| ssh_authorized_keys: | |
| - ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key | |
| - ssh-rsa AAAAB... hackzilla@hackzilla.org | |
| users: | |
| - name: hackzilla | |
| passwd: $6$WQ74... | |
| groups: | |
| - sudo | |
| - docker | |
| - wheel | |
| - portage | |
| - core | |
| ssh-authorized-keys: | |
| - ssh-rsa AAAAB... |
Once you've created your cluster, you should verify that etcd is correctly setup and you can do that with etcdctl command.
$ etcdctl ls --recursive /varnish /varnish/backends /varnish/backends/project /varnish/backends/project/node1:81 /varnish/backends/project/node2:81 /varnish/backends/project/node3:81
Now that etcd is correctly setup, we can move onto fleet. Fleet uses etcd to communicate with the other nodes.
We need submit the 2 services to fleet.
$ fleetctl submit project.backend.@.service project-discovery.backend.@.service $ fleetctl list-unit-files UNIT HASH DSTATE STATE TMACHINE project.backend... xxxxxxx inactive inactive - project-discovery... yyyyyyy inactive inactive -
And to start the services in fleet:
$ fleetctl start project-discovery.backend.@{1..3}.service project.backend.@{1..3}.service
Unit project.backend.@1.service launched on 0e0a1f59.../192.168.0.50
Unit project.backend.@2.service launched on 30a73182.../192.168.0.51
Unit project-discovery.backend.@1.service launched on 0e0a1f59.../192.168.0.50
Unit project-discovery.backend.@2.service launched on 30a73182.../192.168.0.51
Unit project.backend.@3.service launched on 610a163d.../192.168.0.52
Unit project-discovery.backend.@3.service launched on 610a163d.../192.168.0.52
$ fleetctl list-units
UNIT MACHINE ACTIVE SUB
project.backend.@1.service 0e0a1f59.../192.168.0.50 active running
project.backend.@2.service 30a73182.../192.168.0.51 active running
project.backend.@3.service 610a163d.../192.168.0.52 active running
project-discovery.backend.@1.service 0e0a1f59.../192.168.0.50 active running
project-discovery.backend.@2.service 30a73182.../192.168.0.51 active running
project-discovery.backend.@3.service 610a163d.../192.168.0.52 active running
Now that fleet is running the services, any additional nodes you add to the same etcd network will pick up these services, but only 3 will run at any one time.
In the next post, I'll show you how to automatically take advantage of these nodes.