feat(haproxy): add a lot to haproxy, and generate_certs playbook + docs

This commit is contained in:
Bertrand Lanson 2024-04-01 23:59:48 +02:00
parent 5204358a20
commit 8016d19efe
15 changed files with 462 additions and 216 deletions

3
.gitignore vendored
View File

@ -3,4 +3,5 @@
.vscode
roles/ednz_cloud.*
vault_config.yml
consul_config.yml
consul_config.yml
**/certificates/**

104
docs/haproxy_servers.md Normal file
View File

@ -0,0 +1,104 @@
# Deploying HAProxy frontends
This documentation explains each steps necessary to successfully deploy HAProxy frontends for your deployment, using the ednz_cloud.hashistack ansible collection.
## Prerequisites
You should, before attempting any deployment, have read through the [Quick Start Guide](./quick_start.md). These steps are necessary in order to ensure smooth operations going forward.
## Variables
### Basics
First, in order to deploy the HAproxy frontends, you need to enable the deployment.
```yaml
enable_haproxy: "yes"
```
You can also configure the version of haproxy you would like to use. This has very little impact, and should most likely be left outouched to whatever the collection version is defaulting to (which is the version that it is tested against).
```yaml
haproxy_version: "2.8"
```
This version can either be `latest`, or any `X`, `X.Y`, `X.Y.Z` tag of the [haproxytech/haproxy-debian](https://hub.docker.com/r/haproxytech/haproxy-debian) docker image.
For production deployment, it is recommended to use the `X.Y.Z` syntax.
The `deployment_method` variable will define how to install vault on the nodes.
By default, it runs haproxy inside a docker container, but this can be changed to `host` to install haproxy from the package manager.
Note that not all versions of haproxy are available as a package on all supported distributions. Please refer to the documentation of [ednz_cloud.deploy_haproxy](https://github.com/ednz-cloud/deploy_haproxy) for details about supported versions when installing from the package manager.
```yaml
deployment_method: "docker"
```
### General Settings
There aren't many settings that you can configure to deploy the HAProxy frontends. First you'll need to configure a Virtual IP, and pass it in the `globals.yml` configuration file.
```yaml
hashistack_external_vip_interface: "eth0"
hashistack_external_vip_addr: "192.168.121.100"
```
This is used to configure keepalived to automatically configure this VIP on one of the frontend, and handle failover.
You also need to configure the names that will resolve to your different applications (consul, nomad, vault). These names should resolve to your Virtual IP, and will be used to handle host-based routing on haproxy.
```yaml
consul_fqdn: consul.ednz.lab
vault_fqdn: vault.ednz.lab
nomad_fqdn: nomad.ednz.lab
```
With this configuration, querying `http://consul.ednz.lab` will give you the consul UI and API, through haproxy.
> Note: subpaths are not yet supported, so you cannot set the fqdn to `generic.domain.tld/consul`. This feature will be added in a future release.
### Enabling external TLS
To enable external TLS for your APIs and UIs, you will need to set the following variable.
```yaml
enable_tls_external: true
```
This will enable the https listener for haproxy and configure the http listener to be a https redirect only.
## Managing external TLS certificates
### Generating certificates with hashistack-ansible
If you don't care about having trusted certificates (e.g. for developement or testing purposes), you can generate some self-signed certificates for your applications using the `generate_certs.yml` playbook.
```bash
ansible-playbook -i multinode.ini ednz_cloud.hashistack.generate_certs.yml
```
This will generate self-signed certificates for each applications that has been enabled in your `globals.yml`, and for then respective fqdn (also configured in `globals.yml`).
These certificates are going to be placed in `etc/hashistack/certificates/external/`, and will be named after each fqdn. These files should be encrypted using something like ansible-vault, as they are sensitive.
### Managing your own TLS certificates
Similarly, you can manage your own TLS certificates, signed by your own CA. Your certificates should be placed in the `etc/hashistack/certificates/external/` directory, similar to the self-signed ones, and be named like:
`<your_fqdn>.pem` and `<your_fqdn>.pem.key`, for each application.
At the moment, setting all certificates in a single file is not supported, but will be added in a later release.
These certificates will be copied over to the `haproxy_servers` hosts, in `/var/lib/haproxy/certs/`.
### Managing certificates externally
In case you already have systems in place to deploy and renew your certificates, you can simply enable the options in `globals.yml` to not manage certificates directly in hashistack-ansible.
```yaml
external_tls_externally_managed_certs: true
```
Enabling this option will prevents the playbooks from trying to copy certificates over, but the HAProxy nodes will still expect them to be present. It is up to you to copy them over.

View File

@ -1,3 +1,6 @@
---
# - name: Include a playbook from a collection
# ansible.builtin.import_playbook: ednz_cloud.hashistack.generate_certs.yml
- name: Include a playbook from a collection
ansible.builtin.import_playbook: ednz_cloud.hashistack.deploy.yml

View File

@ -16,6 +16,17 @@
state: present
when: "'vault_servers' in group_names"
- name: "Install python-consul library with pip"
ansible.builtin.include_role:
name: ednz_cloud.manage_pip_packages
vars:
manage_pip_packages_install_prereqs: true
manage_pip_packages_list:
- name: python-consul
version_constraint: latest
state: present
when: "'haproxy_servers' in group_names"
- name: "Include ednz_cloud.install_docker"
ansible.builtin.include_role:
name: ednz_cloud.install_docker

View File

@ -10,15 +10,6 @@
ansible.builtin.import_tasks:
file: tasks/load_vars.yml
- name: "Deploy Haproxy & Keepalived"
ansible.builtin.import_tasks:
file: tasks/haproxy/haproxy_deploy.yml
when:
- enable_haproxy | bool
- "'haproxy_servers' in group_names"
tags:
- haproxy
- name: "Deploy Consul Control Plane"
ansible.builtin.import_tasks:
file: tasks/consul/consul_deploy.yml
@ -37,6 +28,15 @@
tags:
- consul
- name: "Deploy Haproxy & Keepalived"
ansible.builtin.import_tasks:
file: tasks/haproxy/haproxy_deploy.yml
when:
- enable_haproxy | bool
- "'haproxy_servers' in group_names"
tags:
- haproxy
- name: "Deploy Vault"
ansible.builtin.import_tasks:
file: tasks/vault/vault_deploy.yml

View File

@ -0,0 +1,60 @@
---
# hashistack deployment playbook
- name: "Generate certificates"
hosts: all
strategy: linear
gather_facts: true
become: true
tasks:
- name: "Generate self-signed certificates" # noqa: run-once[task]
delegate_to: localhost
run_once: true
block:
- name: "Create temporary cert directory in {{ sub_configuration_directories['certificates'] }}"
ansible.builtin.file:
path: "{{ sub_configuration_directories['certificates'] }}/external"
state: directory
owner: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
mode: "0755"
- name: "Generate self-signed certificate"
block:
- name: "Create private keys"
community.crypto.openssl_privatekey:
path: "{{ sub_configuration_directories['certificates'] }}/external/{{ item.fqdn }}.pem.key"
owner: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
loop:
- name: nomad
fqdn: "{{ nomad_fqdn }}"
- name: vault
fqdn: "{{ vault_fqdn }}"
- name: consul
fqdn: "{{ consul_fqdn }}"
- name: "Create certificate signing request"
community.crypto.openssl_csr_pipe:
privatekey_path: "{{ sub_configuration_directories['certificates'] }}/external/{{ item.fqdn }}.pem.key"
common_name: "{{ item.fqdn }}"
organization_name: Ansible, Inc.
register: csr
loop:
- name: nomad
fqdn: "{{ nomad_fqdn }}"
- name: vault
fqdn: "{{ vault_fqdn }}"
- name: consul
fqdn: "{{ consul_fqdn }}"
- name: "Create self-signed certificate from CSR"
community.crypto.x509_certificate:
path: "{{ sub_configuration_directories['certificates'] }}/external/{{ item.item.fqdn }}.pem"
csr_content: "{{ item.csr }}"
privatekey_path: "{{ sub_configuration_directories['certificates'] }}/external/{{ item.item.fqdn }}.pem.key"
provider: selfsigned
owner: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
loop: "{{ csr.results }}"
- fail:

View File

@ -1,37 +1,7 @@
---
##########################
# General options ########
##########################
enable_haproxy: "yes"
enable_vault: "no"
enable_consul: "yes"
enable_nomad: "no"
nomad_version: latest
consul_version: latest
vault_version: latest
deployment_method: "docker"
consul_fqdn: consul.ednz.lab
vault_fqdn: vault.ednz.lab
nomad_fqdn: nomad.ednz.lab
hashistack_external_vip_interface: "eth0"
hashistack_external_vip_addr: "192.168.121.100"
hashistack_internal_vip_interface: "eth1"
# hashistack_internal_vip_interface: "{{ hashistack_external_vip_interface }}"
hashistack_internal_vip_addr: "192.168.100.100"
# hashistack_internal_vip_addr: "{{ hashistack_external_vip_addr }}"
# api_interface: "eth0"
api_interface: "eth1"
api_interface_address: "{{ ansible_facts[api_interface]['ipv4']['address'] }}"
##########################
# Helper options #########
##########################
##################
# helper options #
##################
# manage_pip_packages_allow_break_system_packages: true
@ -49,6 +19,7 @@ nomad_versions:
configuration_directory: "{{ lookup('env', 'PWD') }}/etc/hashistack"
sub_configuration_directories:
certificates: "{{ configuration_directory }}/certificates"
nomad_servers: "{{ configuration_directory }}/nomad_servers"
vault_servers: "{{ configuration_directory }}/vault_servers"
consul_servers: "{{ configuration_directory }}/consul_servers"
@ -59,9 +30,9 @@ default_container_extra_volumes:
- "/etc/timezone:/etc/timezone"
- "/etc/localtime:/etc/localtime"
#################
# Support options
#################
###################
# support options #
################# #
hashistack_supported_distributions:
- ubuntu

View File

@ -1,69 +1,4 @@
---
#####################################################
# #
# Editable #
# #
#####################################################
consul_domain: consul
consul_datacenter: dc1
consul_primary_datacenter: dc1
consul_leave_on_terminate: true
consul_rejoin_after_leave: true
consul_enable_script_checks: true
consul_gossip_encryption_key: "{{ 'mysupersecretgossipencryptionkey'|b64encode }}"
################################
# consul address configuration #
################################
consul_address_configuration:
# The address to which Consul will bind client interfaces,
# including the HTTP and DNS servers.
client_addr: "0.0.0.0"
# The address that should be bound to for internal cluster communications.
bind_addr: "{{ api_interface_address }}"
# The advertise address is used to change the address that we advertise to other nodes in the cluster.
advertise_addr: "{{ api_interface_address }}"
############################
# consul ACL configuration #
############################
consul_acl_configuration:
enabled: true
default_policy: "deny" # can be allow or deny
enable_token_persistence: true
############################
# consul DNS configuration #
############################
consul_dns_configuration:
allow_stale: true
enable_truncate: true
only_passing: true
###########################
# consul ui configuration #
###########################
consul_ui_configuration:
enabled: "{{ 'consul_servers' in group_names }}"
#####################################
# consul service mesh configuration #
#####################################
consul_mesh_configuration:
enabled: true
#######################
# extra configuration #
#######################
consul_extra_configuration: {}
#####################################################
# #
# Non-Editable #
@ -74,14 +9,9 @@ consul_extra_configuration: {}
# consul haproxy backend #
##########################
consul_haproxy_frontends:
- name: consul_external
options:
- description consul external http frontend
- mode http
- bind :80
- acl is_consul hdr(host) -i {{ consul_fqdn }}
- use_backend consul_external if is_consul
consul_haproxy_frontend_options:
- acl is_consul hdr(host) -i {{ consul_fqdn }}
- use_backend consul_external if is_consul
consul_haproxy_backends:
- name: consul_external
@ -92,11 +22,12 @@ consul_external_backend_options:
- option forwardfor
- option httpchk
- http-check send meth GET uri /
- default-server inter 2s fastinter 1s downinter 1s
consul_external_backend_servers: |
[
{% for host in groups['consul_servers'] %}
'server {{ hostvars[host].api_interface_address }} {{ hostvars[host].api_interface_address }}:{{ hashi_consul_configuration.ports.http }} check inter 5s'{% if not loop.last %},{% endif %}
'server consul-{{ hostvars[host].api_interface_address }} {{ hostvars[host].api_interface_address }}:{{ hashi_consul_configuration.ports.http }} check inter 5s'{% if not loop.last %},{% endif %}
{% endfor %}
]
@ -105,9 +36,6 @@ consul_external_backend_servers: |
############################
consul_default_agent_policy: |
agent_prefix "" {
policy = "write"
}
node_prefix "" {
policy = "write"
}

View File

@ -0,0 +1,191 @@
---
##########################
# General options ########
##########################
enable_haproxy: "yes"
enable_vault: "yes"
enable_consul: "yes"
enable_nomad: "no"
haproxy_version: "2.8"
nomad_version: "latest"
consul_version: "latest"
vault_version: "latest"
deployment_method: "docker"
consul_fqdn: consul.ednz.lab
vault_fqdn: vault.ednz.lab
nomad_fqdn: nomad.ednz.lab
hashistack_external_vip_interface: "eth0"
hashistack_external_vip_addr: "192.168.121.100"
hashistack_internal_vip_interface: "eth1"
# hashistack_internal_vip_interface: "{{ hashistack_external_vip_interface }}"
hashistack_internal_vip_addr: "192.168.100.100"
# hashistack_internal_vip_addr: "{{ hashistack_external_vip_addr }}"
# api_interface: "eth0"
api_interface: "eth1"
api_interface_address: "{{ ansible_facts[api_interface]['ipv4']['address'] }}"
########################
# external tls options #
########################
enable_tls_external: true
external_tls_externally_managed_certs: false
#####################################################
# #
# Consul #
# #
#####################################################
consul_domain: consul
consul_datacenter: dc1
consul_primary_datacenter: dc1
consul_leave_on_terminate: true
consul_rejoin_after_leave: true
consul_enable_script_checks: true
consul_gossip_encryption_key: "{{ 'mysupersecretgossipencryptionkey'|b64encode }}"
################################
# consul address configuration #
################################
consul_address_configuration:
# The address to which Consul will bind client interfaces,
# including the HTTP and DNS servers.
client_addr: "0.0.0.0"
# The address that should be bound to for internal cluster communications.
bind_addr: "{{ api_interface_address }}"
# The advertise address is used to change the address that we advertise to other nodes in the cluster.
advertise_addr: "{{ api_interface_address }}"
############################
# consul ACL configuration #
############################
consul_acl_configuration:
enabled: true
default_policy: "deny" # can be allow or deny
enable_token_persistence: true
############################
# consul DNS configuration #
############################
consul_dns_configuration:
allow_stale: true
enable_truncate: true
only_passing: true
###########################
# consul ui configuration #
###########################
consul_ui_configuration:
enabled: "{{ 'consul_servers' in group_names }}"
#####################################
# consul service mesh configuration #
#####################################
consul_mesh_configuration:
enabled: true
#######################
# extra configuration #
#######################
consul_extra_configuration: {}
#####################################################
# #
# Vault #
# #
#####################################################
vault_cluster_name: vault
vault_enable_ui: true
vault_seal_configuration:
key_shares: 3
key_threshold: 2
#################
# vault storage #
#################
vault_storage_configuration:
raft:
path: "{{ hashi_vault_data_dir }}/data"
node_id: "{{ ansible_hostname }}"
retry_join: |
[
{% for host in groups['vault_servers'] %}
{
'leader_api_addr': 'http://{{ hostvars[host].api_interface_address }}:8200'
}{% if not loop.last %},{% endif %}
{% endfor %}
]
##################
# vault listener #
##################
vault_enable_tls: false
vault_listener_configuration:
tcp:
address: "0.0.0.0:8200"
tls_disable: true
vault_tls_listener_configuration:
tcp:
tls_disable: false
tls_cert_file: "{{ hashi_vault_extra_files_dst }}/tls/cert.pem"
tls_key_file: "{{ hashi_vault_extra_files_dst }}/tls/key.pem"
vault_extra_listener_configuration: {}
########################
# service registration #
########################
vault_enable_service_registration: false
vault_service_registration_configuration:
consul:
address: "127.0.0.1:8500"
scheme: "http"
token: ""
#################
# vault plugins #
#################
vault_enable_plugins: true
vault_plugin_directory: "{{ hashi_vault_extra_files_dst }}/plugin"
###########
# logging #
###########
vault_enable_log_to_file: false
vault_logging_configuration:
log_level: info
log_format: standard
log_rotate_duration: 24h
log_rotate_max_files: 30
###########################
# vault container volumes #
###########################
extra_vault_container_volumes: []
#####################
# extra configuration
#####################
vault_extra_configuration: {}

View File

@ -6,11 +6,11 @@
#####################################################
deploy_haproxy_deploy_method: "{{ deployment_method }}"
deploy_haproxy_version: "2.8"
deploy_haproxy_version: "{{ haproxy_version }}"
deploy_haproxy_env_variables: {}
deploy_haproxy_start_service: true
deploy_haproxy_cert_dir: ""
deploy_haproxy_cert_dir: "{{ sub_configuration_directories['certificates']~'/external' if (enable_tls_external and not external_tls_externally_managed_certs) }}"
deploy_haproxy_extra_container_volumes: []
deploy_haproxy_global:
- log /dev/log local0
@ -29,9 +29,31 @@ deploy_haproxy_defaults:
- timeout client 5000
- timeout server 5000
deploy_haproxy_frontends: "{{ consul_haproxy_frontends }}"
deploy_haproxy_frontends:
- name: external_http
options: >-
{%- set haproxy_options = [
'description hashistack external http frontend',
'mode http',
'bind :80'
] -%}
deploy_haproxy_backends: "{{ consul_haproxy_backends }}"
{%- if enable_tls_external -%}
{%- set tls_cert_paths = [] -%}
{%- for item in ['consul', 'nomad', 'vault'] if vars['enable_' + item] | bool -%}
{%- set crt_option = '/var/lib/haproxy/certs/' + vars[item + '_fqdn'] + '.pem' -%}
{%- set _ = tls_cert_paths.append(crt_option) -%}
{%- endfor -%}
{%- set tls_options = ['bind :443 ssl crt ' + tls_cert_paths | join(' crt ') ] -%}
{%- set _ = tls_options.append('http-request redirect scheme https unless { ssl_fc }') -%}
{%- set haproxy_options = haproxy_options + tls_options -%}
{%- endif -%}
{%- set haproxy_options = haproxy_options + consul_haproxy_frontend_options + vault_haproxy_frontend_options -%}
{{ haproxy_options }}
deploy_haproxy_backends: "{{ consul_haproxy_backends + vault_haproxy_backends }}"
deploy_haproxy_listen:
- name: monitoring
@ -67,7 +89,6 @@ deploy_keepalived_unicast_peers: "{{ groups['haproxy_servers'] | difference([ans
deploy_keepalived_auth_passwd: "password"
deploy_keepalived_virtual_ips:
- "{{ hashistack_external_vip_addr }}/32 dev {{ hashistack_external_vip_interface }}"
# - "{{ hashistack_internal_vip_addr }}/32 dev {{ hashistack_internal_vip_interface }}"
deploy_keepalived_notify_script: notify.sh
deploy_keepalived_custom_scripts_src: tasks/haproxy/files/keepalived/scripts.d

View File

@ -1,95 +1,39 @@
---
#####################################################
# #
# Vault Configuration #
# Non-Editable #
# #
#####################################################
vault_cluster_name: vault
vault_enable_ui: true
vault_seal_configuration:
key_shares: 3
key_threshold: 2
#########
# storage
#########
vault_storage_configuration:
raft:
path: "{{ hashi_vault_data_dir }}/data"
node_id: "{{ ansible_hostname }}"
retry_join: |
[
{% for host in groups['vault_servers'] %}
{
'leader_api_addr': 'http://{{ hostvars[host].api_interface_address }}:8200'
}{% if not loop.last %},{% endif %}
{% endfor %}
]
##########
# listener
##########
vault_enable_tls: false
vault_listener_configuration:
tcp:
address: "0.0.0.0:8200"
tls_disable: true
vault_tls_listener_configuration:
tcp:
tls_disable: false
tls_cert_file: "{{ hashi_vault_extra_files_dst }}/tls/cert.pem"
tls_key_file: "{{ hashi_vault_extra_files_dst }}/tls/key.pem"
vault_extra_listener_configuration: {}
######################
# service registration
######################
vault_enable_service_registration: false
vault_service_registration_configuration:
consul:
address: "127.0.0.1:8500"
scheme: "http"
token: ""
#########
# plugins
#########
vault_enable_plugins: true
vault_plugin_directory: "{{ hashi_vault_extra_files_dst }}/plugin"
#########
# logging
#########
vault_enable_log_to_file: false
vault_logging_configuration:
log_level: info
log_format: standard
log_rotate_duration: 24h
log_rotate_max_files: 30
#########################
# vault container volumes
# vault haproxy backend #
#########################
extra_vault_container_volumes: []
vault_haproxy_frontend_options:
- acl is_vault hdr(host) -i {{ vault_fqdn }}
- use_backend vault_external if is_vault
#####################
# extra configuration
#####################
vault_haproxy_backends:
- name: vault_external
options: "{{ vault_external_backend_options + vault_external_backend_servers }}"
vault_extra_configuration: {}
vault_external_backend_options:
- description vault external http backend
- option forwardfor
- option httpchk GET /v1/sys/health?standbyok=true&sealedcode=200&standbycode=200&uninitcode=200
- http-check expect status 200
- default-server inter 2s fastinter 1s downinter 1s
###############
# configuration
###############
vault_external_backend_servers: |
[
{% for host in groups['vault_servers'] %}
'server vault-{{ hostvars[host].api_interface_address }} {{ hostvars[host].api_interface_address }}:8200 check inter 5s'{% if not loop.last %},{% endif %}
{% endfor %}
]
########################
# vault role variables #
########################
hashi_vault_start_service: true
hashi_vault_version: "{{ vault_versions[deployment_method] }}"

View File

@ -1,3 +1,4 @@
---
- name: "Consul"
block:
- name: "Include ednz_cloud.hashistack.hashicorp_consul"
@ -5,10 +6,16 @@
name: ednz_cloud.hashistack.hashicorp_consul
- name: "Wait for consul cluster to initialize" # noqa: run-once[task]
ansible.builtin.wait_for:
timeout: 15
delegate_to: localhost
run_once: true
ansible.builtin.uri:
url: "http://{{ api_interface_address }}:8500"
validate_certs: no
return_content: yes
status_code:
- 200
until: uri_output.status == 200
retries: 24
delay: 5
register: uri_output
- name: "Initialize consul cluster" # noqa: run-once[task]
community.general.consul_acl_bootstrap:
@ -18,10 +25,7 @@
state: present
run_once: true
delegate_to: "{{ groups['consul_servers'] | first }}"
# retries: 5
# delay: 5
register: _consul_init_secret
# until: _consul_init_secret.result is defined
when: hashi_consul_configuration.acl.enabled
- name: "Write consul configuration to file" # noqa: run-once[task] no-handler

View File

@ -1,3 +1,4 @@
---
- name: "HAProxy"
block:
- name: "Include ednz_cloud.deploy_haproxy"
@ -11,6 +12,11 @@
- name: "Register haproxy services in consul"
when: enable_consul | bool
block:
- name: "Load consul cluster variables"
ansible.builtin.include_vars:
file: "{{ sub_configuration_directories.consul_servers }}/consul_config.yml"
name: _consul_cluster_config
- name: "Register haproxy services in consul"
community.general.consul:
token: "{{ _consul_cluster_config.root_token.secret_id }}"

View File

@ -1,3 +1,4 @@
---
- name: "Vault"
block:
- name: "Include ednz_cloud.hashistack.hashicorp_consul"
@ -47,4 +48,5 @@
key_shares: "{{ _vault_cluster_config['keys'] }}"
retries: 5
delay: 5
until: _unseal_status.changed or not _unseal_status.failed
register: _unseal_status

View File

@ -116,7 +116,7 @@ def run_module():
api_url=module.params["api_url"], key_shares=module.params["key_shares"]
)
if client.sys.is_sealed():
if hvac.Client(url=module.params["api_url"]).sys.is_sealed():
module.fail_json(
msg="Vault unsealing failed. The unseal operation worked, but the vault is still sealed, maybe you didn't pass enough keys ?"
)