Compare commits

..

10 Commits

Author SHA1 Message Date
b1a0ccdf3e Merge pull request 'feat/nomad-deployment' (#10) from feat/nomad-deployment into main
All checks were successful
build-deploy / Bump version and create changelog with commitizen (push) Successful in 7s
Reviewed-on: ansible-collections/hashistack#10
2024-07-10 17:36:41 +00:00
ed27759dcd
feat: add barebone driver options for nomad (not fully working)
All checks were successful
development / Check commit compliance (push) Successful in 26s
pull-requests-open / Check commit compliance (pull_request) Successful in 26s
2024-07-10 19:32:59 +02:00
48a7724759
docs: add documentation on running preflight and bootstrap playbooks
All checks were successful
development / Check commit compliance (push) Successful in 6s
2024-07-03 22:37:00 +02:00
042e3593ca
chore: markdown syntax to get appropriate style in docs
All checks were successful
development / Check commit compliance (push) Successful in 6s
2024-07-03 22:21:31 +02:00
9417437715
feat: add docs on generating credentials
All checks were successful
development / Check commit compliance (push) Successful in 6s
2024-07-03 22:17:57 +02:00
fd9a0e3c55
fix: implement longer wait to stabilize consul cluster before bootstrapping to avoid timeout errors
All checks were successful
development / Check commit compliance (push) Successful in 30s
2024-07-03 21:43:14 +02:00
074da0289a
feat(nomad): move variables to globals.yml, adjust bootstrap module for nomad ACLs
All checks were successful
development / Check commit compliance (push) Successful in 6s
2024-07-02 23:06:16 +02:00
95a1d80f76
feat: add nomad deployment options, variables, and playbooks
All checks were successful
development / Check commit compliance (push) Successful in 26s
2024-06-25 15:16:42 +02:00
b32815066f
feat: use new vault and consul roles, and only allow for host deployment after docker support drop
All checks were successful
development / Check commit compliance (push) Successful in 1m33s
2024-06-20 00:32:33 +02:00
08909ceed0 feat: add global variables for nomad deployment
All checks were successful
development / Check commit compliance (push) Successful in 28s
2024-05-16 17:29:41 +02:00
40 changed files with 1173 additions and 234 deletions

6
.gitmodules vendored
View File

@ -1,6 +0,0 @@
[submodule "roles/hashicorp_consul"]
path = roles/hashicorp_consul
url = https://github.com/ednz-cloud/hashicorp_consul
[submodule "roles/hashicorp_vault"]
path = roles/hashicorp_vault
url = https://github.com/ednz-cloud/hashicorp_vault

View File

@ -0,0 +1 @@
# Adding extra configuration options

View File

@ -35,7 +35,7 @@ Note that not all versions of haproxy are available as a package on all supporte
deployment_method: "docker"
```
### General Settings
### General settings
There aren't many settings that you can configure to deploy the HAProxy frontends. First you'll need to configure a Virtual IP, and pass it in the `globals.yml` configuration file.

82
docs/nomad_clusters.md Normal file
View File

@ -0,0 +1,82 @@
# Deploying a Nomad cluster
This documentation explains each steps necessary to successfully deploy a Nomad cluster using the ednz_cloud.hashistack ansible collection.
## Prerequisites
You should, before attempting any deployment, have read through the [Quick Start Guide](./quick_start.md). These steps are necessary in order to ensure smooth operations going forward.
## Variables
### Basics
First, in order to deploy a nomad cluster, you need to enable it.
```yaml
enable_nomad: "yes"
```
Selecting the nomad version to install is done with the `nomad_version` variable.
```yaml
nomad_version: latest
```
The vault version can either be `latest` or `X.Y.Z`.
For production deployment, it is recommended to use the `X.Y.Z` syntax.
### General settings
First, you can change some general settings for nomad, like the dc and region options.
```yaml
nomad_datacenter: dc1
nomad_region: global
```
### ACLs settings
By default, ACLs are enabled on nomad, and automatically bootstrapped.
You can change this by editing the `nomad_acl_configuration` variable:
```yaml
nomad_acl_configuration:
enabled: true
token_ttl: 30s
policy_ttl: 60s
role_ttl: 60s
```
### Consul integration settings
By default, if consul if also enabled, nomad will use it to register itself as a consul service and also use consul to automatically join the cluster.
```yaml
nomad_enable_consul_integration: "{{ enable_consul | bool }}"
nomad_consul_integration_configuration:
address: "127.0.0.1:{{ hashicorp_consul_configuration.ports.https if consul_enable_tls else hashicorp_consul_configuration.ports.http }}"
auto_advertise: true
ssl: "{{ consul_enable_tls | bool }}"
token: "{{ _credentials.consul.tokens.nomad.server.secret_id if nomad_enable_server else _credentials.consul.tokens.nomad.client.secret_id}}"
tags: []
```
Optionally, you can add tags to you nomad services, or disable the consul integration if you don't plan on using it.
### Vault integration settings
Vault integration for nomad is by default disabled, as it requires some vault configuration that is out of the scope of this collection.
You can, once you have deployed and configured vault (or if you are using an external vault not managed by the collection), enable the integration
```yaml
nomad_enable_vault_integration: false
nomad_vault_integration_configuration: {}
```
For configuration options, please refer to the [Official Documentation](https://developer.hashicorp.com/nomad/docs/configuration/vault)
### Drivers settings
### Internal TLS

View File

@ -60,7 +60,7 @@ source /path/to/venv/bin/activate
pip install -U pip
```
4. Install [Ansible](http://www.ansible.com/). Hashistack Ansible requires at least Ansible **7**(or ansible-core **2.15**)
4. Install [Ansible](http://www.ansible.com/). Hashistack-Ansible requires at least Ansible **7**(or ansible-core **2.15**)
```bash
pip install 'ansible-core>=2.15'
@ -111,3 +111,34 @@ ansible-galaxy install -r ./collections/ansible_collections/ednz_cloud/hashistac
This will install roles that are not packaged with the collection, but are still required in order to run the playbooks.
You should now have some roles inside `./roles/`.
## Generate Credentials
Before deploying your infrastructure with Hashistack-Ansible, you need to generate credentials that will be used to bootstrap the various clusters.
This can be done by running the `generate_credentials.yml` playbook.
```bash
ansible-playbook -i inventory/inventory.ini ednz_cloud.hashistack.generate_credentials.yml
```
This will create and populate `etc/hashistack/secrets/credentials.yml`
> [!WARNING]
> This file is VERY SENSITIVE, as it holds the root tokens and other credentials for consul and nomad clusters.
This does not generate vault credentials, as it is not possible to generate those in advance. These credentials will be generated, if you enable the vault deployment, during the bootstrap process of the vault cluster, and stored in `etc/hashistack/secrets/vault.yml`
> [!WARNING]
> It is HIGHLY recommended to encrypt these two files before enventually commiting them to source control. You can do so using tools like [ansible-vault](https://docs.ansible.com/ansible/latest/cli/ansible-vault.html) or [sops](https://github.com/getsops/sops).
## Running preflight checks and bootstrap playbooks
Before running the main deployment playbook, you might want to run the `bootstrap` and `preflight` playbooks, which do a number of checks to ensure all hosts are setup correctly for deployment.
```bash
ansible-playbook -i inventory/inventory.ini ednz_cloud.hashistack.bootstrap.yml
ansible-playbook -i inventory/inventory.ini ednz_cloud.hashistack.preflight.yml
```
These playbooks will run a number of checks, and installations, in order to ensure the target hosts, as well as your deployment environment are correctly setup in order to install all the components.

1
docs/tls_guide.md Normal file
View File

@ -0,0 +1 @@
# TLS Guide

View File

@ -26,15 +26,7 @@ The vault version can either be `latest` or `X.Y.Z`.
For production deployment, it is recommended to use the `X.Y.Z` syntax.
The `deployment_method` variable will define how to install vault on the nodes.
By default, it runs vault inside a docker container, but this can be changed to `host` to install vault from the package manager.
```yaml
deployment_method: "docker"
```
### General Settings
### General settings
First, you can change some general settings for vault.
@ -46,14 +38,14 @@ vault_seal_configuration:
key_threshold: 2
```
### Storage Settings
### Storage settings
The storage configuration for vault can be edited as well. By default, vault will be configured to setup `raft` storage between all declared vault servers (in the `vault_servers` group).
```yaml
vault_storage_configuration:
raft:
path: "{{ hashi_vault_data_dir }}/data"
path: "{{ hashicorp_vault_data_dir }}/data"
node_id: "{{ ansible_hostname }}"
retry_join: |
[
@ -79,9 +71,9 @@ vault_storage_configuration:
database: "vault"
```
### Listener Settings
### Listener settings
#### TCP Listeners
#### TCP listeners
By default, TLS is **disabled** for vault. This goes against the Hashicorp recommendations on the matter, but there is no simple way to force the use of TLS (yet), without adding a lot of complexity to the deployment.

View File

@ -6,15 +6,13 @@
# enable_haproxy: "yes"
# enable_vault: "yes"
# enable_consul: "yes"
enable_nomad: "no"
# enable_nomad: "yes"
# haproxy_version: "2.8"
# nomad_version: "1.7.7"
# nomad_version: "1.8.1"
# consul_version: "1.18.1"
# vault_version: "1.16.2"
# deployment_method: "docker"
# consul_fqdn: consul.ednz.lab
# vault_fqdn: vault.ednz.lab
# nomad_fqdn: nomad.ednz.lab
@ -114,9 +112,9 @@ consul_enable_tls: true
# extra_consul_container_volumes: []
#######################
# extra configuration #
#######################
##############################
# consul extra configuration #
##############################
# consul_extra_configuration: {}
# consul_extra_files_list: []
@ -139,7 +137,7 @@ consul_enable_tls: true
# vault_storage_configuration:
# raft:
# path: "{{ hashi_vault_data_dir }}/data"
# path: "{{ hashicorp_vault_data_dir }}/data"
# node_id: "{{ ansible_hostname }}"
# retry_join: |
# [
@ -204,9 +202,83 @@ vault_enable_tls: true
# extra_vault_container_volumes: []
#####################
# extra configuration
#####################
#############################
# vault extra configuration #
#############################
# vault_extra_configuration: {}
# vault_extra_files_list: []
#####################################################
# #
# Nomad #
# #
#####################################################
# nomad_datacenter: dc1
# nomad_region: global
###########################
# nomad ACL configuration #
###########################
# nomad_acl_configuration:
# enabled: true
# token_ttl: 30s
# policy_ttl: 60s
# role_ttl: 60s
############################
# nomad consul integration #
############################
# nomad_enable_consul_integration: "{{ enable_consul | bool }}"
# nomad_consul_integration_configuration:
# address: "127.0.0.1:{{ hashicorp_consul_configuration.ports.https if consul_enable_tls else hashicorp_consul_configuration.ports.http }}"
# auto_advertise: true
# ssl: "{{ consul_enable_tls | bool }}"
# token: "{{ _credentials.consul.tokens.nomad.server.secret_id if nomad_enable_server else _credentials.consul.tokens.nomad.client.secret_id}}"
# tags: []
############################
# nomad vault integration #
############################
# nomad_enable_vault_integration: false
# nomad_vault_integration_configuration: {}
###############################
# nomad drivers configuration #
###############################
# nomad_driver_enable_docker: yes
# nomad_driver_enable_podman: no
# nomad_driver_enable_raw_exec: no
# nomad_driver_enable_java: no
# nomad_driver_enable_qemu: no
# nomad_driver_extra_configuration: {}
######################
# nomad internal tls #
######################
nomad_enable_tls: true
nomad_tls_configuration:
http: true
rpc: true
ca_file: "/etc/ssl/certs/ca-certificates.crt"
cert_file: "{{ nomad_certificates_directory }}/cert.pem"
key_file: "{{ nomad_certificates_directory }}/key.pem"
verify_server_hostname: true
# nomad_certificates_directory: "{{ hashicorp_nomad_config_dir }}/tls"
# nomad_certificates_extra_files_dir:
# - src: "{{ sub_configuration_directories['certificates'] }}/nomad/{{ inventory_hostname }}"
# dest: "{{ nomad_certificates_directory }}"
#############################
# nomad extra configuration #
#############################
# nomad_extra_configuration: {}
# nomad_extra_files_list: []

View File

@ -11,7 +11,7 @@ platforms:
- name: proxy01
box: generic/${MOLECULE_TEST_OS}
cpus: 2
memory: 4096
memory: 2048
interfaces:
- network_name: private_network
ip: 192.168.100.91
@ -24,7 +24,7 @@ platforms:
- name: proxy02
box: generic/${MOLECULE_TEST_OS}
cpus: 2
memory: 4096
memory: 2048
interfaces:
- network_name: private_network
ip: 192.168.100.92
@ -37,7 +37,7 @@ platforms:
- name: hashistack01
box: generic/${MOLECULE_TEST_OS}
cpus: 4
memory: 4096
memory: 2048
interfaces:
- network_name: private_network
ip: 192.168.100.101
@ -51,7 +51,7 @@ platforms:
- name: hashistack02
box: generic/${MOLECULE_TEST_OS}
cpus: 4
memory: 4096
memory: 2048
interfaces:
- network_name: private_network
ip: 192.168.100.102
@ -65,7 +65,7 @@ platforms:
- name: hashistack03
box: generic/${MOLECULE_TEST_OS}
cpus: 4
memory: 4096
memory: 2048
interfaces:
- network_name: private_network
ip: 192.168.100.103
@ -76,6 +76,19 @@ platforms:
- vault_servers
- consul_servers
- nomad_servers
- name: hashistack04
box: generic/${MOLECULE_TEST_OS}
cpus: 4
memory: 2048
interfaces:
- network_name: private_network
ip: 192.168.100.104
auto_config: true
type: static
groups:
- common
- nomad_clients
- consul_agents
provisioner:
name: ansible
config_options:

View File

@ -6,8 +6,8 @@ roles:
- name: ednz_cloud.manage_pip_packages
- name: ednz_cloud.install_docker
- name: ednz_cloud.docker_systemd_service
- name: ednz_cloud.deploy_haproxy
- name: ednz_cloud.deploy_keepalived
# - name: ednz_cloud.deploy_haproxy
# - name: ednz_cloud.deploy_keepalived
collections:
- name: ednz_cloud.hashistack

View File

@ -35,13 +35,11 @@
install_docker_auto_update: false
install_docker_start_service: true
install_docker_compose: false
install_docker_compose_version: latest
install_docker_python_packages: true
install_docker_python_packages_version: latest
install_docker_python_packages: false
install_docker_users:
- "{{ ansible_user }}"
install_docker_daemon_options: {}
when: deployment_method == 'docker'
#! when: "'nomad_agents' in group_names"
- name: "Ensure /etc/localtime exists"
ansible.builtin.file:

View File

@ -12,23 +12,23 @@
tags:
- always
- name: "Deploy Consul Control Plane"
ansible.builtin.import_tasks:
file: tasks/consul/consul_deploy.yml
when:
- enable_consul | bool
- "'consul_servers' in group_names"
- name: "Deploy Consul"
tags:
- consul
when:
- enable_consul | bool
block:
- name: "Deploy Consul Control Plane"
ansible.builtin.import_tasks:
file: tasks/consul/consul_deploy.yml
when:
- "'consul_servers' in group_names"
- name: "Deploy Consul Agents"
ansible.builtin.include_role:
name: ednz_cloud.hashistack.hashicorp_consul
when:
- enable_consul | bool
- "'consul_agents' in group_names"
tags:
- consul
- name: "Deploy Consul Agents"
ansible.builtin.include_role:
name: ednz_cloud.hashicorp_consul
when:
- "'consul_agents' in group_names"
- name: "Deploy Haproxy & Keepalived"
ansible.builtin.import_tasks:
@ -47,3 +47,22 @@
- "'vault_servers' in group_names"
tags:
- vault
- name: "Deploy Nomad"
tags:
- nomad
when:
- enable_nomad | bool
block:
- name: "Deploy Nomad Control Plane"
ansible.builtin.import_tasks:
file: tasks/nomad/nomad_deploy.yml
when:
- "('nomad_servers' in group_names)"
- name: "Deploy Nomad Clients"
ansible.builtin.include_role:
name: ednz_cloud.hashicorp_nomad
when:
- "('nomad_clients' in group_names)"
- "('nomad_servers' not in group_names)"

View File

@ -6,6 +6,12 @@
gather_facts: true
become: true
tasks:
- name: "Import variables"
ansible.builtin.import_tasks:
file: tasks/load_vars.yml
tags:
- always
- name: "Create temporary cert directory in {{ sub_configuration_directories['certificates'] }}" # noqa: run-once[task]
ansible.builtin.file:
path: "{{ sub_configuration_directories['certificates'] }}/external"
@ -221,8 +227,8 @@
'IP:' + api_interface_address,
'IP:127.0.0.1'
] -%}
{%- if hashi_consul_configuration.server -%}
{%- set _ = sans_list.append('DNS:server.' ~ hashi_consul_configuration.datacenter ~ '.' ~ hashi_consul_configuration.domain) -%}
{%- if hashicorp_consul_configuration.server -%}
{%- set _ = sans_list.append('DNS:server.' ~ hashicorp_consul_configuration.datacenter ~ '.' ~ hashicorp_consul_configuration.domain) -%}
{%- endif -%}
{{ sans_list }}
community.crypto.openssl_csr_pipe:
@ -272,4 +278,88 @@
group: "{{ lookup('env', 'USER') }}"
mode: "0644"
# - fail:
- name: "Create Nomad certificates"
when:
- "('nomad_servers' in group_names) or ('nomad_clients' in group_names)"
vars:
nomad_private_key_path: "{{ sub_configuration_directories['certificates'] }}/nomad/{{ inventory_hostname }}/key.pem"
nomad_certificate_path: "{{ sub_configuration_directories['certificates'] }}/nomad/{{ inventory_hostname }}/cert.pem"
block:
- name: "Create temporary cert directory in {{ sub_configuration_directories['certificates'] }}" # noqa: run-once[task]
ansible.builtin.file:
path: "{{ sub_configuration_directories['certificates'] }}/nomad/{{ inventory_hostname }}"
state: directory
owner: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
mode: "0755"
- name: "Create Nomad certificate keys"
community.crypto.openssl_privatekey:
path: "{{ nomad_private_key_path }}"
owner: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
- name: "Create CSRs for Nomad servers"
vars:
nomad_csr_sans: >-
{%- set sans_list = [
'DNS:' + inventory_hostname,
'DNS:localhost',
'IP:' + api_interface_address,
'IP:127.0.0.1'
] -%}
{%- if hashicorp_nomad_configuration.server.enabled -%}
{%- set _ = sans_list.append('DNS:server.' ~ hashicorp_nomad_configuration.region ~ '.nomad') -%}
{%- if (enable_consul | bool) -%}
{%- set _ = sans_list.append('DNS:nomad.service.consul') -%}
{%- endif -%}
{%- endif -%}
{%- if hashicorp_nomad_configuration.client.enabled -%}
{%- set _ = sans_list.append('DNS:client.' ~ hashicorp_nomad_configuration.region ~ '.nomad') -%}
{%- endif -%}
{{ sans_list }}
community.crypto.openssl_csr_pipe:
privatekey_path: "{{ nomad_private_key_path }}"
common_name: "{{ inventory_hostname }}"
subject_alt_name: "{{ nomad_csr_sans }}"
key_usage_critical: true
key_usage:
- Digital Signature
- Key Encipherment
extended_key_usage:
- TLS Web Server Authentication
- TLS Web Client Authentication
organization_name: EDNZ Cloud
use_common_name_for_san: false
register: nomad_csr
- name: "Sign certificates with internal CA"
community.crypto.x509_certificate:
path: "{{ nomad_certificate_path }}"
csr_content: "{{ nomad_csr.csr }}"
provider: ownca
ownca_path: "{{ hashistack_ca_cert_path }}"
ownca_privatekey_path: "{{ hashistack_ca_key_path }}"
ownca_not_after: "+365d"
ownca_not_before: "-1d"
- name: "Concatenate CA and Child certificates"
block:
- name: "Read content of ca.crt"
ansible.builtin.slurp:
src: "{{ hashistack_ca_cert_path }}"
register: ca_crt_content
- name: "Read content of cert.pem"
ansible.builtin.slurp:
src: "{{ nomad_certificate_path }}"
register: cert_pem_content
- name: "Concatenate certificates"
ansible.builtin.copy:
content: |
{{ cert_pem_content['content'] | b64decode }}{{ ca_crt_content['content'] | b64decode }}
dest: "{{ nomad_certificate_path }}"
owner: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
mode: "0644"

View File

@ -8,25 +8,51 @@
tasks:
- name: "Generate consul credentials"
block:
- name: "Generate consul gossip encryption key"
ansible.builtin.set_fact:
_consul_gossip_encryption_key: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | b64encode }}"
- name: "Generate consul root credentials"
ansible.builtin.set_fact:
_consul_root_token: "{{ lookup('password', '/dev/null chars=ascii_letters,digits') | to_uuid }}"
_consul_root_token: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
- name: "Generate consul agents credentials"
ansible.builtin.set_fact:
_cosul_agents_accessor: "{{ lookup('password', '/dev/null chars=ascii_letters,digits') | to_uuid }}"
_consul_agents_token: "{{ lookup('password', '/dev/null chars=ascii_letters,digits') | to_uuid }}"
_consul_agents_accessor: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
_consul_agents_token: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
- name: "Generate consul vault credentials"
ansible.builtin.set_fact:
_cosul_vault_accessor: "{{ lookup('password', '/dev/null chars=ascii_letters,digits') | to_uuid }}"
_consul_vault_token: "{{ lookup('password', '/dev/null chars=ascii_letters,digits') | to_uuid }}"
_consul_vault_accessor: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
_consul_vault_token: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
- name: "Generate consul nomad server credentials"
ansible.builtin.set_fact:
_consul_nomad_server_accessor: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
_consul_nomad_server_token: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
- name: "Generate consul nomad client credentials"
ansible.builtin.set_fact:
_consul_nomad_client_accessor: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
_consul_nomad_client_token: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
- name: "Generate nomad credentials"
block:
- name: "Generate nomad gossip encryption key"
ansible.builtin.set_fact:
_nomad_gossip_encryption_key: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | b64encode }}"
- name: "Generate nomad root credentials"
ansible.builtin.set_fact:
_nomad_root_token: "{{ lookup('password', '/dev/null chars=ascii_letters,digits') | to_uuid }}"
_nomad_root_token: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_letters','digits']) | to_uuid }}"
- name: "Ensure secrets directory is created"
ansible.builtin.file:
path: "{{ sub_configuration_directories['secrets'] }}"
state: directory
owner: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
mode: '0755'
- name: "Write credentials file"
ansible.builtin.template:
@ -35,5 +61,3 @@
owner: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
mode: '0644'
# - fail:

View File

@ -5,18 +5,6 @@
manage_pip_packages_allow_break_system_packages: "{{ ansible_distribution == 'Debian' and ansible_distribution_version == '12' }}"
vault_versions:
host: "{{ vault_version if vault_version != 'latest' else vault_version + '*' }}"
docker: "{{ vault_version }}"
consul_versions:
host: "{{ consul_version if consul_version != 'latest' else consul_version + '*' }}"
docker: "{{ consul_version }}"
nomad_versions:
host: "{{ nomad_version if nomad_version != 'latest' else nomad_version + '*' }}"
docker: "{{ nomad_version }}"
configuration_directory: "{{ lookup('env', 'PWD') }}/etc/hashistack"
sub_configuration_directories:
secrets: "{{ configuration_directory }}/secrets"

View File

@ -57,7 +57,7 @@ consul_default_agent_policy: |
# consul internal tls #
#######################
consul_certificates_directory: "{{ hashi_consul_config_dir }}/tls"
consul_certificates_directory: "{{ hashicorp_consul_config_dir }}/tls"
consul_certificates_extra_files_dir:
- src: "{{ sub_configuration_directories['certificates'] }}/consul/{{ inventory_hostname }}"
dest: "{{ consul_certificates_directory }}"
@ -66,29 +66,27 @@ consul_certificates_extra_files_dir:
# consul role variables #
#########################
hashi_consul_start_service: true
hashi_consul_version: "{{ consul_versions[deployment_method] }}"
hashi_consul_deploy_method: "{{ deployment_method }}"
hashi_consul_env_variables: {}
hashi_consul_config_dir: "/etc/consul.d"
hashi_consul_data_dir: "/opt/consul"
hashi_consul_extra_files: true
hashi_consul_extra_files_list: "{{ ([] +
hashicorp_consul_start_service: true
hashicorp_consul_service_name: "consul"
hashicorp_consul_version: "{{ consul_version }}"
hashicorp_consul_env_variables: {}
hashicorp_consul_config_dir: "/etc/consul.d"
hashicorp_consul_data_dir: "/opt/consul"
hashicorp_consul_extra_files: true
hashicorp_consul_extra_files_list: "{{ ([] +
(consul_certificates_extra_files_dir if consul_enable_tls else []) +
(vault_plugin_extra_files_dir if vault_enable_plugins else []) +
vault_extra_files_list)
consul_extra_files_list)
| unique
| sort
}}"
hashi_consul_extra_container_volumes: "{{ default_container_extra_volumes | union(extra_consul_container_volumes) | unique | sort }}"
hashi_consul_envoy_install: false
hashi_consul_envoy_version: v1.27.2
hashi_consul_configuration:
hashicorp_consul_envoy_install: false
hashicorp_consul_envoy_version: v1.27.2
hashicorp_consul_configuration:
domain: "{{ consul_domain }}"
datacenter: "{{ consul_datacenter }}"
primary_datacenter: "{{ consul_primary_datacenter }}"
data_dir: "{{ hashi_consul_data_dir }}"
encrypt: "{{ consul_gossip_encryption_key }}"
data_dir: "{{ hashicorp_consul_data_dir }}"
encrypt: "{{ _credentials.consul.gossip_encryption_key }}"
server: "{{ 'consul_servers' in group_names }}"
retry_join: "{{
groups['consul_servers'] |
@ -102,14 +100,11 @@ hashi_consul_configuration:
leave_on_terminate: "{{ consul_leave_on_terminate }}"
rejoin_after_leave: "{{ consul_rejoin_after_leave }}"
enable_script_checks: "{{ consul_enable_script_checks }}"
enable_syslog: "{{ deployment_method == 'host' }}"
log_level: INFO
enable_syslog: true
acl: "{{ consul_acl_configuration }}"
dns_config: "{{ consul_dns_configuration }}"
ports:
dns: 8600
grpc: 8502
grpc_tls: 8503
server: 8300
serf_lan: 8301
serf_wan: 8302
@ -119,8 +114,12 @@ hashi_consul_configuration:
expose_max_port: 21755
# this is used to circumvent jinja limitation to convert string to integer
hashi_consul_configuration_string: |
bootstrap_expect: {{ (groups['consul_servers'] | length) }}
hashicorp_consul_configuration_string: |
ports:
http: {{ (consul_api_port.http|int) if not consul_enable_tls else ('-1' | int) }}
https: {{ (consul_api_port.https|int) if consul_enable_tls else ('-1' | int) }}
grpc: {{ ('8502'|int) if not consul_enable_tls else ('-1' | int) }}
grpc_tls: {{ ('8503'|int) if consul_enable_tls else ('-1' | int) }}
hashicorp_consul_servers_configuration_string: |
bootstrap_expect: {{ (groups['consul_servers'] | length) }}

View File

@ -9,12 +9,10 @@ enable_consul: "yes"
enable_nomad: "yes"
haproxy_version: "2.8"
nomad_version: "1.7.7"
nomad_version: "1.8.1"
consul_version: "1.18.1"
vault_version: "1.16.2"
deployment_method: "docker"
consul_fqdn: consul.ednz.lab
vault_fqdn: vault.ednz.lab
nomad_fqdn: nomad.ednz.lab
@ -46,7 +44,6 @@ consul_primary_datacenter: dc1
consul_leave_on_terminate: true
consul_rejoin_after_leave: true
consul_enable_script_checks: true
consul_gossip_encryption_key: "{{ 'mysupersecretgossipencryptionkey'|b64encode }}"
################################
# consul address configuration #
@ -114,9 +111,9 @@ consul_tls_configuration:
extra_consul_container_volumes: []
#######################
# extra configuration #
#######################
##############################
# consul extra configuration #
##############################
consul_extra_configuration: {}
consul_extra_files_list: []
@ -139,7 +136,7 @@ vault_seal_configuration:
vault_storage_configuration:
raft:
path: "{{ hashi_vault_data_dir }}/data"
path: "{{ hashicorp_vault_data_dir }}"
node_id: "{{ ansible_hostname }}"
retry_join: |
[
@ -204,9 +201,79 @@ vault_logging_configuration:
extra_vault_container_volumes: []
#####################
# extra configuration
#####################
#############################
# vault extra configuration #
#############################
vault_extra_configuration: {}
vault_extra_files_list: []
#####################################################
# #
# Nomad #
# #
#####################################################
nomad_datacenter: dc1
nomad_region: global
###########################
# nomad ACL configuration #
###########################
nomad_acl_configuration:
enabled: true
token_ttl: 30s
policy_ttl: 60s
role_ttl: 60s
############################
# nomad consul integration #
############################
nomad_enable_consul_integration: "{{ enable_consul | bool }}"
nomad_consul_integration_configuration:
address: "127.0.0.1:{{ hashicorp_consul_configuration.ports.https if consul_enable_tls else hashicorp_consul_configuration.ports.http }}"
auto_advertise: true
ssl: "{{ consul_enable_tls | bool }}"
token: "{{ _credentials.consul.tokens.nomad.server.secret_id if nomad_enable_server else _credentials.consul.tokens.nomad.client.secret_id}}"
tags: []
############################
# nomad vault integration #
############################
nomad_enable_vault_integration: false
nomad_vault_integration_configuration: {}
###############################
# nomad drivers configuration #
###############################
nomad_driver_enable_docker: yes
nomad_driver_enable_podman: no
nomad_driver_enable_raw_exec: no
nomad_driver_enable_java: no
nomad_driver_enable_qemu: no
nomad_driver_extra_configuration: {}
######################
# nomad internal tls #
######################
nomad_enable_tls: false
nomad_tls_configuration:
http: true
rpc: true
ca_file: "/etc/ssl/certs/ca-certificates.crt"
cert_file: "{{ nomad_certificates_directory }}/cert.pem"
key_file: "{{ nomad_certificates_directory }}/key.pem"
verify_server_hostname: true
#############################
# nomad extra configuration #
#############################
nomad_extra_configuration: {}
nomad_extra_files_list: []

View File

@ -5,7 +5,7 @@
# #
#####################################################
deploy_haproxy_deploy_method: "{{ deployment_method }}"
deploy_haproxy_deploy_method: "host"
deploy_haproxy_version: "{{ haproxy_version }}"
deploy_haproxy_env_variables: {}
@ -49,11 +49,11 @@ deploy_haproxy_frontends:
{%- set haproxy_options = haproxy_options + tls_options -%}
{%- endif -%}
{%- set haproxy_options = haproxy_options + consul_haproxy_frontend_options + vault_haproxy_frontend_options -%}
{%- set haproxy_options = haproxy_options + consul_haproxy_frontend_options + vault_haproxy_frontend_options + nomad_haproxy_frontend_options -%}
{{ haproxy_options }}
deploy_haproxy_backends: "{{ consul_haproxy_backends + vault_haproxy_backends }}"
deploy_haproxy_backends: "{{ consul_haproxy_backends + vault_haproxy_backends + nomad_haproxy_backends }}"
deploy_haproxy_listen:
- name: monitoring
@ -73,7 +73,7 @@ deploy_haproxy_listen:
- monitor-uri /health
- http-request use-service prometheus-exporter if { path /metrics }
deploy_keepalived_deploy_method: "{{ deployment_method }}"
deploy_keepalived_deploy_method: "host"
deploy_keepalived_version: "latest"
deploy_keepalived_start_service: true
deploy_keepalived_env_variables: {}

View File

@ -1,18 +1,199 @@
---
#####################################################
# #
# Nomad Configuration #
# Non-Editable #
# #
#####################################################
hashi_nomad_cni_plugins_install: true
hashi_nomad_start_service: true
hashi_nomad_cni_plugins_version: latest
hashi_nomad_cni_plugins_install_path: /opt/cni/bin
hashi_nomad_version: latest
hashi_nomad_deploy_method: host # deployment method, either host or docker
hashi_nomad_env_variables: {}
hashi_nomad_data_dir: /opt/nomad
hashi_nomad_extra_files: false
hashi_nomad_extra_files_src: /tmp/extra_files
hashi_nomad_extra_files_dst: /etc/nomad.d/extra_files
hashi_nomad_configuration: {}
####################
# nomad api config #
####################
nomad_api_addr: "{{ nomad_api_scheme }}://{{ api_interface_address }}:{{ nomad_api_port[nomad_api_scheme] }}"
nomad_api_scheme: "{{ 'https' if nomad_enable_tls else 'http' }}"
nomad_api_port:
http: "{{ nomad_address_configuration.ports.http }}"
https: "{{ nomad_address_configuration.ports.http }}"
#########################
# nomad haproxy backend #
#########################
nomad_haproxy_frontend_options:
- acl is_nomad hdr(host) -i {{ nomad_fqdn }}
- use_backend nomad_external if is_nomad
nomad_haproxy_backends:
- name: nomad_external
options: "{{ nomad_external_backend_options + nomad_external_backend_servers }}"
nomad_external_backend_options:
- description nomad external http backend
- option forwardfor
- option httpchk
- http-check send meth GET uri /
- default-server inter 2s fastinter 1s downinter 1s
nomad_external_backend_servers: |
[
{% for host in groups['nomad_servers'] %}
'server nomad-{{ hostvars[host].api_interface_address }} {{ hostvars[host].api_interface_address }}:{{ hostvars[host].nomad_api_port[nomad_api_scheme] }} check {{ 'ssl verify none ' if nomad_enable_tls }}inter 5s'{% if not loop.last %},{% endif %}
{% endfor %}
]
###############################
# nomad address configuration #
###############################
nomad_address_configuration:
bind_addr: "{{ api_interface_address }}"
addresses:
http: "{{ api_interface_address }}"
rpc: "{{ api_interface_address }}"
serf: "{{ api_interface_address }}"
advertise:
http: "{{ api_interface_address }}"
rpc: "{{ api_interface_address }}"
serf: "{{ api_interface_address }}"
ports:
http: 4646
rpc: 4647
serf: 4648
#################################
# nomad autopilot configuration #
#################################
nomad_autopilot_configuration: {}
############################
# nomad consul integration #
############################
nomad_consul_integration_tls_configuration:
ca_file: "/etc/ssl/certs/ca-certificates.crt"
nomad_consul_integration_server_configuration:
server_auto_join: true
nomad_consul_integration_client_configuration:
client_auto_join: true
grpc_address: "127.0.0.1:{{ hashicorp_consul_configuration.ports.grpc_tls if consul_enable_tls else hashicorp_consul_configuration.ports.grpc }}"
nomad_consul_integration_client_tls_configuration:
grpc_ca_file: "/etc/ssl/certs/ca-certificates.crt"
nomad_consul_integration_server_policy: |
agent_prefix "" {
policy = "read"
}
node_prefix "" {
policy = "read"
}
service_prefix "" {
policy = "write"
}
acl = "write"
mesh = "write"
nomad_consul_integration_client_policy: |
agent_prefix "" {
policy = "read"
}
node_prefix "" {
policy = "read"
}
service_prefix "" {
policy = "write"
}
#############################
# nomad leave configuration #
#############################
# node will leave the cluster if the process is stopped
# and if it is only a client
nomad_leave_on_interrupt: "{{ (('nomad_clients' in group_names) and not ('nomad_servers' in group_names)) | bool }}"
nomad_leave_on_terminate: "{{ (('nomad_clients' in group_names) and not ('nomad_servers' in group_names)) | bool }}"
##########################
# nomad ui configuration #
##########################
nomad_ui_configuration:
enabled: "{{ ('nomad_servers' in group_names) | bool }}"
##############################
# nomad server configuration #
##############################
nomad_enable_server: "{{ ('nomad_servers' in group_names) | bool }}"
nomad_server_configuration:
enabled: "{{ nomad_enable_server }}"
data_dir: "{{ hashicorp_nomad_data_dir }}/server"
encrypt: "{{ _credentials.nomad.gossip_encryption_key }}"
##############################
# nomad client configuration #
##############################
nomad_enable_client: "{{ ('nomad_clients' in group_names) | bool }}"
nomad_client_configuration:
enabled: "{{ nomad_enable_client }}"
state_dir: "{{ hashicorp_nomad_data_dir }}/client"
bridge_network_name: nomad
bridge_network_subnet: "172.26.64.0/20"
###############################
# nomad drivers configuration #
###############################
nomad_driver_configuration:
raw_exec:
enabled: "{{ nomad_driver_enable_raw_exec | bool }}"
######################
# nomad internal tls #
######################
nomad_certificates_directory: "{{ hashicorp_nomad_config_dir }}/tls"
nomad_certificates_extra_files_dir:
- src: "{{ sub_configuration_directories['certificates'] }}/nomad/{{ inventory_hostname }}"
dest: "{{ nomad_certificates_directory }}"
########################
# nomad role variables #
########################
hashicorp_nomad_start_service: true
hashicorp_nomad_service_name: "nomad"
hashicorp_nomad_cni_plugins_install: true
hashicorp_nomad_cni_plugins_version: latest
hashicorp_nomad_cni_plugins_install_path: /opt/cni/bin
hashicorp_nomad_version: "{{ nomad_version }}"
hashicorp_nomad_env_variables: {}
hashicorp_nomad_config_dir: "/etc/nomad.d"
hashicorp_nomad_data_dir: /opt/nomad
hashicorp_nomad_extra_files: true
hashicorp_nomad_extra_files_list: "{{ ([] +
(nomad_certificates_extra_files_dir if nomad_enable_tls else []) +
nomad_extra_files_list)
| unique
| sort
}}"
hashicorp_nomad_configuration:
datacenter: "{{ nomad_datacenter }}"
region: "{{ nomad_region }}"
bind_addr: "0.0.0.0"
data_dir: "{{ hashicorp_nomad_data_dir }}"
leave_on_interrupt: "{{ nomad_leave_on_interrupt }}"
leave_on_terminate: "{{ nomad_leave_on_terminate }}"
acl: "{{ nomad_acl_configuration }}"
server: "{{ nomad_server_configuration }}"
client: "{{ nomad_client_configuration }}"
ui: "{{ nomad_ui_configuration }}"
# this is used to circumvent jinja limitation to convert string to integer
hashicorp_nomad_configuration_string: |
server:
bootstrap_expect: {{ (groups['nomad_servers'] | length) }}

View File

@ -35,7 +35,7 @@ vault_external_backend_servers: |
# vault internal tls #
######################
vault_certificates_directory: "{{ hashi_vault_config_dir }}/tls"
vault_certificates_directory: "{{ hashicorp_vault_config_dir }}/tls"
vault_certificates_extra_files_dir:
- src: "{{ sub_configuration_directories['certificates'] }}/vault/{{ inventory_hostname }}"
dest: "{{ vault_certificates_directory }}"
@ -44,7 +44,7 @@ vault_certificates_extra_files_dir:
# vault plugins #
#################
vault_plugin_directory: "{{ hashi_vault_config_dir }}/plugin"
vault_plugin_directory: "{{ hashicorp_vault_config_dir }}/plugin"
vault_plugin_extra_files_dir:
- src: "{{ sub_configuration_directories['vault_servers'] }}/plugin"
dest: "{{ vault_plugin_directory }}"
@ -62,24 +62,24 @@ vault_service_registration_policy: |
# vault role variables #
########################
hashi_vault_start_service: true
hashi_vault_version: "{{ vault_versions[deployment_method] }}"
hashi_vault_deploy_method: "{{ deployment_method }}"
hashi_vault_env_variables: {}
hashi_vault_config_dir: "/etc/vault.d"
hashi_vault_data_dir: "/opt/vault"
hashi_vault_extra_files: true
hashi_vault_extra_files_list: "{{ ([] +
hashicorp_vault_start_service: true
hashicorp_vault_service_name: "vault"
hashicorp_vault_version: "{{ vault_version }}"
hashicorp_vault_env_variables: {}
hashicorp_vault_config_dir: "/etc/vault.d"
hashicorp_vault_data_dir: "/opt/vault"
hashicorp_vault_extra_files: true
hashicorp_vault_extra_files_list: "{{ ([] +
(vault_certificates_extra_files_dir if vault_enable_tls else []) +
(vault_plugin_extra_files_dir if vault_enable_plugins else []) +
vault_extra_files_list)
| unique
| sort
}}"
hashi_vault_extra_files_src: "{{ sub_configuration_directories.vault_servers }}/config"
hashi_vault_extra_files_dst: "{{ hashi_vault_config_dir }}/config"
hashi_vault_extra_container_volumes: "{{ default_container_extra_volumes | union(extra_vault_container_volumes) | unique | sort }}"
hashi_vault_configuration:
hashicorp_vault_extra_files_src: "{{ sub_configuration_directories.vault_servers }}/config"
hashicorp_vault_extra_files_dst: "{{ hashicorp_vault_config_dir }}/config"
hashicorp_vault_extra_container_volumes: "{{ default_container_extra_volumes | union(extra_vault_container_volumes) | unique | sort }}"
hashicorp_vault_configuration:
cluster_name: "{{ vault_cluster_name }}"
cluster_addr: "{{ 'https' if vault_enable_tls else 'http'}}://{{ api_interface_address }}:8201"
api_addr: "{{ 'https' if vault_enable_tls else 'http'}}://{{ api_interface_address }}:8200"

View File

@ -12,15 +12,22 @@ consul01
consul02
consul03
[consul_agents:children]
haproxy_servers
vault_servers
[nomad_servers]
nomad01
nomad02
nomad03
[nomad_clients]
nomad-client01
nomad-client02
nomad-client03
[consul_agents:children]
haproxy_servers
vault_servers
nomad_servers
nomad_clients
[deployment]
localhost ansible_connection=local

View File

@ -211,10 +211,6 @@
msg: "Please check the api_interface settings - interface {{ api_interface }} is not active"
when: not hostvars[inventory_hostname].ansible_facts[api_interface]['active']
- name: "Debug"
ansible.builtin.debug:
msg: "{{ api_interface_address }}"
- name: "Checking the api_interface ip address configuration"
ansible.builtin.fail:
msg: "Please check the api_interface settings - interface {{ api_interface }} ip address problem"
@ -288,20 +284,3 @@
that:
- "ansible_facts.service_mgr == 'systemd'"
when: inventory_hostname in groups['common']
- name: "Checking that python SDK for docker is installed"
when: deployment_method == 'docker'
vars:
wanted_docker_sdk_package: "python3-docker"
block:
- name: "Get packages facts"
ansible.builtin.package_facts:
manager: auto
- name: "Checking that python SDK for docker is installed"
ansible.builtin.assert:
that:
- "wanted_docker_sdk_package in ansible_facts.packages"
fail_msg: >-
The python sdk for docker is really out of date, you need to install
a more recent version of it in order to use this tool.

View File

@ -1,21 +1,29 @@
---
- name: "Consul"
block:
- name: "Include ednz_cloud.hashistack.hashicorp_consul"
- name: "Include ednz_cloud.hashicorp_consul"
ansible.builtin.include_role:
name: ednz_cloud.hashistack.hashicorp_consul
name: ednz_cloud.hashicorp_consul
- name: "Wait for consul cluster to initialize" # noqa: run-once[task]
ansible.builtin.uri:
url: "{{ consul_api_addr }}"
validate_certs: no
return_content: yes
status_code:
- 200
until: uri_output.status == 200
retries: 24
delay: 5
register: uri_output
block:
- name: "Wait for consul nodes to stabilize"
ansible.builtin.wait_for:
host: "{{ api_interface_address }}"
port: "{{ consul_api_port[consul_api_scheme] }}"
delay: 10
- name: "Waiting for consul api to respond"
ansible.builtin.uri:
url: "{{ consul_api_addr }}"
validate_certs: no
return_content: yes
status_code:
- 200
until: uri_output.status == 200
retries: 24
delay: 5
register: uri_output
- name: "Initialize consul cluster" # noqa: run-once[task]
community.general.consul_acl_bootstrap:
@ -27,11 +35,10 @@
run_once: true
delegate_to: "{{ groups['consul_servers'] | first }}"
register: _consul_init_secret
when: hashi_consul_configuration.acl.enabled
when: hashicorp_consul_configuration.acl.enabled
- name: "Create consul agents token"
when:
- _consul_init_secret.changed
- consul_acl_configuration.enabled
block:
- name: "Create consul agents token" # noqa: run-once[task] no-handler
@ -63,9 +70,9 @@
state: present
register: _consul_agent_token
- name: "Restart consul service"
- name: "Restart consul service" # noqa: no-handler
ansible.builtin.service:
name: "consul_container"
name: "{{ hashicorp_consul_service_name }}"
state: restarted
throttle: 1
when: _consul_agent_token.changed

View File

@ -2,22 +2,33 @@
# hashistack configuration merging for consul
- name: "Consul | Merge stringified configuration"
vars:
_config_to_merge: "{{ hashi_consul_configuration_string }}"
_config_to_merge: "{{ hashicorp_consul_configuration_string }}"
ansible.builtin.set_fact:
hashi_consul_configuration: "{{
hashi_consul_configuration |
hashicorp_consul_configuration: "{{
hashicorp_consul_configuration |
combine(_config_to_merge|from_yaml, recursive=true)
}}"
when:
- hashi_consul_configuration_string is defined
- hashicorp_consul_configuration_string is defined
- name: "Consul | Merge servers specific stringified configuration"
vars:
_config_to_merge: "{{ hashicorp_consul_servers_configuration_string }}"
ansible.builtin.set_fact:
hashicorp_consul_configuration: "{{
hashicorp_consul_configuration |
combine(_config_to_merge|from_yaml, recursive=true)
}}"
when:
- hashicorp_consul_configuration_string is defined
- "'consul_servers' in group_names"
- name: "Consul | Merge addresses configuration"
vars:
_config_to_merge: "{{ consul_address_configuration }}"
ansible.builtin.set_fact:
hashi_consul_configuration: "{{
hashi_consul_configuration |
hashicorp_consul_configuration: "{{
hashicorp_consul_configuration |
combine(_config_to_merge, recursive=true)
}}"
when: consul_address_configuration is defined
@ -27,8 +38,8 @@
_config_to_merge:
tls: "{{ consul_tls_configuration }}"
ansible.builtin.set_fact:
hashi_consul_configuration: "{{
hashi_consul_configuration |
hashicorp_consul_configuration: "{{
hashicorp_consul_configuration |
combine(_config_to_merge, recursive=true)
}}"
when: consul_enable_tls
@ -43,14 +54,14 @@
tokens:
agent: "{{ _credentials.consul.tokens.agent.secret_id }}"
ansible.builtin.set_fact:
hashi_consul_configuration: "{{ hashi_consul_configuration | default({}) | combine(_config_to_merge, recursive=true) }}"
hashicorp_consul_configuration: "{{ hashicorp_consul_configuration | default({}) | combine(_config_to_merge, recursive=true) }}"
- name: "Consul | Merge extra configuration settings"
vars:
_config_to_merge: "{{ consul_extra_configuration }}"
ansible.builtin.set_fact:
hashi_consul_configuration: "{{
hashi_consul_configuration |
hashicorp_consul_configuration: "{{
hashicorp_consul_configuration |
combine(_config_to_merge, recursive=true)
}}"
when: consul_extra_configuration is defined

View File

@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
set -e
function do_ping() {

View File

@ -15,6 +15,10 @@
- name: "Register haproxy services in consul"
community.general.consul:
token: "{{ _credentials.consul.root_token.secret_id }}"
host: "127.0.0.1"
scheme: "{{ consul_api_scheme }}"
port: "{{ consul_api_port[consul_api_scheme] }}"
validate_certs: false
service_name: haproxy
service_port: 80
interval: 20s

View File

@ -45,3 +45,10 @@
when:
- enable_vault | bool
- "'vault_servers' in group_names"
- name: "Merge nomad configurations"
ansible.builtin.import_tasks:
file: "nomad/nomad_vars.yml"
when:
- enable_nomad | bool
- "('nomad_servers' in group_names) or ('nomad_clients' in group_names)"

View File

@ -142,9 +142,6 @@
delegate_to: localhost
when: _hashistack_ca_directory.stat.exists and _hashistack_ca_directory.stat.isdir
- ansible.builtin.debug:
msg: "{{ _hashistack_cacert_files }}"
- name: "Ensure remote ca directory exists"
ansible.builtin.file:
path: "{{ hashistack_remote_config_dir }}/ca"
@ -207,10 +204,3 @@
when:
- enable_vault | bool
- "'vault_servers' in group_names"
- debug:
msg: "{{ deploy_haproxy_frontends }}"
- debug:
msg: "{{ deploy_haproxy_backends }}"
# - fail:

View File

@ -13,9 +13,6 @@
delegate_to: localhost
when: _hashistack_ca_directory.stat.exists and _hashistack_ca_directory.stat.isdir
- ansible.builtin.debug:
msg: "{{ _hashistack_cacert_files }}"
- name: "Ensure remote ca directory exists"
ansible.builtin.file:
path: "{{ hashistack_remote_config_dir }}/ca"
@ -46,7 +43,7 @@
loop: "{{ _hashistack_copied_ca.results }}"
register: _hashistack_usr_local_share_ca_certificates
- name: "Update the trust store"
- name: "Update the trust store" # noqa: no-handler
ansible.builtin.command: update-ca-certificates
changed_when: false
when: _hashistack_usr_local_share_ca_certificates.changed

View File

@ -44,7 +44,3 @@
_credentials: "{{ _credentials | combine(_config_to_merge, recursive=true) }}"
when: _vault_credentials_file.stat.exists
delegate_to: localhost
- name: "Debug _credentials"
ansible.builtin.debug:
msg: "{{ _credentials }}"

View File

@ -0,0 +1,83 @@
---
- name: "Nomad"
block:
- name: "Create consul tokens for service registration"
when:
- enable_consul
- nomad_enable_consul_integration
delegate_to: "{{ groups['consul_servers'] | first }}"
vars:
_consul_host: "{{ hostvars[groups['consul_servers'][0]].api_interface_address }}"
_consul_port: "{{ hostvars[groups['consul_servers'][0]].consul_api_port[hostvars[groups['consul_servers'][0]].consul_api_scheme] }}"
_consul_scheme: "{{ hostvars[groups['consul_servers'][0]].consul_api_scheme }}"
run_once: true
block:
- name: "Create server credentials"
block:
- name: "Create consul server policy"
community.general.consul_policy:
token: "{{ _credentials.consul.root_token.secret_id }}"
host: "{{ _consul_host }}"
port: "{{ _consul_port }}"
scheme: "{{ _consul_scheme }}"
validate_certs: false
state: present
name: nomad-server-policy
rules: "{{ nomad_consul_integration_server_policy }}"
register: _consul_nomad_server_policy
- name: "Create consul server token"
community.general.consul_token:
token: "{{ _credentials.consul.root_token.secret_id }}"
host: "{{ _consul_host }}"
port: "{{ _consul_port }}"
scheme: "{{ _consul_scheme }}"
validate_certs: false
accessor_id: "{{ _credentials.consul.tokens.nomad.server.accessor_id }}"
secret_id: "{{ _credentials.consul.tokens.nomad.server.secret_id }}"
policies:
- id: "{{ _consul_nomad_server_policy.policy.ID }}"
state: present
when: _consul_nomad_server_policy.changed
- name: "Create client credentials"
block:
- name: "Create consul client policy"
community.general.consul_policy:
token: "{{ _credentials.consul.root_token.secret_id }}"
host: "{{ _consul_host }}"
port: "{{ _consul_port }}"
scheme: "{{ _consul_scheme }}"
validate_certs: false
state: present
name: nomad-client-policy
rules: "{{ nomad_consul_integration_client_policy }}"
register: _consul_nomad_client_policy
- name: "Create consul client token"
community.general.consul_token:
token: "{{ _credentials.consul.root_token.secret_id }}"
host: "{{ _consul_host }}"
port: "{{ _consul_port }}"
scheme: "{{ _consul_scheme }}"
validate_certs: false
accessor_id: "{{ _credentials.consul.tokens.nomad.client.accessor_id }}"
secret_id: "{{ _credentials.consul.tokens.nomad.client.secret_id }}"
policies:
- id: "{{ _consul_nomad_client_policy.policy.ID }}"
state: present
when: _consul_nomad_client_policy.changed
- name: "Include ednz_cloud.hashicorp_nomad"
ansible.builtin.include_role:
name: ednz_cloud.hashicorp_nomad
- name: "Initialize nomad cluster" # noqa: run-once[task]
ednz_cloud.hashistack.nomad_acl_bootstrap:
bootstrap_secret: "{{ _credentials.nomad.root_token.secret_id }}"
api_url: "{{ nomad_api_addr }}"
tls_verify: false
run_once: true
delegate_to: "{{ groups['nomad_servers'] | first }}"
register: _nomad_init_secret
when: hashicorp_nomad_configuration.acl.enabled

View File

@ -0,0 +1,129 @@
---
# hashistack configuration merging for nomad
- name: "Nomad | Merge stringified configuration"
vars:
_config_to_merge: "{{ hashicorp_nomad_configuration_string }}"
ansible.builtin.set_fact:
hashicorp_nomad_configuration: "{{
hashicorp_nomad_configuration |
combine(_config_to_merge|from_yaml, recursive=true)
}}"
when:
- hashicorp_nomad_configuration_string is defined
- "'nomad_servers' in group_names"
- name: "Nomad | Merge addresses configuration"
vars:
_config_to_merge: "{{ nomad_address_configuration }}"
ansible.builtin.set_fact:
hashicorp_nomad_configuration: "{{
hashicorp_nomad_configuration |
combine(_config_to_merge, recursive=true)
}}"
when: nomad_address_configuration is defined
- name: "Nomad | Merge consul integration configuration"
when:
- enable_consul | bool
- nomad_enable_consul_integration | bool
block:
- name: "Nomad | Merge consul tls configuration"
when:
- nomad_consul_integration_configuration.ssl is defined
- nomad_consul_integration_configuration.ssl | bool
block:
- name: "Nomad | Merge consul default client configuration"
vars:
_config_to_merge: "{{ nomad_consul_integration_tls_configuration }}"
ansible.builtin.set_fact:
nomad_consul_integration_configuration: "{{
nomad_consul_integration_configuration |
combine(_config_to_merge, recursive=true)
}}"
- name: "Nomad | Merge consul configuration for nomad servers"
when:
- nomad_enable_server
block:
- name: "Nomad | Merge consul default server configuration"
vars:
_config_to_merge: "{{ nomad_consul_integration_server_configuration }}"
ansible.builtin.set_fact:
nomad_consul_integration_configuration: "{{
nomad_consul_integration_configuration |
combine(_config_to_merge, recursive=true)
}}"
- name: "Nomad | Merge consul configuration for nomad clients"
when:
- nomad_enable_client
block:
- name: "Nomad | Merge consul default client configuration"
vars:
_config_to_merge: "{{ nomad_consul_integration_client_configuration }}"
ansible.builtin.set_fact:
nomad_consul_integration_configuration: "{{
nomad_consul_integration_configuration |
combine(_config_to_merge, recursive=true)
}}"
- name: "Nomad | Merge consul tls client configuration"
vars:
_config_to_merge: "{{ nomad_consul_integration_client_tls_configuration }}"
ansible.builtin.set_fact:
nomad_consul_integration_configuration: "{{
nomad_consul_integration_configuration |
combine(_config_to_merge, recursive=true)
}}"
when:
- nomad_consul_integration_configuration.ssl is defined
- nomad_consul_integration_configuration.ssl | bool
- name: "Nomad | Merge consul block into main configuration"
vars:
_config_to_merge:
consul: "{{ nomad_consul_integration_configuration }}"
ansible.builtin.set_fact:
hashicorp_nomad_configuration: "{{
hashicorp_nomad_configuration |
combine(_config_to_merge, recursive=true)
}}"
- name: "Nomad | Merge TLS configuration"
vars:
_config_to_merge:
tls: "{{ nomad_tls_configuration }}"
ansible.builtin.set_fact:
hashicorp_nomad_configuration: "{{
hashicorp_nomad_configuration |
combine(_config_to_merge, recursive=true)
}}"
when: nomad_enable_tls
- name: "Nomad | Merge plugin configuration"
vars:
_config_to_merge:
plugin: "{{
nomad_driver_configuration |
combine(nomad_driver_extra_configuration, recursive=true)
}}"
ansible.builtin.set_fact:
hashicorp_nomad_configuration: "{{
hashicorp_nomad_configuration |
combine(_config_to_merge, recursive=true)
}}"
when: "'nomad_clients' in group_names"
- name: "Nomad | Merge extra configuration settings"
vars:
_config_to_merge: "{{ nomad_extra_configuration }}"
ansible.builtin.set_fact:
hashicorp_nomad_configuration: "{{
hashicorp_nomad_configuration |
combine(_config_to_merge, recursive=true)
}}"
when: nomad_extra_configuration is defined
- name: "Print nomad configuration"
ansible.builtin.debug:
msg: "{{ hashicorp_nomad_configuration }}"

View File

@ -38,13 +38,13 @@
state: present
when: _consul_vault_policy.changed
- name: "Include ednz_cloud.hashistack.hashicorp_consul"
- name: "Include ednz_cloud.hashicorp_consul"
ansible.builtin.include_role:
name: ednz_cloud.hashistack.hashicorp_vault
name: ednz_cloud.hashicorp_vault
- name: "Initialize vault cluster" # noqa: run-once[task]
ednz_cloud.hashistack.vault_init:
api_url: "{{ hashi_vault_configuration['api_addr'] }}"
api_url: "{{ hashicorp_vault_configuration['api_addr'] }}"
tls_verify: "{{ vault_tls_verify }}"
key_shares: "{{ vault_seal_configuration['key_shares'] }}"
key_threshold: "{{ vault_seal_configuration['key_threshold'] }}"
@ -77,7 +77,7 @@
- name: "Unseal the bootstrap node" # noqa: run-once[task] no-handler
ednz_cloud.hashistack.vault_unseal:
api_url: "{{ hashi_vault_configuration['api_addr'] }}"
api_url: "{{ hashicorp_vault_configuration['api_addr'] }}"
tls_verify: "{{ vault_tls_verify }}"
key_shares: "{{ _credentials.vault['keys'] }}"
run_once: true
@ -87,7 +87,7 @@
- name: "Unseal all vault nodes"
ednz_cloud.hashistack.vault_unseal:
api_url: "{{ hashi_vault_configuration['api_addr'] }}"
api_url: "{{ hashicorp_vault_configuration['api_addr'] }}"
tls_verify: "{{ vault_tls_verify }}"
key_shares: "{{ _credentials.vault['keys'] }}"
retries: 5

View File

@ -13,8 +13,8 @@
_config_to_merge:
service_registration: "{{ vault_service_registration_configuration }}"
ansible.builtin.set_fact:
hashi_vault_configuration: "{{
hashi_vault_configuration |
hashicorp_vault_configuration: "{{
hashicorp_vault_configuration |
combine(_config_to_merge)
}}"
when: vault_enable_service_registration
@ -24,8 +24,8 @@
_config_to_merge:
plugin_directory: "{{ vault_plugin_directory }}"
ansible.builtin.set_fact:
hashi_vault_configuration: "{{
hashi_vault_configuration |
hashicorp_vault_configuration: "{{
hashicorp_vault_configuration |
combine(_config_to_merge)
}}"
when: vault_enable_plugins
@ -34,8 +34,8 @@
vars:
_config_to_merge: "{{ vault_logging_configuration }}"
ansible.builtin.set_fact:
hashi_vault_configuration: "{{
hashi_vault_configuration |
hashicorp_vault_configuration: "{{
hashicorp_vault_configuration |
combine(_config_to_merge)
}}"
when: vault_enable_log_to_file
@ -44,8 +44,8 @@
vars:
_config_to_merge: "{{ vault_extra_configuration }}"
ansible.builtin.set_fact:
hashi_vault_configuration: "{{
hashi_vault_configuration |
hashicorp_vault_configuration: "{{
hashicorp_vault_configuration |
combine(_config_to_merge)
}}"
when: vault_extra_configuration is defined

View File

@ -1,5 +1,6 @@
---
consul:
gossip_encryption_key: "{{ _consul_gossip_encryption_key }}"
root_token:
secret_id: "{{ _consul_root_token }}"
tokens:
@ -9,5 +10,14 @@ consul:
vault:
accessor_id: "{{ _consul_vault_accessor }}"
secret_id: "{{ _consul_vault_token }}"
nomad:
server:
accessor_id: "{{ _consul_nomad_server_accessor }}"
secret_id: "{{ _consul_nomad_server_token }}"
client:
accessor_id: "{{ _consul_nomad_client_accessor }}"
secret_id: "{{ _consul_nomad_client_token }}"
nomad:
root_token: "{{ _nomad_root_token }}"
gossip_encryption_key: "{{ _nomad_gossip_encryption_key }}"
root_token:
secret_id: "{{ _nomad_root_token }}"

View File

@ -0,0 +1,153 @@
#!/usr/bin/python
from __future__ import absolute_import, division, print_function
from typing import Tuple
__metaclass__ = type
DOCUMENTATION = r"""
---
module: ednz_cloud.hashistack.nomad_acl_bootstrap
short_description: Manages the ACL bootstrap of HashiCorp Nomad.
description:
- This module bootstraps HashiCorp Nomad ACL, ensuring that it is securely set up for use.
requirements:
- C(requests) (L(Python library,https://requests.readthedocs.io/en/latest/))
options:
api_url:
description: The URL of the HashiCorp Nomad API.
required: true
type: str
bootstrap_secret:
description:
- The secret to use for the bootstrap operation.
required: false
type: str
tls_verify:
description:
- Whether to verify the TLS certificate of the Nomad API URL.
- Default is true.
required: false
type: bool
default: true
author:
- Bertrand Lanson (@ednz_cloud)
"""
EXAMPLES = r"""
# Example: Bootstrap HashiCorp Nomad ACL with default settings
- name: Bootstrap HashiCorp Nomad ACL
ednz_cloud.hashistack.nomad_acl_bootstrap:
api_url: https://nomad.example.com
# Example: Bootstrap HashiCorp Nomad ACL with a custom bootstrap secret
- name: Bootstrap HashiCorp Nomad ACL with custom settings
ednz_cloud.hashistack.nomad_acl_bootstrap:
api_url: https://nomad.example.com
bootstrap_secret: 2b778dd9-f5f1-6f29-b4b4-9a5fa948757a
"""
RETURN = r"""
state:
description:
- Information about the state of HashiCorp Nomad after ACL bootstrap.
- This is a complex dictionary with details of the bootstrap.
type: dict
returned: always
sample:
- AccessorID: "b780e702-98ce-521f-2e5f-c6b87de05b24",
- SecretID: "3f4a0fcd-7c42-773c-25db-2d31ba0c05fe",
- Name: "Bootstrap Token",
- Type: "management",
- Policies: null,
- Global: true,
- CreateTime: "2017-08-23T22:47:14.695408057Z",
- CreateIndex: 7,
- ModifyIndex: 7
"""
from ansible.module_utils.basic import AnsibleModule
import traceback
try:
import requests
except ImportError:
HAS_REQUESTS = False
REQUESTS_IMPORT_ERROR = traceback.format_exc()
else:
REQUESTS_IMPORT_ERROR = None
HAS_REQUESTS = True
def bootstrap_nomad_acl(
api_url: str, tls_verify: bool, bootstrap_secret: str
) -> Tuple[bool, dict]:
payload = {}
if bootstrap_secret:
payload["BootstrapSecret"] = bootstrap_secret
response = None
try:
response = requests.post(
f"{api_url}/v1/acl/bootstrap", json=payload, verify=tls_verify
)
response.raise_for_status()
return True, response.json()
except requests.exceptions.HTTPError as e:
if response is not None and response.status_code == 400:
try:
error_message = response.json().get(
"Errors", ["Nomad ACL bootstrap already done"]
)[0]
except ValueError:
error_message = response.text
return False, {"message": error_message}
raise ValueError(f"Nomad ACL bootstrap failed: {str(e)}")
except requests.exceptions.RequestException as e:
raise ValueError(f"Nomad ACL bootstrap failed: {str(e)}")
def run_module():
module_args = dict(
api_url=dict(type="str", required=True),
bootstrap_secret=dict(type="str", required=False, no_log=True),
tls_verify=dict(type="bool", required=False, default=True),
)
result = dict(changed=False, state="")
module = AnsibleModule(argument_spec=module_args, supports_check_mode=False)
if not HAS_REQUESTS:
module.fail_json(
msg="Missing required library: requests", exception=REQUESTS_IMPORT_ERROR
)
try:
changed, response_data = bootstrap_nomad_acl(
api_url=module.params["api_url"],
tls_verify=module.params["tls_verify"],
bootstrap_secret=module.params.get("bootstrap_secret"),
)
result["changed"] = changed
result["state"] = response_data
module.exit_json(**result)
except ValueError as e:
module.fail_json(msg=str(e))
def main():
run_module()
if __name__ == "__main__":
main()

View File

@ -44,12 +44,12 @@ author:
EXAMPLES = r"""
# Example: Initialize HashiCorp Vault with default settings
- name: Initialize HashiCorp Vault
my_namespace.my_collection.my_test:
ednz_cloud.hashistack.vault_init:
api_url: https://vault.example.com
# Example: Initialize HashiCorp Vault with custom key shares and threshold
- name: Initialize HashiCorp Vault with custom settings
my_namespace.my_collection.my_test:
ednz_cloud.hashistack.vault_init:
api_url: https://vault.example.com
key_shares: 7
key_threshold: 4

@ -1 +0,0 @@
Subproject commit 56696c3552308225d4e5b91efc8e4bf75d31d2f3

@ -1 +0,0 @@
Subproject commit 738c347df8efd4965eda14167171343be13bed75

View File

@ -3,15 +3,31 @@
roles:
- name: ednz_cloud.manage_repositories
src: https://github.com/ednz-cloud/manage_repositories.git
version: main
- name: ednz_cloud.manage_apt_packages
src: https://github.com/ednz-cloud/manage_apt_packages.git
version: main
- name: ednz_cloud.manage_pip_packages
src: https://github.com/ednz-cloud/manage_pip_packages.git
version: main
- name: ednz_cloud.install_docker
src: https://github.com/ednz-cloud/install_docker.git
version: main
- name: ednz_cloud.docker_systemd_service
src: https://github.com/ednz-cloud/docker_systemd_service.git
version: main
- name: ednz_cloud.deploy_haproxy
src: https://github.com/ednz-cloud/deploy_haproxy.git
version: main
- name: ednz_cloud.deploy_keepalived
src: https://github.com/ednz-cloud/deploy_keepalived.git
version: main
- name: ednz_cloud.hashicorp_nomad
src: https://github.com/ednz-cloud/hashicorp_nomad.git
version: v0.4.0
- name: ednz_cloud.hashicorp_consul
src: https://github.com/ednz-cloud/hashicorp_consul.git
version: v0.2.0
- name: ednz_cloud.hashicorp_vault
src: https://github.com/ednz-cloud/hashicorp_vault.git
version: v0.2.0