feat(vault): wrote some more documentation on using the tool

docs/quick_start.md

@@ -101,3 +101,13 @@ ansible-galaxy collection install ednxzu.hashistack:==<version>
 ```
 
 You should now have a directory under `./collections/ansible_collections/ednxzu/hashistack`
+
+8. Install the other dependencies required by `ednxzu.hashistack`
+
+```bash
+ansible-galaxy install -r ./collections/ansible_collections/ednxzu/hashistack/roles/requirements.yml
+```
+
+This will install roles that are not packaged with the collection, but are still required in order to run the playbooks.
+
+You should now have some roles inside `./roles/`.

docs/vault_clusters.md

@@ -5,3 +5,111 @@ This documentation explains each steps necessary to successfully deploy a Vault
 ## Prerequisites
 
 You should, before attempting any deployment, have read through the [Quick Start Guide](./quick_start.md). These steps are necessary in order to ensure smooth operations going forward.
+
+## Variables
+
+### Basics
+
+First, in order to deploy a Vault cluster, you need to enable it.
+
+```yaml
+enable_vault: "yes"
+```
+
+Selecting the vault version to install is done with the `vault_version` variable.
+
+```yaml
+vault_version: latest
+```
+
+The vault version can either be `latest` or `X.Y.Z`.
+
+For production deployment, it is recommended to use the `X.Y.Z` syntax.
+
+The `deployment_method` variable will define how to install vault on the nodes.
+
+By default, it runs vault inside a docker container, but this can be changed to `host` to install vault from the package manager.
+
+```yaml
+deployment_method: "docker"
+```
+
+### General Settings
+
+First, you can change some general settings for vault.
+
+```yaml
+vault_cluster_name: vault
+vault_enable_ui: true
+vault_seal_configuration:
+  key_shares: 3
+  key_threshold: 2
+```
+
+### Storage Settings
+
+The storage configuration for vault can be edited as well. By default, vault will be configured to setup `raft` storage between all declared vault servers (in the `vault_servers` group).
+
+```yaml
+vault_storage_configuration:
+  raft:
+    path: "{{ hashi_vault_data_dir }}/data"
+    node_id: "{{ ansible_hostname }}"
+    retry_join: |
+      [
+      {% for host in groups['vault_servers'] %}
+        {
+          'leader_api_addr': 'http://{{ hostvars[host].api_interface_address }}:8200'
+        }{% if not loop.last %},{% endif %}
+      {% endfor %}
+      ]
+```
+
+While this is the [recommended](https://developer.hashicorp.com/vault/docs/configuration/storage#integrated-storage-vs-external-storage) way to configure storage for vault, you can edit this variable to enable any storage you want. Refer to the [vault documentation](https://developer.hashicorp.com/vault/docs/configuration/storage) for compatibility and syntax details about this variable.
+
+Example:
+
+```yaml
+# MySQL storage configuration
+vault_storage_configuration:
+  mysql:
+    address: "10.1.10.10:3006"
+    username: "vault"
+    password: "vault"
+    database: "vault"
+```
+
+### Listener Settings
+
+#### TCP Listeners
+
+By default, TLS is **disabled** for vault. This goes against the Hashicorp recommendations on the matter, but there is no simple way to force the use of TLS (yet), without adding a lot of complexity to the deployment.
+
+The listener configuration settings can be modified in `vault_listener_configuration` variable.
+
+```yaml
+vault_listener_configuration:
+  tcp:
+    address: "0.0.0.0:8200"
+    tls_disable: true
+```
+By default, vault will listen on all interfaces, on port 8200. you can change it by modifying the `tcp.address` property, and adding you own listener preferences.
+
+#### Enabling TLS for Vault
+
+In order to enable TLS for Vault, you simply need to set the `vault_enable_tls` variable to `true`.
+
+At the moment, hashistack-Ansible does nothing to help you generate the certificates and renew them. All it does is look inside the `etc/hashistack/vault_servers/tls` directory on the deployment node, and copy the files to the destination hosts in `/etc/vault.d/config/tls`. The listener expect **2 files** by default, a `cert.pem`, and a `key.pem` file.
+
+Please refer to the [vault documentation](https://developer.hashicorp.com/vault/docs/configuration/listener/tcp) for details bout enabling TLS on vault listeners.
+
+In case you want to add more configuration to the vault listeners, you can add it to the `vault_extra_listener_configuration` variable, which by default is empty. This variable will be merge with the rest ofthe listener configuration variables, and takes precedence over all the others.
+
+> **Waring**
+> At the moment, hashistack-ansible does not support setting up multiple TCP listeners. Only one can be set.
+
+### Plugins for Vault
+
+To enable plugin support for Vault, you can set the `vault_enable_plugins` variable to true. This variable will add the necessary configuration options in the vault.json file to enable support. Once enabled, you can simply place your compiled plugin files into the `etc/hashistack/vault_servers/plugin` directory. They will be copied over to the `/etc/vault.d/config/plugin` directory on the target nodes.
+
+Refer to the [vault documentation](https://developer.hashicorp.com/vault/docs/plugins/plugin-management) for details about enabling and using plugins.

playbooks/deploy.yml

@@ -35,7 +35,7 @@
           ansible.builtin.include_role:
             name: ednxzu.hashistack.hashicorp_consul
 
-        - name: "Initialize consul cluster"
+        - name: "Initialize consul cluster" # noqa: run-once[task]
           ednxzu.hashistack.consul_acl_bootstrap:
             api_addr: "{{ hashi_consul_configuration['advertise_addr'] }}"
           run_once: true
@@ -45,9 +45,20 @@
           register: _consul_init_secret
           until: not _consul_init_secret.failed
 
-        - name: "Print consul token"
+        - name: "Write consul configuration to file"  # noqa: run-once[task] no-handler
-          ansible.builtin.debug:
+          ansible.builtin.copy:
-            msg: "{{ _consul_init_secret }}"
+            content: "{{ _consul_init_secret.state | to_nice_yaml}}"
+            dest: "{{ sub_configuration_directories.consul_servers }}/consul_config"
+            mode: '0644'
+          when: _consul_init_secret.changed
+          run_once: true
+          delegate_to: localhost
+
+        - name: "Load consul cluster variables"
+          ansible.builtin.include_vars:
+            file: "{{ sub_configuration_directories.consul_servers }}/consul_config"
+            name: _consul_cluster_config
+
 
     - name: "Vault"
       when:

playbooks/group_vars/all.yml

@@ -3,7 +3,7 @@
 # General options ########
 ##########################
 
-enable_vault: "no"
+enable_vault: "yes"
 enable_consul: "yes"
 enable_nomad: "no"
 

plugins/modules/consul_acl_bootstrap.py

@@ -6,12 +6,54 @@ from typing import Tuple
 __metaclass__ = type
 
 DOCUMENTATION = r"""
+---
+module: ednxzu.hashistack.consul_acl_bootstrap
+
+short_description: Bootstraps ACL for a Consul cluster.
+
+version_added: "1.0.0"
+
+description:
+    - This module bootstraps ACL (Access Control List) for a Consul cluster. It performs the ACL bootstrap operation,
+      creating the initial tokens needed for secure communication within the cluster.
+
+options:
+    api_addr:
+        description: The address of the Consul API.
+        required: true
+        type: str
+    scheme:
+        description: The URL scheme to use (http or https).
+        required: false
+        type: str
+        default: http
+    port:
+        description: The port on which the Consul API is running.
+        required: false
+        type: int
+        default: 8500
+
+author:
+    - Bertrand Lanson (@ednxzu)
 """
 
 EXAMPLES = r"""
+# Example: Bootstrap ACL for a Consul cluster
+- name: Bootstrap ACL for Consul cluster
+  ednxzu.hashistack.consul_acl_bootstrap:
+    api_addr: 127.0.0.1
+    scheme: http
+    port: 8500
 """
 
 RETURN = r"""
+state:
+    description: Information about the state of ACL bootstrap for the Consul cluster.
+    type: dict
+    returned: always
+    sample:
+        accessor_id: "uuuuuuuu-uuuu-iiii-dddd-111111111111",
+        secret_id: "uuuuuuuu-uuuu-iiii-dddd-222222222222"
 """
 
 from ansible.module_utils.basic import AnsibleModule
@@ -40,7 +82,7 @@ def bootstrap_acl(scheme: str, api_addr: str, port: int) -> Tuple[bool, dict]:
             "secret_id": response.json()["SecretID"],
         }
     elif response.status_code == 403:
-        return False, "Cluster has already been bootstrapped"
+        return False, {"message": "Cluster has already been bootstrapped"}
     else:
         response.raise_for_status()  # Raise an exception for other status codes
 
@@ -54,11 +96,7 @@ def run_module():
 
     result = dict(changed=False, state="")
 
-    module = AnsibleModule(argument_spec=module_args, supports_check_mode=True)
+    module = AnsibleModule(argument_spec=module_args, supports_check_mode=False)
-
-    api_addr = module.params["api_addr"]
-    scheme = module.params["scheme"]
-    port = module.params["port"]
 
     try:
         if not HAS_REQUESTS:
@@ -68,9 +106,10 @@ def run_module():
                 )
             )
 
-        # Perform ACL Bootstrap
         acl_bootstrap_result, response_data = bootstrap_acl(
-            api_addr=api_addr, port=port
+            scheme=module.params["scheme"],
+            api_addr=module.params["api_addr"],
+            port=module.params["port"],
         )
 
         result["changed"] = acl_bootstrap_result

plugins/modules/vault_init.py

@@ -1,6 +1,7 @@
 #!/usr/bin/python
 
 from __future__ import absolute_import, division, print_function
+from typing import Tuple
 
 __metaclass__ = type
 
@@ -10,11 +11,12 @@ module: ednxzu.hashistack.vault_init
 
 short_description: Manages the initialization of HashiCorp Vault.
 
-version_added: "1.0.0"
-
 description:
     - This module initializes HashiCorp Vault, ensuring that it is securely set up for use.
 
+requirements:
+  - C(hvac) (L(Python library,https://hvac.readthedocs.io/en/stable/overview.html))
+
 options:
     api_url:
         description: The URL of the HashiCorp Vault API.
@@ -55,22 +57,25 @@ EXAMPLES = r"""
 
 RETURN = r"""
 state:
-    description: Information about the state of HashiCorp Vault after initialization.
+    description:
-    type: complex
+        - Information about the state of HashiCorp Vault after initialization.
+        - This is a complex dictionary with the following keys:
+            - keys
+            - keys_base64
+            - root_token
+        - If the vault is already initialized, it will return a simple dict with a message stating it.
+    type: dict
     returned: always
-    sample: {
+    sample:
-        "keys": [
+        keys:
-            "wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww",
+            - wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
-            "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
+            - xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-            "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"
+            - yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
-        ],
+        keys_base64:
-        "keys_base64": [
+            - wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww
-            "wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww",
+            - xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-            "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
+            - yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
-            "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy"
+        root_token: hvs.zzzzzzzzzzzzzzzzzzzzzzzz
-        ],
-        "root_token": "hvs.xxxxxxxxxxxxxxxxxxxxxxxx"
-    }
 """
 
 from ansible.module_utils.basic import AnsibleModule
@@ -86,6 +91,20 @@ else:
     HAS_HVAC = True
 
 
+def initialize_vault(
+    api_url: str, key_shares: int, key_threshold: int
+) -> Tuple[bool, dict]:
+    client = hvac.Client(url=api_url)
+
+    try:
+        if not client.sys.is_initialized():
+            return True, client.sys.initialize(key_shares, key_threshold)
+        else:
+            return False, {"message": "Vault is already initialized"}
+    except hvac.exceptions.VaultError as e:
+        raise hvac.exceptions.VaultError(f"Vault initialization failed: {str(e)}")
+
+
 def run_module():
     module_args = dict(
         api_url=dict(type="str", required=True),
@@ -95,33 +114,28 @@ def run_module():
 
     result = dict(changed=False, state="")
 
-    module = AnsibleModule(argument_spec=module_args, supports_check_mode=True)
+    module = AnsibleModule(argument_spec=module_args, supports_check_mode=False)
 
     if not HAS_HVAC:
         module.fail_json(
             msg="Missing required library: hvac", exception=HVAC_IMPORT_ERROR
         )
 
-    if module.check_mode:
-        module.exit_json(**result)
-
-    vault_init_result = None
-    client = hvac.Client(url=module.params["api_url"])
-
     try:
-        if not client.sys.is_initialized():
+        vault_init_result, response_data = initialize_vault(
-            vault_init_result = client.sys.initialize(
+            module.params["api_url"],
-                module.params["key_shares"], module.params["key_threshold"]
+            module.params["key_shares"],
+            module.params["key_threshold"],
         )
-            result["state"] = vault_init_result
-    except Exception as e:
-        module.fail_json(msg=f"Vault initialization failed: {str(e)}")
 
-    if vault_init_result:
+        result["changed"] = vault_init_result
-        result["changed"] = True
+        result["state"] = response_data
 
         module.exit_json(**result)
 
+    except ValueError as e:
+        module.fail_json(msg=str(e))
+
 
 def main():
     run_module()

plugins/modules/vault_unseal.py

@@ -1,6 +1,7 @@
 #!/usr/bin/python
 
 from __future__ import absolute_import, division, print_function
+from typing import Tuple
 
 __metaclass__ = type
 
@@ -80,6 +81,18 @@ else:
     HAS_HVAC = True
 
 
+def unseal_vault(api_url: str, key_shares: list) -> Tuple[bool, dict]:
+    client = hvac.Client(url=api_url)
+
+    try:
+        if client.sys.is_sealed():
+            return True, client.sys.submit_unseal_keys(key_shares)
+        else:
+            return False, {"message": "Vault is already unsealed"}
+    except hvac.exceptions.VaultError as e:
+        raise hvac.exceptions.VaultError(f"Vault unsealing failed: {str(e)}")
+
+
 def run_module():
     module_args = dict(
         api_url=dict(type="str", required=True),
@@ -87,15 +100,7 @@ def run_module():
     )
     result = dict(changed=False, state="")
 
-    module = AnsibleModule(argument_spec=module_args, supports_check_mode=True)
+    module = AnsibleModule(argument_spec=module_args, supports_check_mode=False)
-
-    if not HAS_HVAC:
-        module.fail_json(
-            msg="Missing required library: hvac", exception=HVAC_IMPORT_ERROR
-        )
-
-    if module.check_mode:
-        module.exit_json(**result)
 
     client = hvac.Client(url=module.params["api_url"])
 
@@ -103,15 +108,21 @@ def run_module():
         module.exit_json(**result)
 
     try:
-        key_shares = module.params["key_shares"]
+        if not HAS_HVAC:
-        vault_unseal_result = client.sys.submit_unseal_keys(key_shares)
+            module.fail_json(
-        result["state"] = vault_unseal_result
+                msg="Missing required library: hvac", exception=HVAC_IMPORT_ERROR
+            )
+        vault_unseal_result, response_data = unseal_vault(
+            api_url=module.params["api_url"], key_shares=module.params["key_shares"]
+        )
 
         if client.sys.is_sealed():
-            module.fail_json(msg="Vault unsealing failed.")
+            module.fail_json(
-        else:
+                msg="Vault unsealing failed. The unseal operation worked, but the vault is still sealed, maybe you didn't pass enough keys ?"
-            result["changed"] = True
+            )
 
+        result["changed"] = vault_unseal_result
+        result["state"] = response_data
     except hvac.exceptions.VaultError as ve:
         module.fail_json(msg=f"Vault unsealing failed: {ve}")