While none of the the configuration management tools I’ve used are perfect, Ansible is by far my top pick. There is a bit of a learning curve though. I think most people’s experience with Ansible goes something like this:
- Review the technically correct and verbose docs.
- Decide to google for real-world examples, instead.
- Notice that there are many different ways to organize a playbook, and that there is no clear Right Way of doing it.
- Realize you haven’t made any progress on your task, panic a little bit, and just start hacking at a playbook until it works well enough without caring about organization or future reusability.
Then, after some time passes, you get a new project that requires Ansible:
- Copy and paste from your most recent Ansible repo and make improvements until things work adequately.
- Repeat step #1 forever.
This iterative process is kludgey and makes it hard to see the progression (and sense) of your work unless you remember the order of all the projects you’ve worked on. And later when you go back to work on an existing project none of it makes sense because it was several iterations ago and you’ve made many changes and improvements since then. (“What idiot wrote this crap?! Oh right .. it was me.”)
Thankfully Ansible has a way of reusing shared code: Roles. I think people’s experience with creating their first Ansible role goes something like this:
- Read about the directory structure of roles, the various places they can exist on disk, and try to envision an approach to conveniently develop both your playbook and the role simultaneously.
- Stare out the window for a few minutes and whisper to yourself “…I just need to place a few files, a user account, and a cronjob.”
- Give up and return to copy/paste
In this Lab Note I offer a simple approach to wrangle shared Ansible roles that works well for us.
But first, what goes in a role?
It’s up to you, but you really want to have some clear boundaries for each role so that it has a single, well, “role”. Maybe you have a common role that is included in EVERY project. Maybe 95% of the time you install Postgres — therefore your “common” role should NOT manage Postgres. Maybe you need to give analytics tools access to a few of your Postgres databases — that should be it’s own role.
Here are examples of roles we have at Cofense Labs.
cfl-common:
- Set timezone to UTC, as god intended
- Set the hostname and add it to /etc/hosts
- Harden the sshd service
- Manage things in /etc/skel
- Configure a system-wide profile.d config (set the default bash prompt to clearly identify if this is a development, production, or other environment)
- Create a service user
- Manage various public keys for access and encryption
- Configure mail to relay through AWS SES (if this is an EC2 instance)
cfl-postgres:
- Install and configure Postgres (pg_hba.conf, etc.)
- Create a database for the service and install common extensions (like citext)
- Create scripts in /usr/local/bin to properly dump, load, and conduct an encrypted offsite backup of the database
- Manage a crontab for encrypted offsite backups
cfl-mariadb:
- Same as cfl-postgres, but for MariaDB
cfl-data-analytics:
- Create a shell account for our data analytics tools with proper access control
- Create a readonly Postgres (or MariaDB) user for those services to use
cfl-nginx-passenger:
- Include the geerlingguy.passenger role
- Remove the default config that geerlingguy.passenger puts in place
- Manage nginx HTTP and HTTPS configurations for the web application
- Include the geerlingguy.certbot role (if this is not development environment)
- Generate a self-signed cert/key (if this is a development environment)
- Ensure app directory exists with proper permissions
cfl-deploy-rails-app:
- Deploy a Rails application (handled differently for development vs not)
- Manage config/secrets.yml using values from the Ansible vault
- Bundle gems
- Migrate the database
- Precompile assets
- Reload nginx
Just keep in mind that not EVERYTHING needs to go in a role. One-off, service-specific things should still be in your playbook (or its tasks/ directory, if you prefer).
Create your playbook
Lets assume your git server is at my-git-server.example.com and you’re working on something called “projectx”. This project has a service that lives at [email protected]/projectx/projectx-service.git.
Create a playbook and push it up to [email protected]/projectx/projectx-ansible.git. (Actually, you don’t have to push it up right now if you don’t want to, but you do have to at least `git init` it.)
Create your role
If you don’t have one already, create a place on your git server to house your shared Ansible roles (for example, a Gitlab group or sub-group). Lets assume: my-git-server.example.com/ansible-roles.
Now create an empty new Ansible role called “my-common” using the `ansible-galaxy` command and push it up to your git server. (Note that even though we’re using `ansible-galaxy`, we are NOT going to push your role to the public Ansible Galaxy service.)
$ cd ~/git/work/ansible-roles
$ ansible-galaxy init my-common
$ cd my-common
$ git init
$ git remote add origin git@my-git-server.example.com/ansible-roles/my-common.git
$ git add .
$ git commit -m "Initial commit"
$ git push -u origin master
Try to put some thought into the name and location of your role. You can change it later, but it’s a bit tedious.
Add your role as a git submodule
Git submodules make working with private Ansible roles a breeze. You can have a bunch of outstanding changes in your playbook and the role without affecting anyone else that might be cloning the role while you’re hacking on it, and your playbook’s repo locks the role to a specific git commit, allowing it to work forever — including on fresh clones — even if there are breaking changes made to the role in the future. This allows you to put off updating old playbooks that are using old versions of a role until you have the spare cycles to do that. Or you can just not worry about them if they’re old projects in maintenance mode, because they’ll still work forever with the old versions of the role.
Yes, instead you could use a requirements.txt file to grab roles from a private git server and manage your own semantic versioning for all your roles and coordinate that with your team. Personally I find using git submodules much, much easier …and it prevents you from having to have meetings with your team about version numbers.
To add your role as a submodule in your playbook repo’s roles/ directly, you simply:
$ cd ~/git/work/projectx/projectx-ansible
$ mkdir roles
$ git submodule add git@my-git-server.example.com/ansible-roles/my-common.git roles/my-common
You don’t even have to commit the submodule at this point. You can just get to work.
Develop your role
The `ansible-galaxy` command did some scaffolding — your new role looks like this:
.
├── README.md
├── defaults
│ └── main.yml
├── files
├── handlers
│ └── main.yml
├── meta
│ └── main.yml
├── tasks
│ └── main.yml
├── templates
├── tests
│ ├── inventory
│ └── test.yml
└── vars
└── main.yml
From the Ansible Roles documentation:
- tasks/main.yml – the main list of tasks that the role executes.
- handlers/main.yml – handlers, which may be used within or outside this role.
- library/my_module.py – modules, which may be used within this role (see Embedding modules and plugins in roles for more information).
- defaults/main.yml – default variables for the role. These variables have the lowest priority of any variables available, and can be easily overridden by any other variable, including inventory variables.
- vars/main.yml – other variables for the role.
- files/main.yml – files that the role deploys.
- templates/main.yml – templates that the role deploys.
- meta/main.yml – metadata for the role, including role dependencies.
You don’t need to worry about all that out of the gate, but they’re good to know about. You’ll find myself mostly focusing on:
- defaults
- tasks
- templates
In your playbook, you simply need to include the role:
roles:
- my-common
vars:
hostname: “{{ the_hostname }}”
secret_password: “{{ password_from_vault }}”
Now you can develop your role and playbook side-by-side. When things work, commit changes in the submodule repo, and then commit the submodule (the new git commit) and other changes in the playbook repo.
The next time you need to use my-common in another playbook, check it out as a submodule in projecty-ansible/roles/my-common, iterate, improve, and push without ever breaking anything else already using your shared role.
All third-party trademarks referenced by Cofense whether in logo form, name form or product form, or otherwise, remain the property of their respective holders, and use of these trademarks in no way indicates any relationship between Cofense and the holders of the trademarks.
