{"id":47271,"date":"2017-03-30T18:09:36","date_gmt":"2017-03-30T12:39:36","guid":{"rendered":"http:\/\/www.tothenew.com\/blog\/?p=47271"},"modified":"2017-03-31T09:57:31","modified_gmt":"2017-03-31T04:27:31","slug":"how-to-use-dynamic-inventory-for-aws-with-ansible","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/how-to-use-dynamic-inventory-for-aws-with-ansible\/","title":{"rendered":"How to Use Dynamic Inventory for AWS with Ansible?"},"content":{"rendered":"

Do you know how Ansible works? Well, before we walk you through how to setup dynamic inventory, here’s some food for thoughts on Ansible Configuration Management System<\/a><\/p>\n

A user using configuration management system will often want to save inventory in a different software system. As described in inventory, a basic text-based system is provided by Ansible. Some examples include pulling inventory from a cloud provider, LDAP, Cobbler, and a piece of expensive enterprise CMDB software.<\/p>\n

These options are supported by Ansible through an external inventory system. Some of these such as EC2\/Eucalyptus, Rackspace Cloud, and OpenStack are already a part of the\u00a0contrib\/inventory directory.<\/p>\n

Maintaining an inventory file might not be the best approach if you use Amazon Web Services EC2<\/a>. This is because the hosts may vary over time, they can be managed by external applications, or you might be using AWS autoscaling. In such a case, you can use the\u00a0EC2 external inventory<\/a>\u00a0script.<\/p>\n

Setting up EC2 External Inventory Script With Ansible<\/b><\/p>\n

One way to setup an ec2 external inventory script is to copy the script to \/etc\/Ansible\/ec2.py and chmod +x it. You will also need to copy the <\/span>ec2.ini<\/span><\/a> file to \/etc\/Ansible\/ec2.ini. Once done, you can run Ansible as you would normally do.<\/span><\/p>\n

For making a successful API call to AWS,\u00a0you will need to configure Boto (the Python interface to AWS). There are a\u00a0<\/span>variety of methods<\/span><\/a> available, but the simplest is to export the following environment variables:<\/span><\/p>\n

export AWS_ACCESS_KEY_ID=’AK123′<\/span>
\n<\/span>export AWS_SECRET_ACCESS_KEY=’abc123′<\/span>
\n<\/span><\/p>\n

Now, you will need to set up a few more environment variables to the inventory management script such as\u00a0<\/span><\/p>\n

$ export ANSIBLE_HOSTS=\/etc\/ansible\/ec2.py<\/strong>\u00a0<\/em>This variable\u00a0tells Ansible to use the dynamic EC2 script instead of a static \/etc\/ansible\/hosts file.\u00a0<\/span>Open up ec2.py in a text editor and make sure that the path to ec2.ini config file is defined correctly at the top of the script:<\/p>\n

$ export EC2_INI_PATH=\/etc\/ansible\/ec2.ini <\/strong>This variable tells ec2.py where the ec2.ini config file is located.<\/span><\/p>\n

You can test the script to make sure your config is correct:<\/p>\n

cd \/etc\/ansible<\/span>
\n<\/span>.\/ec2.py –list<\/span><\/p>\n

After some time, you should be able to see the entire EC2 inventory across all regions in JSON. You can have a look at the scripts of EC2.py<\/a> and EC2.ini<\/a><\/span><\/p>\n

Understanding Tags and Host variables<\/b><\/p>\n

Let us first understand how tags in external inventory work followed by how we can use them. Here’s an example playbook command:<\/span><\/p>\n

\/etc\/ansible\/playbooks$ ansible-playbook docker-update.yml –extra-vars \u00a0\u00a0\u00a0\u201cvariable_host=tag_Ecs_true:tag_User_ec2:&tag_Environment_stage\u201d<\/b><\/p>\n

In the above example:<\/span><\/p>\n

    \n
  1. ansible-playbook is the standard command to run a playbook<\/span><\/li>\n
  2. docker-update.yml is the name of our playbook inside the playbooks folder<\/span><\/li>\n
  3. –extra-vars is a parameter which tells the main command to take the following string as an input for the playbook file<\/span><\/li>\n
  4. variable_host is the variable that is being called inside the playbook to place a value for the hosts<\/span><\/li>\n
  5. tag_Ecs_true:tag_User_ec2:&tag_Environment_stage: These are tags and a way to tell playbook where to run the tasks, which user and key file to use to access the hosts<\/span><\/li>\n<\/ol>\n

    Whenever Ansible sees tag_Ecs_true<\/em>, it tries to find all the ec2s<\/em> in AWS account that has the tag Name \u2018Ecs\u2019 and Value \u2018true\u2019.<\/span><\/p>\n

    Simultaneously, Ansible also searches a group variable (inside group_vars folder)<\/em> file with the name tag_Ecs_true<\/em> for any host variable that we may have provided in that file.\u00a0<\/span>The above stands true for all and any \u2018tag_abc_xyz\u2019 provided along with the ansible-playbook command.<\/p>\n

    Let’s break these tags and see what they mean<\/span><\/p>\n

    Key file management:<\/b> tag_Environment_stage — We need to tag all the ec2s in our account with the environment tag and respective value. We make use of this tag for providing the location of the key file required to access those ec2s. As explained earlier, Ansible will search inside the group_vars folder for a file named tag_Environment_stage, where we have already mentioned\u00a0<\/span>ansible_ssh_private_key_file: ~\/.ssh\/environment-staging.pem<\/p>\n

    This\u00a0is nothing but the location of the key file. Similarly, we can manage all the key files.<\/span><\/p>\n

    User management:<\/b> tag_User_ec2 — Now, Ansible will again search inside the group_vars folder for a file named tag_User_ec2, where we have already mentioned\u00a0<\/span>ansible_ssh_user: ec2-user. Thus, it uses ec2-user to access the hosts.<\/p>\n

    Where to run the playbook: <\/b>tag_Ecs_true — This enables Ansible to look for all the hosts with the tag Ecs:true Name:value pair. We need to tag all the ECS instances with this tag so that we can make use of it whenever we need to run a playbook on them.<\/span><\/p>\n

    Patterns and Combinations of Tags<\/b><\/p>\n

    We will need various combinations of tags for which we can make use of standard Ansible patterns.<\/span><\/p>\n

    In the previous example, tag_Ecs_true:tag_User_ec2:&tag_Environment_stage, \u2018:\u2019 means that all the hosts in group tag_Ecs_true and tag_User_ec2 are to be managed if they are in the group tag_Environment_stage also \u2018&\u2019.<\/span><\/p>\n

    Please refer patterns document<\/a><\/span>\u00a0for a\u00a0detailed explanation about patterns.<\/span><\/strong><\/p>\n

    Handling Python Version Dependency\u00a0<\/b><\/p>\n