{"id":46368,"date":"2017-02-24T12:09:27","date_gmt":"2017-02-24T06:39:27","guid":{"rendered":"http:\/\/www.tothenew.com\/blog\/?p=46368"},"modified":"2017-02-24T12:09:27","modified_gmt":"2017-02-24T06:39:27","slug":"how-to-run-fabric-using-aws-lambda-and-paramiko","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/how-to-run-fabric-using-aws-lambda-and-paramiko\/","title":{"rendered":"How to run Fabric using AWS Lambda and Paramiko?"},"content":{"rendered":"

We usually come across few scenarios where we need something to run on the remote servers for the sake of automation. I came across such a use case in AWS<\/a> where I need to run fabric on a server to be triggered from AWS Lambda<\/a>. This fabric script further does few tasks on other servers.<\/p>\n

\"fab\"\"aws-lambda\"<\/p>\n

Background\/Scenario:<\/b> While working on a spot instance, my architecture needs few files to be updated when a spot instance comes up or goes down. AWS Lambda gets triggered by the spot instance according to the events. Since AWS Lambda supports few language\/platforms<\/a> including Python, I thought using Fabric would be a good idea to trigger from Lambda but Fabric needs \u201cfab\u201d binary to get executed and AWS Lambda does allow us to install anything on it. So, incorporated one EC2 instance on which few DevOps tools<\/a> such as Chef <\/a>and Jenkins <\/a>were already running and installed Fabric on it.<\/p>\n

Now the next task was to execute the fabfile present on the server. Since Python on AWS Lambda does not have Paramiko module, I had build a package which contains the Paramiko module along with my pPthon code and uploaded the same on AWS Lambda. This would trigger Fabric configured on the server.<\/p>\n

Here are the steps which can help you to implement the above scenario.<\/p>\n

    \n
  1. Install Fabric on a server and create a fabfile at any desired location containing your logic (say \/opt\/fabfile.py). This could be done on any existing EC2 instance running any utility service as Fabric would need few resource to run.a. Install a virtualenv on the system:\n

    [js]sudo pip install virtualenv
    \n#Or
    \nsudo apt-get install python-virtualenv[\/js]<\/p>\n

    b. Setup\/use virtualenv.<\/p>\n

    [js]#create a folder for your app
    \nvirtualenv ~\/my_app
    \n#Activate the virtualenv
    \ncd ~\/my_app\/bin
    \nsource activate[\/js]<\/p>\n

    c. Install the Paramiko module inside the virtual environment:<\/p>\n

    [js]pip install paramiko[\/js]<\/p>\n

    d. Add your Python source file:<\/p>\n

    [js]cd ~\/my_app
    \nvi my_script.py
    \nimport sys
    \nimport time
    \nimport select
    \nimport paramiko
    \ndef functionName(event, context):
    \n host = ‘x.x.x.x’
    \n i = 1
    \n while True:
    \n try:
    \n ssh = paramiko.SSHClient()
    \n ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    \n ssh.connect(host,username="user",password="Pasword",port=22)
    \n print "Connected to %s" % host
    \n Break
    \n except paramiko.AuthenticationException:
    \n print "Authentication failed when connecting to %s" % host
    \n sys.exit(1)
    \n Except:
    \n print "Could not SSH to %s, waiting for it to start" % host
    \n i += 1
    \n time.sleep(2)
    \n # If we could not connect within time limit
    \n if i == 30:
    \n print "Could not connect to %s. Giving up" % host sys.exit(1)
    \n # Send the command (non-blocking)
    \n stdin, stdout, stderr = ssh.exec_command("sudo \/usr\/bin\/fab -f \/home\/username\/fabfile.py copy")
    \n print "Start Execuitng Fab"
    \n """
    \n while stdout.readline():
    \n print stdout.readline()
    \n ssh.close()
    \n """
    \n while not stdout.channel.exit_status_ready():
    \n # Only print data if there is data to read in the channel
    \n if stdout.channel.recv_ready():
    \n rl, wl, xl = select.select([stdout.channel], [], [], 0.0)
    \n if len(rl) > 0:
    \n # Print data from stdout
    \n print stdout.channel.recv(512),
    \n #
    \n # Disconnect from the host
    \n #
    \n print "Command done, closing SSH connection"
    \n ssh.close()
    \nfunctionName("asdfsafd","Asdfasdf")[\/js]<\/p>\n

    e. Copy all content from lib\/python2.7\/site-packages and lib64\/python2.7\/site-packages directories into ~\/myapp and compress the whole package into a zip file.<\/li>\n

  2. Create a zip package to upload on an AWS Lambda function.<\/li>\n
  3. Upload the zip on the AWS Lambda function. Against \u201cHandler\u201d field in AWS Lambda function write my_script.functionName. After running it, errors could come which are to the unavailability of few \u201c.so\u201d. Add the required \u201c.so\u201d file under \u201cmy_app\u201d and update the zip.<\/span><\/li>\n
  4. After all required \u201c.so\u201d file is available, the Lambda function would do the desired task.<\/span><\/li>\n<\/ol>\n

    This use case is typically helpful for the scenario where we want to perform few tasks from a server which we have used as our utility server for which we have few scripts ready and needs to be triggered after an event.<\/span><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"

    We usually come across few scenarios where we need something to run on the remote servers for the sake of automation. I came across such a use case in AWS where I need to run fabric on a server to be triggered from AWS Lambda. This fabric script further does few tasks on other servers. […]<\/p>\n","protected":false},"author":506,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":29},"categories":[1174,2348,1],"tags":[1853,1679,1780,4440,4441,1358,4442],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/46368"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/506"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=46368"}],"version-history":[{"count":0,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/46368\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=46368"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=46368"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=46368"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}