![\"Screenshot](\"\/blog\/wp-ttn-blog\/uploads\/2016\/07\/Screenshot-from-2016-07-14-105339.png\")
<\/p>\nAs we know Python supports multiple approaches for concurrent programming with threads, sub-processes and some other ways which could help achieving solutions built on multiple CPUs or multi-core CPU.<\/p>\n
<\/p>\n
<\/p>\n
I tried implementing something similar on my existing use case for <\/span>AWS Security Re-Check<\/span><\/a> where I was running a check on AWS for my responsibilities over security. Executing that along with other similar scripts for other categories consumed a lot of time since they were running as a single process and in a sequential fashion. Also since it was a single sequential process, it was able to make use of a single core at a time. So to counter this, I tried to bring parallelism in my script. Let me tell you the two ways I tried with:<\/span><\/p>\n It is the usage of more than one CPU within a single machine. In Python, for <\/span>multiprocessing<\/span><\/a>, there is a package named multiprocessing, By multiprocessing package, a user can fully leverage the multiple processors on a machine. Process are produced by creating objects of Process class of multiprocessing package. Each process has a separate memory space so it is difficult to share objects in multiprocessing.<\/span><\/p>\n In my <\/span>script<\/a>,<\/span> I had multiple functions, and I divided each function into a process and started all the processes, by this I could now run many processes concurrently which earlier used to take place in series, hence reducing the execution time. By multiprocessing, all CPUs of my machine are parallelly processing different functions of my script, also, now the total time to run that script reduced by about ~50%.<\/span><\/p>\n These functions were being called serially initially, like:<\/span><\/p>\n [sourcecode language=”python”] But, after multiprocessing, these were called parallelly as independent processes. [sourcecode language=”python”] By creating an\u00a0object and calling function start() of Process class, the process is started. join() function indicates the program to wait for the process to complete before proceeding to the main process.<\/p>\n Hence, all functions that have to be executed as separate processes can be run by creating a function (here runInParallel), wherein function can run as an individual independent process. \u00a0<\/span><\/p>\n In multithreading, multiple lightweight processes called threads are created which share the same memory pool so precautions have to be taken, or two threads will write the same memory at the\u00a0same time. In Python, for multithreading, there is an inbuilt package \u2018<\/span>multithreading<\/span><\/a>\u2019. By creating objects of Thread class of multithreading package, threads can be easily created.<\/span>\u00a0<\/strong><\/strong><\/p>\nMultiprocessing<\/span><\/h2>\n
\ndef ServerHosting():
\ndef Security_group_opened_for_all():
\ndef Cloudtrail_Status():
\ndef MFA_enabled():
\ndef Access_Permissions_on_S3():
\ndef iam_key_rotate():
\n[\/sourcecode]<\/p>\n
\nIt was achieved as follows:<\/p>\n
\nfrom multiprocessing import Process
\ndef runInParallel(*fns):
\n\u00a0 \u00a0 \u00a0 \u00a0proc=[]
\n \u00a0 \u00a0 \u00a0for fn in fns:
\n \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0process=Process(target=fn)
\n \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0process.start()
\n \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0proc.append(process)
\n\u00a0 \u00a0 \u00a0 \u00a0for process in proc:
\n \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0process.join()
\nrunInParallel(ServerHosting,Security_group_opened_for_all,cloudtrail_Status,MFA_enabled,Access_permissions_on_s3,iam_key_rotate)
\n[\/sourcecode]<\/p>\nMultithreading<\/span><\/h2>\n
![\"img3\"](\"\/blog\/wp-ttn-blog\/uploads\/2016\/07\/img3.png\")
<\/p>\n![\"img4\"](\"\/blog\/wp-ttn-blog\/uploads\/2016\/07\/img4.png\")
<\/p>\n