{"id":44845,"date":"2017-01-05T11:12:25","date_gmt":"2017-01-05T05:42:25","guid":{"rendered":"http:\/\/www.tothenew.com\/blog\/?p=44845"},"modified":"2017-01-16T15:17:35","modified_gmt":"2017-01-16T09:47:35","slug":"why-should-you-use-splunk-for-log-analysis","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/why-should-you-use-splunk-for-log-analysis\/","title":{"rendered":"Why Should You Use Splunk for Log Analysis ?"},"content":{"rendered":"<p>Everyone knows that logs play an important role in the IT industry. Logs are used for various purposes such as IT operations, system and application monitoring, <a title=\"Business Analytics Service\" href=\"http:\/\/www.tothenew.com\/analytics\">business analytics<\/a>, security and compliance and much more.<\/p>\n<p>Having a <a title=\"DevOps Tools\" href=\"http:\/\/www.tothenew.com\/devops-chef-puppet-docker\">centralized logging system<\/a> makes life easy for developers especially when there is a need to troubleshoot the application, detect issues, secure the application due to unexpected hits on services or review the performance of the application, etc. Some of the great features of a centralized logging system are its low-cost maintenance, easy logs searching, graphical UI etc.<\/p>\n<p>Splunk is centralized logs analysis tool for machine generated data, unstructured\/structured and complex multi-line data which provides the following features such as <strong>Easy\u00a0<b>Search\/Navigate,\u00a0Real-Time Visibility,\u00a0Historical Analytics, Reports, Alerts, Dashboards and Visualization.<\/b><\/strong><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-44860\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/splunklive-splunk-for-security-8-638.jpg\" alt=\"splunklive-splunk-for-security-8-638\" width=\"638\" height=\"359\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/splunklive-splunk-for-security-8-638.jpg 638w, \/blog\/wp-ttn-blog\/uploads\/2017\/01\/splunklive-splunk-for-security-8-638-300x168.jpg 300w, \/blog\/wp-ttn-blog\/uploads\/2017\/01\/splunklive-splunk-for-security-8-638-624x351.jpg 624w\" sizes=\"(max-width: 638px) 100vw, 638px\" \/><\/p>\n<p><strong>1)\u00a0Advantages of Splunk and why to use it ?<\/strong><\/p>\n<ul>\n<li style=\"font-weight: 400;\">Analyzes the aggregate of logs from a big service cluster<\/li>\n<li style=\"font-weight: 400;\">Finds real-time logs\u00a0and with faster speed<\/li>\n<li style=\"font-weight: 400;\">Generates report and alerts for the desired search<\/li>\n<li style=\"font-weight: 400;\">Provides enhanced GUI and real-time visibility in dashboard in various formats<\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Provides quick results by reducing the time to troubleshoot and resolve issues\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\">Works like a monitoring, reporting and analysis tool and provides insights<\/li>\n<li style=\"font-weight: 400;\">Does not require other dependent services (like database)<\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Requires minimum HW resources<\/span><\/li>\n<li style=\"font-weight: 400;\">Easy to setup and low-cost maintenance<\/li>\n<li style=\"font-weight: 400;\">Accepts any data type including .csv, JSON log formats etc.<\/li>\n<li style=\"font-weight: 400;\">Monitors AWS infrastructure<\/li>\n<li style=\"font-weight: 400;\">Uploads and indexes log data from a local PC to Splunk directly<\/li>\n<\/ul>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-44870\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/hpproducts-screenshot-light.png\" alt=\"hpproducts-screenshot-light\" width=\"1200\" height=\"684\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/hpproducts-screenshot-light.png 1200w, \/blog\/wp-ttn-blog\/uploads\/2017\/01\/hpproducts-screenshot-light-300x171.png 300w, \/blog\/wp-ttn-blog\/uploads\/2017\/01\/hpproducts-screenshot-light-1024x583.png 1024w, \/blog\/wp-ttn-blog\/uploads\/2017\/01\/hpproducts-screenshot-light-624x355.png 624w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/p>\n<p><strong>2) \u00a0Versions of Splunk<\/strong><\/p>\n<p>Splunk comes in two versions &#8211; Free and Enterprise edition.<\/p>\n<ul>\n<li><strong>Free Version:<\/strong> \u00a0The Splunk Free license is for the low volume of logs, it provides max 500 MB of indexing per day.<\/li>\n<li><strong>Enterprises Version:<\/strong>\u00a0 The Splunk Enterprise and Splunk Cloud licenses supports\u00a0multi-user, distributed deployments. It also offers additional capabilities to support higer data volumes including\u00a0alerting, role-based security, single sign-on, scheduled PDF delivery, clustering, premium Splunk apps, etc.<\/li>\n<\/ul>\n<p>For more details &#8211; https:\/\/www.splunk.com\/en_us\/products\/splunk-enterprise\/free-vs-enterprise.html<\/p>\n<p><strong>3) How to setup Splunk for your infrastructure?<br \/>\n<\/strong>Splunk works on the client-server model. \u00a0Splunk Forwarder\u00a0is used to collect the machine generated data from client side and forward to Splunk server.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-44892\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/diagram_forwarding.png\" alt=\"diagram_forwarding\" width=\"465\" height=\"310\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/01\/diagram_forwarding.png 465w, \/blog\/wp-ttn-blog\/uploads\/2017\/01\/diagram_forwarding-300x200.png 300w\" sizes=\"(max-width: 465px) 100vw, 465px\" \/><\/p>\n<p><strong>3.1: \u00a0Setup a Splunk Servers<\/strong><\/p>\n<p><strong>3.1.1:<\/strong>\u00a0Download and install\u00a0Splunk<\/p>\n<pre>$ cd \/opt\r\n$ wget -O splunk-6.5.1-f74036626f0c-Linux-x86_64.tgz 'https:\/\/www.splunk.com\/bin\/splunk\/DownloadActivityServlet?architecture=x86_64&amp;platform=linux&amp;version=6.5.1&amp;product=splunk&amp;filename=splunk-6.5.1-f74036626f0c-Linux-x86_64.tgz&amp;wget=true'\r\n$ tar -xvzf splunk-6.5.1-f74036626f0c-Linux-x86_64.tgz\r\n$ \/opt\/splunk\/bin\/splunk start<\/pre>\n<p><strong>3.1.2:\u00a0 Enable the receiving port to get logs from Splunk Forwarder.\u00a0<\/strong><\/p>\n<p>We can do this from GUI and CLI as well. Port 9997 is default and it can be changed<\/p>\n<pre>$ \/opt\/splunk\/bin\/splunk enable listen 9997<\/pre>\n<p>Open browser and type\u00a0http:\/\/splunk-server-ip:8000 to access Splunk web console.<\/p>\n<p><strong>3.2:\u00a0Setup Splunk Forwarder<br \/>\n<\/strong><strong>3.2.1: \u00a0Download and install\u00a0Splunk Forwarder<\/strong><\/p>\n<pre>$ cd \/opt\r\n$ wget -O splunkforwarder-6.5.1-f74036626f0c-Linux-x86_64.tgz 'https:\/\/www.splunk.com\/bin\/splunk\/DownloadActivityServlet?architecture=x86_64&amp;platform=linux&amp;version=6.5.1&amp;product=universalforwarder&amp;filename=splunkforwarder-6.5.1-f74036626f0c-Linux-x86_64.tgz&amp;wget=true'\r\n$ tar -xvzf splunkforwarder-6.5.1-f74036626f0c-Linux-x86_64.tgz\r\n$ \/opt\/splunkforwarder\/bin\/splunk start<\/pre>\n<p><strong>3.2.2: &#8211; Add the logs in Splunk Forwarder<\/strong><\/p>\n<pre class=\"line number1 index0 alt2\">$ vim \/opt\/splunkforwarder\/etc\/system\/local\/inputs.conf\r\n[default]\r\nhost = &lt;HOSTAME OF CLIENT&gt;\r\n[monitor:\/var\/log\/secure.log]\r\n[monitor:&lt;other logs path&gt;]<\/pre>\n<div class=\"line number1 index0 alt2\"><strong>3.2.3: &#8211; Configure Splunk Server on Splunk Forwarder.<\/strong><\/div>\n<pre class=\"line number1 index0 alt2\">$ vim \/opt\/splunkforwarder\/etc\/system\/local\/outputs.conf\r\n[tcpout]\r\ndefaultGroup = default-autolb-group\r\n[tcpout:default-autolb-group]\r\nserver = &lt;splunk-server-ip&gt;:9997\r\n[tcpout-server:\/\/splunk-server-ip:9997]<\/pre>\n<div class=\"line number1 index0 alt2\">Now login into Splunk web console and search for configured logs.<\/div>\n<div class=\"line number1 index0 alt2\"><\/div>\n<div class=\"line number1 index0 alt2\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Everyone knows that logs play an important role in the IT industry. Logs are used for various purposes such as IT operations, system and application monitoring, business analytics, security and compliance and much more. Having a centralized logging system makes life easy for developers especially when there is a need to troubleshoot the application, detect [&hellip;]<\/p>\n","protected":false},"author":959,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":281},"categories":[1174,2348,1],"tags":[1789,1784,3805,4360,4361,4354],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/44845"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/959"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=44845"}],"version-history":[{"count":0,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/44845\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=44845"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=44845"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=44845"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}