Splunk Interview Questions and Answers

Q1 : What is Splunk indexer? What are stages of Splunk indexing?
A : The indexer is the Splunk Enterprise component that creates and manages indexes. The primary functions of an indexer are:
1. Indexing incoming data.
2. Searching the indexed data.

Q2 : Define Splunk. Why is it used for the analysis of machine data?
A : Splunk is a great tool to allow visibility into data generated from machines like hardware devices, servers, IoT devices, networks, and other sources.
It is used for analyzing machine data because it provides crucial insights into IT operations, application management, compliance, security, threat and fraud detection.

Q3 : Why Splunk when there are other open-source options?
A : In terms of data analysis, doing business intelligence, providing security and managing IT operation, Splunk faces tough competition. It stands ahead because it is the only tool that can manage all the operations. This is where it makes a difference and helps you scale up your business infrastructure. There could be other options, but they need plugins to support a few features like customer support and taking any data type at input.

Q4 : What is Splunk DB connect?
A : Splunk DB Connect is a generic SQL database plugin for Splunk that allows you to easily integrate database information with Splunk queries and reports.

Q5 : What are buckets? Explain splunk bucket lifecycle?
A : Splunk places indexed data in directories, called as “buckets”. It is physically a directory containing events of a certain period. A bucket moves through several stages as it ages:
Hot – Contains newly indexed data. Open for writing. One or more hot buckets for each index.
Warm – Data rolled from hot. There are many warm buckets.
Colld – Data rolled from warm. There are many cold buckets.
Frozen – Data rolled from cold. The indexer deletes frozen data by default, but you can also archive it. Archived data can later be thawed (Data in frozen buckets is not searchable)
By default, your buckets are located in $SPLUNK_HOME/var/lib/splunk/defaultdb/db. You should see the hot-db there, and any warm buckets you have. By default, Splunk sets the bucket size to 10GB for 64bit systems and 750MB on 32bit systems

Q6 : What is ‘license violation’ in Splunk?
A : “License Violation” occurs when you exceed the data limit. Commercial licensing gets 5 warnings whereas the free version has only 3.

Q7 : What is command for restarting just the splunk webserver?
A : Splunk start splunkweb

Q8 : Explain the use of License Master in Splunk?
A : License Master ensures that your Splunk environment remains within the limits of purchased volume. It also makes sure that the indexers within the Splunk deployment have sufficient capacity to license the right amount of data.

Q9 : What is sourcetype in splunk?
A : Sourcetype is Splunk way of identifying data

Q10 : How to clear Splunk search history?
A : Delete following file on Splunk server
$splunk_home/var/log/splunk/searches.log

Q11 : What is difference between Splunk app and Splunk add on?
A : Basically both contains preconfigured configuration and reports etc, but Splunk add on do not have visual app. Splunk apps have preconfigured visual app.

Q12 : Splunk licenses specify what?
A : How much data you can index per calendar day

Q13 : How are forwarder licenses purchased?
A : They are included with Splunk, no need to purchase separately

Q14 : What is summary index in splunk?
A : The Summary index is the default summary index (the index that plunk Enterprise uses if you do not indicate another one). If you plan to run a variety of summary index reports you may need to create additional summary indexes.

Q15 : What are most important configuration files of Splunk OR can you tell name of few important configuration files in Splunk?
A : props.conf
indexes.conf
inputs.conf
transforms.conf
server.conf

Q16 : What is Splunk app?
A : Splunk app is container/directory of configurations, searches, dashboards , etc in splunk

Q17 : How to troubleshoot splunk performance issues?
A : Check splunkd.log for any errors
Check server performance issues i.e. cpu/memory usag,disk i/o etc
Install SOS (Splunk on Splunk) app and check for warning and errors in dashboard
Check number of saved searches currently running and their system resources consumption
Install Firebug, which is a firefox extension. After it’s installed and enabled, log into Splunk (using firefox), open firebug’s panels, switch to the ‘Net’ panel (you will have to enable it). The Net panel will show you the HTTP requests and responses along with the time spent in each. This will give you a lot of information quickly over which requests are hanging Splunk for a few seconds, and which are blameless. etc..

Q18 : Which is latest Splunk version in use?
A : Lates Version Release – Splunk 6.3

Q19 : Where Splunk default configuration does is stored?
A : $splunkhome/etc/system/default

Q20 : What is Command to enable Splunk to boot start?
A : $SPLUNK_HOME/bin/splunk enable boot-start

Q21 : What is fishbucket or what is fishbucket index?
A : It’s a directory or index at default location /opt/splunk/var/lib/Splunk .It contains seek pointers and CRCs for the files you are indexing, so splunkd can tell if it has read them already. We can access it through GUI by searching for “index=_thefishbucket”

Q22 : What is dispatch directory?
A : $SPLUNK_HOME/var/run/splunk/dispatch contains a directory for each search that is running or has completed. For example, a directory named 1434308943.358 will contain a CSV file of its search results, a search.log with details about the search execution, and other stuff. Using the defaults (which you can override in limits.conf), these directories will be deleted 10 minutes after the search completes – unless the user saves the search results, in which case the results will be deleted after 7 days.

Q23 : What is difference between search head pooling and search head clustering?
A : Both are features provided Splunk for high availability of splunk search head in case anyone search head goes down. Search head cluster is newly introduced and search head pooling will be removed in next upcoming versions. Search head cluster is managed by captain and captain controls its slaves. Search head cluster is more reliable and efficient than search head pooling.

Q24 : What is MapReduce algorithm?
A : MapReduce algorithm is secret behind Splunk fast data searching speed. It’s an algorithm typically used for batch-based large scale parallelization. It’s inspired by functional programming’s map() and reduce () functions.

Q25 : How splunk avoids duplicate indexing of logs?
A : At indexer splunk keeps track of indexed events in a directory called fish buckets (default location /opt/splunk/var/lib/splunk). It contains seek pointers and CRCs for the files you are indexing, so splunkd can tell if it has read them already. – See more at: https://www.learnsplunk.com/splunk-indexer-configuration.html#sthash.t1ixi19P.dpuf