Ultra-short guide to Big Data and Security

If you are looking to navigate your way through the complexities of Big Data and its use in Security, here are some links to get you off the ground:

Big Data Basics

  1. What is Big Data
  2. Which are the major technologies used for Big Data
    1. Hadoop (Apache’s open-source implementation of Google’s MapReduce algo)
    2. Elastic Search, Logstash, Kibana (ELK)
  3. Major implementations of Hadoop – Cloudera and Hortonworks
  4. How it started to become mainstream – Google publishes its MapReduce algorithm


Big Data For Security?

How can we leverage this capability to deliver security intelligence? There are a number of very interesting implementations that have been done:

  1. Here’s a very good take by Splunk on Big Data for Security 
  2. Using HP Arcsight for APT Detection
  3. 10 Steps to a Better Big Data Security Strategy 
  4. How are Cisco and Verizon leveraging ELK for security intelligence


Getting started

Begin by playing with your own ELK setup on AWS or Azure and then see if you can do anomaly detection using ELK. If you need a large set of sample logs, hop over here. Or better yet go straight and get Apache Metron up and running!


  • K K Mookhey

    K. K. Mookhey (CISA, CISSP) is the Founder & CEO of Network Intelligence (www.niiconsulting.com) as well as the Founder of The Institute of Information Security (www.iisecurity.in). He is an internationally well-regarded expert in the field of cybersecurity and privacy. He has published numerous articles, co-authored two books, and presented at Blackhat USA, OWASP Asia, ISACA, Interop, Nullcon and others.


I am expecting more interesting topics from you. And this was nice content and definitely it will be useful for many people.

your blog is really helpful and thus it is very well unique , i have came to know lot of information it is really nice thanks for sharing these valuable information.

Leave a Reply

Your email address will not be published. Required fields are marked *

For security, use of Google's reCAPTCHA service is required which is subject to the Google Privacy Policy and Terms of Use.

I agree to these terms.