Skip to main content

SIEM - 1m Lines of Noise to 1 Line of Music


Security Information and Event Management (or Monitoring) has been a round for a while and was seen as the saviour for compliance initiatives regarding intrusion, abnormal usage, insider threat, Denial of Service attacks and more.

Nearly every computational device will store a record of internal transactions that can be used for monitoring, troubleshooting or forensic analysis.  I recently heard of a murder case using the program history of a washing machine to prove the accused had in fact used the washer, the night of the murder to cleanse away any evidence.  That is probably an extreme example, but any device, script or piece of code worth it's salt will give a verbose view of what is happening either to the console or to a file.

The format of the log file has long been under discussion with several different 'standards' vying to be the standard such as the Common Event Format.  Basically, the transaction history that gets written to the file output should contain the time and date of the transaction, the transaction itself (generally the computational transaction and sometimes a more detailed description) as well as the ID that initiated the transaction and any other context information such as IP address, node or form.

Now the idea behind SIEM is to provide a centralised view of all this log data from network devices such as routers, firewalls and switches, through to directory services, core applications and even proximity door access.  Aggregating the log processing seems like a great idea.  A single point of entry reduces the need for silo'd and point management of so many different log out put streams.  With a centralised view comes with it the opportunity to perform some intelligent analysis of all this information.  The word all shouldn't be underestimated, with many verbose log streams recording basic network activity or logon / logoff activity shifting a million plus lines of information a day when aggregated.

Trying to analyse so much data requires strict rules to help remove the noise and false positives.  One of the key themes the SIEM solution should be able to perform is that of correlation - the linking of the different log streams based on a unique identifier.  Now, not all log data will contain the same identifier.  The directory log on may use email address, whilst a mainframe application may use a userid, whilst the firewall may only contain source and destination IP addresses.  In an ideal world, the SIEM system should be able to correlate and provide a 360 degree view of user activity from door access to packet delivery.  So we have a linked up and centralised view of activity.  Now what?

Like any alerting system, it's important to know what is an alert and what is a false positive.  The process for defining an alert will vary but could include static policy definition.  Creating a criteria that is deemed to be a security risk and flagging if the criteria is met.  For example, if a user accesses a particular file or performs a certain transaction, or the number of requests from a certain source IP address reaches a certain threshold.  To create those policies requires an analysis of previous unsecure activities in order to view what is a risk in the future.

Another approach is to analyse existing 'normal' behaviour and then identify anything that falls outside of this baseline.  This form of behavioural analysis can provide much more depth and scope of alerting.  Baselining can be done by grouping log events together either based on their origin or time, or by individuals and the teams or jobs their perform.  Once a baseline of information has been created, it can be easier to identify deviances and in turn manage the potential risk associated with them.  A deviance could be an activity that has occurred out of hours or from different IP address.  Access exceptions can also be tracked, by looking for individuals who have access above and beyond a typical baselined user from a particular group.

With any large volume of data, it's important to develop an intelligence layer to help drive where limited resources should be focussed.  SIEM solutions are a step in the right direction by creating a platform that can allow further intelligence and context to be applied.

(Simon Moffatt)

Popular posts from this blog

Customer Data: Convenience versus Security

Organisations in both the public and private sector are initiating programmes of work to convert previously physical or offline services, into more digital, on line and automated offerings.  This could include things like automated car tax purchase, through to insurance policy management and electricity meter reading submission and reporting.

Digitization versus Security

This move towards a more on line user experience, brings together several differing forces.  Firstly the driver for end user convenience and service improvement, against the requirements of data security and privacy.  Which should win?  There clearly needs to be a balance of security against service improvement.  Excessive and prohibitive security controls would result in a complex and often poor user experience, ultimately resulting in fewer users.  On the other hand, poorly defined security architectures, lead to data loss, with the impact for personal exposure and brand damage.

Top 5 Security Predictions for 2016

It's that time of year again, when the retrospective and predictive blogs come out of the closet, just before the Christmas festivities begin.  This time last year, the 2015 predictions were an interesting selection of both consumer and enterprise challenges, with a focus on:


Customer Identity ManagementThe start of IoT security awarenessReduced Passwords on MobileConsumer PrivacyCloud Single Sign On
In retrospect, a pretty accurate and ongoing list.  Consumer related identity (cIAM) is hot on most organisation's lips, and whilst the password hasn't died (and probably never will) there are more people using things like swipe login and finger print authentication than ever before.

But what will 2016 bring?


Mobile Payments to be Default for Consumers

2015 has seen the rise in things like Apple Pay and Samsung Pay hitting the consumer high street with venom.  Many retail outlets now provide the ability to "tap and pay" using a mobile device, with many banks also offer…

Online-ification: The Role of Identity

The Wikipedia entry for Digital Transformation, "refers to the changes associated with the application of digital technology in all aspects of human society".  That is a pretty broad statement.

An increased digital presence however, is being felt across all lines of both public and private sector initiatives, reaching everything from being able to pay your car tax on line, through to being able to order a taxi based on your current location.  This increased focus on the 'online-ification' of services and content, drives a need for a loosely coupled and strong view of an individual or thing based digital identity.