Exabeam is an organisation that has tirelessly invested in its built-in material. You only have to peruse their GitHub to find content for vendors/products that are utilised within your business and hundreds of others.

But what happens when you can’t find your log source? Panic sets in. You come to the realisation that you will be going down the custom route. A perilous endeavour? An impossible task? Fear not! In this blog we will simplify the process of incorporating bespoke logs.

Do a double check!

First things first, we need to be 100% sure your log source is not supported. It is not uncommon for products to come under different monikers/abbreviations. Additionally, vendors often acquire established products, leading to changes in branding or availability. For example, you may have issues locating Bluecoat ProxySG until you learn that they were bought by Symantec, and Blue Coat Systems is no longer operational. In this scenario, GitHub’s search feature becomes an invaluable tool, allowing you to explore the repository and uncover relevant information within. By carefully investigating these possibilities, you can avoid unnecessary customizations and find the right solution for your log source needs.

Measuring the value

It is wise to evaluate a potential custom log source and its benefit to your environment, specifically in the context of Advanced Analytics threat detection capabilities. Yes, that in-house database application is key to your company’s day-to-day – but what are the event types that it is generating? Could it assist with detecting anomalous activity? There is a chance you will be putting unnecessary load on the AA engine and eating up your license quota without providing any usefulness in return.

Unleashing the Power of Field Extraction

Field extraction is the foundation of your bespoke content. It involves understanding the logs, determining the relevant information to extract, and discerning its value. Having a knowledge of your business’s needs and use cases will help determine what information needs to be plucked out as well as what will be deemed superfluous.

Making sure your logs have a timestamp and hostname/user field is imperative.  Without these foundational elements, any further extractions will yield futile results. When we observe the data flow in Advanced Analytics, we see logs > events > sessions > models > rules but how can AA build these sessions without knowing when the event occurred and which user or entity’s session it should be stitched to? This roadblock will result in exclusion from models and subsequently rules – rendering AA’s capabilities redundant.

Maintain a good naming scheme

It goes without saying that deploying an efficient and consistent naming scheme will benefit all parties involved. If you are the type of person who likes to alphabetize their spice rack then this may very well come naturally, however, it will most likely take a conscious effort to set and maintain discernible order within your custom content.

This convention applies to all bespoke components by which the log traverses – parsers, event builders, rules, etc. Naming files in a specific way makes them easy to locate and identify. Applying versioning will help differentiate older content from updated content.

Cover all bases

When it comes to subject matter, it is wise to take a comprehensive approach. This involves dedicating a substantial portion of sample logs to develop the content. It is not uncommon for a log source to present 2 or 3 different structures for the same event type. By gathering a broad range of sample logs, you can adequately prepare for various scenarios and protect against any parsing errors.

Make sure your sample pool includes all possible event types for said data source – Including those that are in the >1%! If it is being forwarded to Advanced Analytics, it is assumed useful so all possibilities must be considered before the work begins.

Running a distinct search on your logs across a generous timespan will assist in locating these.

If you are engaging a consultant for your custom content, then it is vital to provide them with a minimum of 10,000 logs (including all event types of course!) to ensure all eventualities are covered.

Regex One-oh-One

Custom parser creation is heavily predicated upon writing Regex (Regular Expression) which performs rapid pattern-matching to extract the relevant field values from your logs.

If you decide to write your own Regex, be sure to deploy efficient syntax. Often, bad/incorrect Regex will be disabled by the Exabeam engine and greedy Regex will put unnecessary strain on your environment’s performance.

Free online tools like Regex101 are crucial in the validation phase and will provide a visual colour-coding for correct matching and capture groups.

APG & Log Stream

Exabeam provides capabilities for creating custom content through the following online tools:

• APG (Auto-Parser Generator) – CIM 1.0 compliant
• Log Stream – CIM 2.0 compliant

These applications are designed to make your life easier, and they certainly do!

Firstly, they provide an intuitive, Regex-101-like interface for you to upload your samples, write your regular expression and capture your fields. You will be made aware of any faulty Regex using the familiar colour-coding.

Secondly, it will create your content packs automatically, a task that previously opened the door to syntax errors – APG/Log Stream will generate a custom_mojito.conf , event_builder.conf, manifest.conf & parsers.conf and zip them up for you

Thirdly, you will have a central repository for all your custom and OOTB content, allowing for simplified collaboration.

Both products are available to Exabeam SaaS customers. APG is now considered legacy as it is only CIM 1.0 compliant, Log Stream is CIM 2.0 compliant and provides additional capabilities to monitor parser health as well as providing a view into the ingestion pipeline.

If you’re facing challenges with your Exabeam instance or need assistance with log source onboarding, we’re here to help. We can help you streamline the process, enhance your threat detection capabilities, and stay one step ahead of potential security risks.

Get in touch with us today to unlock the full potential of Exabeam’s security intelligence platform.