Your SIEM Is Not an Analytics Platform
And Why You Might Need One
If you’re like most teams, you get your security metrics primarily from two sources:
Dashboards and reports within security tooling
Queries against a SIEM
It’s no surprise then, that each month your team spends way too many hours cobbling together reporting, exports, and screenshots into some sort of metrics report. But there’s a fundamental reason this sprint must occur every month: your SIEM is not an analytics warehouse.
You might be saying, but of course it is! How else would you centralize, store, and query across large amounts of security data? That’s literally what it was designed for.
Yes, but no.
Why are you logging into vendor tools to pull metrics instead of landing that data in your SIEM tool with all the rest? The answer for many is that the data is not used for detection, and their ingestion strategy already limits what they ingest to only the needed data to keep costs manageable.
Ah! So the data in the SIEM is optimized for detection usefulness, not usefulness to the business. This also impacts how the SIEM is architected by the vendor, as they need to optimize for real-time alerting on structured security data, where your metrics program needs to optimize for cost-effective, long-term storage and ad-hoc, deep analysis.
What happens when you need to analyze historical data? You may be able to look back at the last quarter, but is your SIEM retaining hot data from 4 months ago? 7? Last year? For many teams, readily available data retention beyond a year is not realistic.
The reality is your SIEM is not set up to support more holistic analysis of your security program. Using the right tools (ranging from a simple Postgres database for smaller programs to a full lake house like Databricks for teams with serious data volume) will streamline your metrics program. It takes some investment and some new skills such as data modeling and pipelines, but opens up a realm of possibilities you didn’t have before while giving back time to your team to do what they were hired to do: security.