Detection Pipeline Lab
A 7-day sample SOC pipeline. Edit a rule, watch alerts and precision move.
Loading the stream and the rule engine.
How it compares
Checked 2026-11-06. The 27 Sigma documents in this pack were run through sigma-cli 4.0.0 with pySigma 2.0.2 and pySigma-validators-sigmahq 0.21.0, using SigmaHQ's own CI configuration (tests/sigma_cli_conf.yml at commit 52f81d6).
| CI check | Here | Where it comes from, and the result |
|---|---|---|
| Parse, compile, condition syntax | Yes | pySigma default validators: 0 errors, 0 issues |
| Technique tags carry their tactic tags | Yes, v19.2 mapping | SigmaHQ validator. 19 missing tactic tags found and added |
| Author, date, description on every document | Yes, correlations too | SigmaHQ validators. 27 findings on correlation rules, fixed |
| UUID ids, unique ids and titles, date formats, spec status and level | Yes | Sigma specification 2.1.0 and SigmaHQ's duplicate-id job |
| Status against rule age | Yes | SigmaHQ wants a regression_tests_path for test status inside 60 days. Here the embedded tests count, so SigmaHQ's tool still reports 27 |
| Positive and negative tests per rule | 46 tests, run on every edit | SigmaHQ regression tests replay EVTX; Splunk contentctl replays attack_data into a live Splunk |
| Field names against the log schema | Yes, for CloudTrail, Okta, GitHub and Windows | Elastic detection-rules validates queries against ECS and integration schemas |
| Title Case, logsource filename prefixes | Not applied | SigmaHQ repository conventions for contributions (204 findings of that kind) |
- Under SigmaHQ's configuration the pack went from 268 to 231 findings; the 231 left are the repository naming conventions and the status age rule above.
- Known limits: no goodlog run against a benign baseline, no replay into a live SIEM, and no version lock or deprecation workflow as in Elastic's detection-rules.
Detection Pipeline Lab FAQ
What is in the sample stream?
Seven days of Okta System Log, AWS CloudTrail, Kubernetes audit, Microsoft 365 unified audit, OCSF endpoint process records and four application systems for Harbourline Forwarding, a fictional freight forwarder. Four attack chains and a set of benign look-alikes are seeded with labels written in advance, so every alert can be scored as a true or false positive. IP addresses come from the documentation ranges.
How are the detections run?
Each detection is Sigma 2.0 YAML. Base rules are matched by sigma.js, the engine behind the pwnsy Detection Workbench, after an enrichment step that joins helpdesk tickets, the change calendar, on-call incidents and a 30-day baseline. Multi-step detections use Sigma correlation rules (event_count, value_count and temporal_ordered), evaluated by this page's JavaScript.
What does the CI tab check?
Every detection carries positive and negative test events. CI compiles each rule, lints it, checks its ATT&CK technique against the v19.2 snapshot, applies the metadata checks SigmaHQ runs (tactic tags, author, date, description, UUID ids, status against rule age) and runs every test. The same 13 rules and 46 tests run on every push in the public GitHub repository edwardtay/detections-as-code-logistics, which also parses each rule with pySigma.
How are precision and time to detect measured?
Alerts from the stream are labelled from the seeded ground truth and passed through the pwnsy Alert Budget engine. Precision is TP / (TP + FP) per rule with a Wilson 95% range. Time to detect runs from the first malicious event to the alert, including source lag and the rule schedule.
What is the public attack data replay?
Seven small samples from splunk/attack_data (Apache-2.0), copied exactly as published, run through the same pipeline: CloudTrail StopLogging and CreateAccessKey, S3 GetObject, Okta push fatigue, password spray and factor deactivation, and Exchange forwarding rules. Each row links to its source file.
Is anything uploaded?
No. The stream is generated in your browser, every rule runs in JavaScript on this page, and downloads are built locally. Nothing is sent to a server.
ATT&CK IDs come from the pinned MITRE ATT&CK v19.2 snapshot and OCSF classes from the OCSF 1.9.0 snapshot used across the detection suite. Replay samples are from splunk/attack_data under the Apache License 2.0, copied unchanged with a link to each source file. All other records are generated and fictional.