TIDIR Target System Architecture
This document defines the target component architecture for the unified Threat Intelligence, Detection, Investigation & Response (TIDIR) platform.
1. System Topology Diagram
The architecture operates across two orthogonal dimensions:
- The Operational Runtime Plane: Four horizontal layers governing event ingestion, computation, detection, and mitigation.
- The Engineering Lifecycle Plane: Five vertical disciplines governing schemas, intelligence curation, Detection-as-Code (DaC), systems automation, and AI harnesses.
2. Layer Definitions & Operational Responsibilities
Layer 1: Data Sources & Environmental Inputs
- Generation, Collection & Transport: Emits raw facts at the point of origin (kernel hooks, control plane APIs, wire taps), buffers at the edge, and transports across network boundaries via secure, compressed streams.
- Multidimensional Inputs: Unifies runtime operational telemetry with external cyber threat intelligence (CTI), organizational context (asset CMDB, directory hierarchies), attack surface exposure (EASM), and security control posture.
- See full spec: Layer 1 Specification.
Layer 2: Pipeline, Storage & Query Fabric
- Line-Rate Normalization: Standardizes raw payloads into Open Cybersecurity Schema Framework (OCSF) objects via an authoritative Schema Registry.
- Value-Based Routing: Diverts high-value security events to hot indexing and stream engines while streaming bulk forensic telemetry into low-cost columnar lakehouse storage.
- Multi-Paradigm Querying: Provides four specialized engines: Real-Time Streaming (< 5s), Scheduled Batch SQL (7–90 day baselines), Federated Query-in-Place, and ML Feature Stores.
- See full spec: Layer 2 Specification.
Layer 3: Threat Intelligence & Detection Engineering
- Machine-Readable Attack Flows: Codifies multi-stage adversary behaviors into structured graphs, prioritizing detection engineering backlogs via threat likelihood and asset exposure.
- Detection-as-Code (DaC): All rules are authored as declarative vendor-neutral code targeting OCSF schema classes, versioned in Git.
- Empirical Test Harness: Validates rules through controlled adversary simulation, synthetic unit tests, and 30-day historical lakehouse backtesting.
- Standardized Findings: Emits OCSF Class 2001 (Security Finding) and Class 2004 (Detection Finding) objects.
- See full spec: Layer 3 Specification.
Layer 4: Incident Response (Investigation, Case Management & SOAR)
- Entity Resolution & Interactive Graph: Synthesizes parent-child process trees, identity pivots, and chronological event timelines from Layer 2 storage.
- Tamper-Evident Evidence Dossier: Records queries, annotations, and artifacts with cryptographic integrity for post-incident review (PIR).
- Blast-Radius Gated SOAR: Separates autonomous low-risk containment (Tier 1) from disruptive actions (Tier 2) requiring signed human-in-the-loop authorization.
- Closed-Loop Feedback: Directly feeds novel IOCs discovered during triage back into Layer 1/3 threat intelligence and rule calibration.
- See full spec: Layer 4 Specification.
3. Data Contracts Across the Architecture
| Boundary | Schema Contract | Purpose |
|---|---|---|
| L1 ➔ L2 Ingress | Native / Schema Registry Envelope | Bounded transport batch carrying origin metadata and raw event facts. |
| L2 Normalization | OCSF (Open Cybersecurity Schema Framework) | Canonical schema across system, identity, network, cloud, and application domains. |
| L3 Detection Target | OCSF Classes (1001, 1007, 3002, 4001, etc.) | Vendor-neutral detection logic decoupled from physical database columns. |
| L3 ➔ L4 Handoff | OCSF Class 2001 & Class 2004 Findings | Standardized security and detection findings carrying evidence, ATT&CK tags, and risk scores. |
| L4 Containment | Declarative Action Specifications | Parameterized containment payloads executed against third-party API connectors. |