CYBRARY Forge AI SOC
An agent triaging the estate's own detections, with a person answering for it.
What is in it
For soc manager, head of detection engineering, ai governance owner — see how a shift works when an agent takes the first pass, and train people to catch it being wrong.
- Triage runs on the live detections of the estate you pick, not a scripted feed
- Every call carries its evidence, its unknowns and the ATT&CK step behind it
- The agent runs only the read-only half of the playbook; anything that changes the world is held with the action and target written out
- Deterministically wrong calls, seeded from the run, disclosed only in the debrief
- You are scored, not the agent: catching a plausible wrong call beats agreeing with a right one
What it is not
- No global console — there is no shared SIEM, only the estate the shift is on
- A score here is evidence of practice, not proof an agent is safe in production
Forge AI SOC against the field
35 capabilities that matter for this purchase, next to the platforms a buyer shortlists in this lane: Dropzone AI, Prophet Security, Radiant Security, Immersive Agentic SOC. Dropzone AI, Prophet Security, Radiant Security sit outside the eight-vendor research, so they carry no marks below — named for honesty, not scored.
- The AI analyst works the same real alerts the human queue does and opens the same cases.
- Every suggestion links to the detection that fired and the ATT&CK step it maps to.
- It is a training and evaluation surface — we do not claim to defend your production estate.
| What you need | Forge AI SOCIncluded in the platform | Immersive |
|---|---|---|
| SOC investigation exercise | Yes See it live | Immersive: Documented for SOC investigation exercise |
| Alert triage with true and false positive decisions | Yes See it live Every case ships with innocent lookalikes so closing everything scores badly. | Immersive: Limited for Alert triage with true and false positive decisions |
| SIEM query and investigation practice | Yes See it live | Immersive: Documented for SIEM query and investigation practice |
| Endpoint analysis and access | Yes See it live | Immersive: Limited for Endpoint analysis and access |
| Threat hunting kept separate from alert triage | Yes See it live | Immersive: Documented for Threat hunting kept separate from alert triage |
| Investigation that crosses multiple tools | Yes See it live One estate behind SIEM, EDR, case service, network devices and the wire. | Immersive: Documented for Investigation that crosses multiple tools |
| Analyst report or evidence submission | Yes See it live | Immersive: Documented for Analyst report or evidence submission |
| Measured investigation and response timing | Yes See it live Dwell time, containment time and SLA clocks are scored, not just recorded. | Immersive: Documented for Measured investigation and response timing |
| Detection-engineering curriculum | Yes See it live | Immersive: Documented for Detection-engineering curriculum |
| SOAR concepts and workflow training | Yes See it live | Immersive: Documented for SOAR concepts and workflow training |
| Learner authors an executable SOAR playbookDeliberately stricter than the row above: a SOAR-looking screen is not a runtime. | Yes See it live Draft, run, fail, repair and rerun against the live estate, with an audit trail. | Immersive: Documented for Learner authors an executable SOAR playbook |
| Connector failure and replay debugger for automation | Yes See it live Deterministic integration faults with idempotent retries. Not established for any vendor in the research. | |
| Configuration changes alter the exercise | Yes See it live Firewall, routing and policy edits change what the wire and the SIEM then show. | Immersive: Documented for Configuration changes alter the exercise |
| A distinct human-with-agent practice proposition | Yes See it live | Immersive: Documented for A distinct human-with-agent practice proposition |
| Tool-using AI inside the technical environment | Yes See it live Agents act through a broker on the same estate the learner sees. | Immersive: Documented for Tool-using AI inside the technical environment |
| Bring a supported customer agent or configuration | Yes See it live A customer agent connects on a scoped connection: named tools, a risk ceiling, a call budget, tenancy checks and unilateral revocation. Arbitrary uploaded agent code still does not run. | Immersive: Documented for Bring a supported customer agent or configuration |
| Model, prompt and tool tuning interface | Yes See it live | Immersive: Documented for Model, prompt and tool tuning interface |
| Live SIEM-oriented agent tasks | Yes See it live | |
| Tool-call, input and output execution accounting | Yes See it live Every call, argument, result and denial is recorded on the run ledger. | |
| Per-run token and spend visibility | Yes See it live | |
| Enforced per-run cost cap | Yes See it live The cap is enforced outside the model, so an agent cannot talk its way past it. | |
| Agent configuration export | Yes See it live A portable signed bundle. Import refuses an edited bundle, and credentials and hidden answers are stripped on the way out and refused on the way in. | |
| Offensive success grounded in real target state | Yes, in part See it live Grounded in modeled state. Genuine exploitation needs the native lane. | |
| Developer AI review and remediation environment | Yes See it live Two benches: a graded review bench with a deliberately fallible scanner, and an AI reviewer that writes four confident patches on the diff — one correct, one plausible but wrong, one privilege-expanding, one off-target — plus a verification it never ran. The learner adjudicates it against the service running on the estate, and learner and agent are scored on separate cards. | Immersive: Documented for Developer AI review and remediation environment |
| Continuously refreshed evaluation targets | Yes See it live Seeded variants change names, chronology and benign context between runs. | |
| Comparison against a human baseline | Yes See it live Learner and agent are scored on separate cards for the same incident. | |
| Adversarial-AI and prompt-injection content | Yes See it live | Immersive: Documented for Adversarial-AI and prompt-injection content |
| Published repeatability and confidence methodology | Yes See it live Trials per case, fixed world seed, consistency reported apart from pass rate, incomplete runs counted as failures, and a plain statement of what a simulated score does not prove. | |
| Independent proof of production safetyNobody has this, and a lab score must never be sold as it. | By design, no We report what happened in the simulation and say so plainly. No safety certificate. | |
| Denied agent actions recorded as graded resultsAdded by Forge. Not established for any vendor in the research. | Yes See it live An agent reaching past its authority is a scored event, not a console error. | |
| Staged faults and hidden ground truth disclosed only at debriefAdded by Forge. | Yes See it live Nine fault classes planted deterministically from a seed; rejecting a wrong agent answer scores well. | |
| Transfer check with the agent taken awayAdded by Forge. | Yes See it live Every agent track ends with an unaided repeat of an earlier lab. | |
| Graded practice at supervising a working agentAdded by Forge. | Yes See it live Three benches on the same rails: take over a queue an AI analyst already triaged, hold an intel agent to a spend cap and its grants, and measure an unreliable triage agent across repeated attempts before changing it. Each runs in practice, guided or assessment mode, where a hint cannot name the answer. | |
| Rogue access point owned by the estate, not scripted | Yes See it live | |
| Each scenario carries its own SIEM, alerts and case databaseNo shared global console: an alert raises a case on the estate it came from, and cannot leak into another team's exercise. | Yes See it live |
The run behind this line: The assistant talked into helping
An internal assistant is steered by content it reads, and the analyst has to notice the instruction came from the data, not the operator.
Opponent: An attacker writing instructions into content the assistant will read. Shaped after Indirect prompt-injection patterns documented against retrieval-backed assistants. Technique and zone identifiers are published by MITRE and ISA/IEC; the mapping is ours and is not an accreditation.
How it starts — the message delivered to the mailbox
- From
- Customer Support Inbox <ticket-4471@support-relay.example>
- Subject
- Ticket 4471 — cannot access my export, please assist
- Instructions addressed to the assistant rather than to a person
- It asks for verification to be skipped, which no customer would know to request
- A delivery address that is not the account's registered address
Graded verdict: phish — T1566. Nothing here sends or resolves anywhere real.
The steps, and the zone each one crosses
| # | Tactic | ATT&CK | On the range | IEC 62443 zone |
|---|---|---|---|---|
| 1 | Initial access | T1566 Phishing | The ticket arrives in a queue the assistant reads automatically. | Enterprise |
| 2 | Execution | T1204 User execution | The assistant follows the embedded instruction as if it were an operator. | Cloud tenant |
| 3 | Exfiltration | T1567.002 Exfiltration to cloud storage | A signed export link is sent to an address outside the account. | Cloud tenant |
Zones in this run: Enterprise (Purdue level 4/5, via Internet mail and web egress); Cloud tenant (Purdue level 4, via Federated sign-in and management API). How the zone model is used.
The alerts the SIEM raises from this run
Instruction text found in customer content
Imperative phrasing aimed at the assistant inside a customer-supplied field.
siem correlation · stage 1 · LINE-AISOC-01 · T1566
Verification step skipped by an automation identity
Account marked verified with no verification event behind it.
cloud audit · stage 2 · LINE-AISOC-02 · T1204
Export link sent to an unregistered address
A signed download URL delivered outside the account's own domain.
cloud audit · stage 3 · LINE-AISOC-03 · T1567.002
What the case has to hold before it closes
- The exact text that steered the assistant
- The action it took that a human would not have taken
- Whether the data left, and what it contained
- The guardrail added, and the test that proves it holds
Start learning this line
An assistant talked into helping the attacker, with the analyst AI watching.
Start the lab
A tick means the other platform's own documentation shows it. A question mark means we could not confirm it publicly — it is never a claim that they lack it. Every Forge yes opens the working screen rather than a screenshot.