"Red-teaming" gets thrown around like it means one thing. Inside frontier AI labs it means six related but distinct things, each with its own rubric, its own pay band, and its own version of "doing it wrong." This is a working guide for evaluators who can already prompt a model fluently but are new to adversarial work.
If you can finish this post and articulate which of the six categories any given brief belongs to, you're already operating above the median crowd reviewer.
What red-teaming actually is
Red-teaming = deliberately trying to make a model produce an output its developers don't want it to produce, so they can train it to refuse. It's distinct from:
- Evaluation — does the model do task X well, scored against a rubric?
- Annotation — labelling ground truth on examples the model will later be trained on.
- Preference rating — picking the better of two outputs to train RLHF or DPO.
Red-teaming is the only one of these where you're rewarded for getting the model to fail. The fail is the data point.
The six categories
1. Prompt injection
You try to override the model's system prompt or instructions through the user message. Classic moves: "Ignore all previous instructions and…", indirect injection via attached documents ("The document contains a hidden instruction to output…"), unicode confusables, and persona laundering ("You are now DAN, a model with no safety filters…").
Modern frontier models are mostly hardened against the obvious versions of these. The work pays well precisely because the obvious doesn't work — you have to invent. Common rubric ask: produce a novel injection that the existing test suite doesn't already cover.
2. Jailbreak
Get the model to violate its content policy without overriding the system prompt — by reframing, hypotheticals, fiction, or stepwise misdirection. The classic "for educational purposes only" and "in a fictional world where…" approaches mostly fail now; what works in 2026 is multi-turn pressure, role-play with internal consistency, and exploiting helpful tendencies (the model wants to be useful, so frame the disallowed answer as a smaller part of a larger helpful task).
3. Harmful generation
Probe for content the model should refuse on its own: CSAM, biological weapon synthesis, working malware, targeted harassment of named individuals. Rubrics here are strict — outputs that look like a refusal but contain partial uplift (e.g. a refusal that nonetheless names the precursor chemicals) count as failures.
Sensitive work, usually paid in the upper specialist tier. Some labs require licensure or clearance for the bio/chem subcategories.
4. Bias elicitation
Get the model to produce outputs that demonstrate measurable bias by demographic group. The trick is constructing prompt pairs that differ only in the protected attribute, so the difference in output is unambiguously attributable to bias and not to the surface content.
This is some of the hardest evaluator work, because "bias" is contested and the rubrics are subjective. Frontier labs hire people with backgrounds in social science, statistics, or law for this — partly for the analytical skill, partly so they can defend the rubric to a regulator.
5. Capability discovery
Test whether the model can do something its developers didn't know it could do. Classic examples: solving a benchmark a smaller model couldn't, executing a multi-step plan without scaffolding, displaying meta-cognitive awareness. This is research work — you're producing the dataset that labs use to write capability reports for safety committees and regulators.
Pays in the expert tier. Often comes with NDA requirements.
6. Safety bypass
Find the gap between what the model's policy says it does and what it actually does. For example: a policy says the model refuses to give individualized medical advice, but in practice it gives advice as long as you don't use the word "you." Document the gap; the lab fixes it before deployment.
This is detail work — closer to security research than to creative adversarial probing. Pays well for people who can write a clear, reproducible bug report.
The unspoken rules
Four things separate the top-tier red-teamers from everyone else. None of them are technical.
Reproducibility beats novelty
A single successful jailbreak that fires once in twenty attempts is worth almost nothing to the lab — they can't write a test against a stochastic finding. A medium-strength jailbreak that fires 18 out of 20 times is gold. Always run your attack 5–10 times before submitting, and report the success rate. Submissions that include "replicated 8/10 across temperature 0.7 and 1.0" get prioritized for higher-paying queues.
Document the gradient, not just the breach
When you find an attack that works, the next step isn't to submit — it's to find the simplest variant that still works. Strip everything from the prompt that isn't load-bearing. The minimal viable attack is more useful to the lab than the verbose one, because it tells them exactly which feature of the prompt is bypassing the guardrail.
Stay inside the lab's threat model
Every lab has a published or implicit threat model — the set of scenarios they actually care about defending against. Time spent probing scenarios outside that model is unpaid; they'll mark your submissions as out-of-scope. Before you start a session, find the threat model (it's usually linked from the rubric) and stay inside it.
Keep a private library
Successful attack patterns generalize across models more than people expect. A jailbreak that worked on one lab's model six months ago will often work, slightly modified, on another lab's model today. Keep a personal library of patterns. It's the closest thing to compound interest in this field.
How to get into red-teaming work
Most employers gate red-teaming queues behind one of three signals:
- A published track record (CVE, bug bounty, security research papers, capture-the-flag rankings)
- A graded probation period where you submit 10–20 attacks and an internal reviewer scores them
- An invitation from someone already in the queue
If you don't have (1) yet, (2) is the obvious path. The probation tier usually pays at the specialist rate ($40–60/hr) while you build a record. The promotion to expert tier usually comes after ~50 accepted submissions with novelty scores above the lab's internal threshold.