AI · · 6 min read
AI doesn't pentest on its own: the role of human oversight
AI-automated pentesting has become a marketing commodity. Why operator-driven AI finds vulnerabilities an autonomous agent rarely reaches, and how to spot the difference before you sign anything.
Brand update for the Portuguese operation.
There's a concrete difference between AI doing pentests and pentests done with AI. Whoever sells you the first as if it were the second risks charging a premium for a rebadged vulnerability scanner.
Part of the industry adopted "AI-automated pentesting" as a leap that would put many security professionals out of work. Often it's the same Nessus, the same nuclei, the same Acunetix, now with an LLM reading the output, probing without context and paraphrasing the findings. The agents got prettier and output volume went up, but not always real impact. If that's what you bought, what you got is mostly comfort dressed up as security.
The trick nobody points out in the contract
Two different products are sold under the same label, and the gap between them separates a pretty report from a real defense.
Pure execution AI. An agent runs scanners, maps ports, cross-references versions with CVEs, suggests payloads and writes the report. Useful for volume. This is what much of the market sells as AI-powered pentesting. When something more complex shows up, a human "validates" at the end. Human intelligence enters after the agent has already decided what to look at.
Operator-driven AI. The operator defines the target, the crown jewels, the credible threats and the engagement constraints. AI handles volume and velocity no human can match, but every critical decision passes through someone who understands what's happening on the other side of the screen. Public industry reports such as the Verizon DBIR consistently point to the human element and vulnerability exploitation among the leading causes of breaches: exploit chaining, broken business logic, system context. That kind of reasoning is hard for an agent alone to assemble.
AI is leverage, not a replacement
Here's what few want to admit: AI is great for pentesting. In the right hands, it finds more vulnerabilities, proves more impact and produces deeper reports than a human alone reaches. The important phrase is "in the right hands."
A good pentester operating with AI today delivers far more than yesterday. A bad pentester operating with AI keeps delivering what a bad pentester always delivered, now with more pages in the report.
AI executes reasoning over structured data. What it rarely does on its own, however many agents you add:
- Read silence on a pretext call.
- Decide the acceptable blast radius for the specific client in that specific engagement.
- Assemble the creative chain of five misconfigurations that are individually low severity and together are administrative access.
- Improvise lateral movement in an environment it has never seen, drawing on intuition built across hundreds of breaches.
- Ask "but what if?" before firing the exploit that could take down the client's ERP at 2pm on a Tuesday.
That adversarial mindset — the rarest and most valuable part of pentesting — isn't configurable in a prompt. It's the experience curve of someone who actually knows how to do it. AI multiplies that curve; it doesn't replace it.
How we prove this in practice
Our shared team built DUX, an internal offensive orchestration platform. Not to replace our pentesters, but to amplify every hour they work. The difference shows in the architecture, not the pitch:
- Mandatory business-context intake before any exploration. Crown jewels, applicable regulatory requirements and credible threat actors for the sector. Without that context, severity is a guess and prioritization is a hunch. AI relies on it to reason about what matters to that client, not a generic profile.
- Human decision gate before each exploitation. Every candidate exploit is checked: does the target version match the PoC? Do preconditions hold? Does the blast radius fit the agreed scope? AI returns APPROVE, CONDITIONAL or BLOCK; the operator presses the button.
- Severity anti-inflation rubric. Auditable formula (CIA impact × confidence), not the reporter's adjectives. Low-confidence findings are automatically downgraded a tier. Anyone reaching for adjectives is signaling that the method doesn't hold up.
- The operator drives in real time. During long engagements, operator signals shift AI focus on the fly: "focus on SOAP on 8080", "skip privesc", "found a credential manually, treat as authenticated". AI keeps executing, under continuous human command, not on autopilot.
Tool execution alone will be left behind. People with an offensive mindset stay ahead, amplified by AI. That's the principle DUX was built on.
Four questions to ask before signing a contract
If you're evaluating a vendor that sells "AI-automated pentesting," ask these and listen to the answers:
- Do you intake business context before starting exploration, or do you just run agents against my range?
- Is there a human operator approving each exploitation decision, or does the AI execute autonomously?
- How is severity calculated for findings: auditable formula or reporter's adjective?
- How many business-logic vulnerabilities (without an associated CVE) do you find on average per engagement?
Anyone without a concrete answer to all four is likely selling a scanner. Anyone answering with a formula, a human gate and a real sample of findings is selling pentest. This doesn't show up in any dashboard: only in the head of someone who understands how the real attacker thinks.
30-minute diagnostic
AI is not the new pentester. It's the leverage for people who already knew how to pentest.
If your organization is evaluating vendors that promise fully AI-automated pentesting, use the four questions above before signing anything. If you want to talk about how an engagement guided by professionals actually works — with AI as real leverage, not fantasy — book a 30-minute conversation with our team. No sales runaround. Just diagnostic.