Limitations of AI-Only Vulnerability Assessment Without Human Review

Automated vulnerability findings need human judgment to establish real business risk.

· 8 min read
Cover illustration for “Limitations of AI-Only Vulnerability Assessment Without Human Review”
AI Offensive Tools · November 7, 2026 · 8 min read · 1,858 words

AI-only vulnerability assessment can quickly find weaknesses that are already known. It cannot reliably decide what those weaknesses let an attacker do to a particular business. That distinction is the subject of this article and its central conclusion. Automated findings are useful inputs, but without human investigation and validation, they are not enough to support the security assurance companies expect from a penetration test.

What AI-only vulnerability assessment actually does — and what it does not

“AI security testing” covers different kinds of tools. AI-enhanced scanners use machine learning to reduce false positives and prioritize findings. Agentic platforms use LLM-powered agents to work out how an application behaves and try multi-step sequences. AI-assisted workflows put those capabilities in the hands of a human tester. Treating these as interchangeable makes both the promises and the criticism less precise.

Tools can correlate DNS records, certificate transparency logs, and cloud metadata to map exposed systems. They can detect known SQL injection variants, cross-site scripting patterns, missing headers, and known vulnerabilities. They can also navigate complex authentication flows faster and provide coverage between human-led assessments. These are useful capabilities.

The distinction is between identifying a weakness and establishing its consequences. A known pattern gives a tool something to search for, but an attack path depends on how weaknesses interact within a particular environment. Agentic tools can attempt more than simple signature matching, yet attempting a sequence is not the same as reliably proving business risk. Bridging that gap requires judgment.

Where the boundary actually falls: the specific tasks AI cannot perform reliably

Exploit chaining exposes the gap most clearly. A tester may combine unrelated misconfigurations to move from a standard user account to administrator access. Another path might let a Lambda function that has been compromised reach every S3 bucket. Testers must find each weakness, recognize how they connect, and validate the resulting access. AI-only tools do not reliably complete that work without a tester interpreting the findings.

A payment flow that allows fraudulent refunds may fail because of how the process is designed. An authorization chain can also allow privilege escalation even when each of its parts looks ordinary. Testing these paths requires understanding what the application is supposed to allow. Signatures alone cannot supply that understanding.

MTI’s 2026 analysis shows how severity can be misjudged, using an internal admin portal that has no MFA. An AI platform immediately rates it high-severity, while a tester investigates whether access is VPN-only, what network segmentation exists, and which compensating controls apply. Those facts can reverse the conclusion.

FRSecure’s evaluation found a related failure in the opposite direction. AI tools labeled some findings low-severity when they should have been medium-severity. The weakness appeared in the output, but its importance was concealed. The evaluation also found that tools struggled to deepen their investigation or pivot after basic reconnaissance, leaving the initial assessment largely unchallenged.

Why LLMs adding "business risk intelligence" to findings does not close the gap

FRSecure found that some tools used LLMs to add business risk intelligence to findings, even though that intelligence depends on stakeholder conversations and an understanding of actual operations. Scan output does not show which systems the business relies on for revenue or which data flows are business-critical.

MTI explains the mechanism behind this limitation. LLMs generate responses through statistical prediction without independently understanding the organization’s priorities, revenue dependencies, or regulatory exposure. A model can describe why a vulnerability might be dangerous without knowing whether that description fits this business. A confident account can therefore exaggerate a low-impact edge case or understate a serious exposure.

Accountability adds another limit to autonomous judgment. Research on autonomous offensive-security agents describes non-deterministic policies with outputs that are hard to explain both before and after the agents are deployed. That complicates the task of examining how an agent reached a decision. A skilled tester instead connects a severity judgment to stakeholder conversations, data handling, and validated behavior. That reasoning can be questioned, corrected, and assigned to someone professionally responsible for it.

What the empirical comparison between AI tools and human testers shows

FRSecure’s evaluation, published October 2026, found a large gap between the AI products tested and the human-led assessments of the same environments. Human testers produced far more findings and uncovered more severe weaknesses. A low-severity DNS setting was the only issue that the AI products alone found. Their contribution did not approach the human testers’ results.

Mature commercial vulnerability scanners also regularly outperformed the AI tools. Those scanners were far cheaper than the AI products evaluated, making the result especially relevant to purchasing decisions. FRSecure also found that many products labeled as reasoning systems relied on high-level orchestration of conditional logic. An “AI penetration testing” label therefore did not establish either depth or value.

These results should not become a claim that every AI agent always loses to every human. Justin W. Lin and colleagues’ study, “Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing,” found that ARTEMIS outperformed most human participants in a live enterprise environment. Existing agents generally performed worse, while ARTEMIS showed technical sophistication comparable to the strongest participants.

For buyers, that distinction limits what any product label can establish. FRSecure found useful supporting roles in repetitive work, note organization, test suggestions, and scan summaries. Those benefits and a strong result from a different agent do not prove that an assessment a company buys gives it complete security assurance. That depends on what was investigated and validated in the actual engagement.

Why this matters specifically for companies using AI-only results for compliance and security assurance

ISO/IEC 27001:2022 addresses technical vulnerability management in Annex A 8.8 and security testing in development and acceptance in Annex A 8.29. ISO/IEC 27002:2022 names penetration testing in its implementation guidance. Fully autonomous output without human validation is normally treated as vulnerability scanning, so presenting it as penetration testing creates an audit gap.

SOC 2 does not make every evaluation identical. AICPA Trust Services Criteria CC4.1 names penetration testing as an example of an evaluation management may use. Where a penetration test is expected, auditors look for a recent test within the Type 2 observation window, and enterprise customers routinely request the report. Automated output without human sign-off is unlikely to meet those expectations.

HIPAA also distinguishes existing obligations from proposed requirements. The Security Rule requires periodic technical and nontechnical evaluation under §164.308(a)(8). Proposed rulemaking published January 6, 2025 would make penetration testing a requirement at least every 12 months. Its comment period closed March 7, 2025, with final action targeted for July 2027. That proposal points toward more explicit testing requirements, but it is not an enacted annual-testing obligation.

The security risk remains no matter which framework applies. When an assessment cannot reliably find exploit chains or business-logic flaws, a report with few high-severity findings does not prove that there are few high-severity vulnerabilities. The report may describe the tool’s detection limits. Checking a compliance box does not resolve that uncertainty.

What genuine human judgment adds at each stage of a penetration test

Human involvement changes the investigation from the beginning. MTI maps the lifecycle across planning, reconnaissance, enumeration, exploitation, validation, and reporting, with AI assisting the human tester. During planning, a tester asks which compromised systems would cause the greatest harm. The scope depends on revenue dependencies, data flows, and regulatory exposure. A map of reachable endpoints cannot answer those questions.

Reconnaissance and enumeration then provide evidence for further investigation. Automation can gather that evidence quickly, while a tester decides which observations to look into more closely. Unexpected behavior can redirect the engagement toward an authorization chain or cross-tenant access flaw. The investigation need not stay on its original path. This ability to revise a hypothesis is central to adversarial testing.

Exploitation and validation turn suspicion into evidence. A working exploit shows that a weakness can produce the claimed result. Yet a tester must also judge when exploitation could crash production systems and should not proceed. A credible assessment distinguishes proven findings from flagged issues that remain unvalidated.

Reporting connects the technical evidence to decisions. A tester can explain why a finding threatens this business, which controls reduce its impact, and what remediation deserves priority. Engineering teams and boards need different explanations of the same evidence. Human judgment makes those explanations specific to operational consequences rather than leaving a generic severity score to speak for itself.

How to tell whether the penetration test you're buying includes real human judgment

For a buyer, the practical step is to examine how the work is done before signing. Ask how findings are verified and who reviews and signs off on the report. Proven exploitation is stronger evidence than labels such as “possible” or “potential,” with production-safety limits made explicit. Ask about hands-on credentials as well. OSCP requires demonstrated exploitation skill, while CISSP and CEH emphasize management-level and knowledge-based competence respectively.

Scope reveals what the vendor intends to investigate. Pricing driven only by IP or page count suggests automated scanning rather than a manual assessment shaped by attack surface, business criticality, and data flows. A sample report provides another useful check. Strong reporting documents a path from a specific entry point through particular findings to a business-impacting objective. Generic scanner output with a company name added does not show that this work was done.

Communication and follow-through deserve equal scrutiny. Critical vulnerabilities should be surfaced when discovered. Ask whether retesting after remediation is included or sold as a separate engagement. A polished report alone does not show that the remediation cycle is complete.

A whitebox engagement adds a further level of access. Source code, cloud configurations, and documentation let testers look into paths that external scanning cannot see. The additional evidence available from the system’s internal design can make business-logic flaws and configuration errors findable. The depth promised should match the access and manual investigation provided.

Where continuous automated scanning fits in a security program that already has human-led testing

Continuous scanning solves a different problem from a periodic penetration test. AI-enhanced tools can check deployments for known vulnerability patterns within hours, flag regressions, and find systems that have just been exposed. That gives teams coverage between deeper assessments. It also reduces the repetitive enumeration human testers must perform, leaving more engagement time for investigation that requires judgment.

Pull-request scanning moves some of that detection earlier. It can catch newly introduced vulnerabilities when code changes, before those changes ship, rather than waiting for a quarterly scan or annual test. This is a useful part of CI/CD security because it follows the development cycle. It does not establish that a payment workflow is safe from abuse or that tenant boundaries hold across a complex authorization chain. Those questions still need deeper testing.

FRSecure recommends continuous AI testing as a stopgap between traditional penetration tests and as assistance for skilled analysts. In compliance work, that continuous record supports the evidence of ongoing security activity without substituting for a human-reviewed assessment where one is expected. The roles are distinct and complementary. Automated tools watch for known patterns as systems change, while human-led tests investigate what an attacker can actually accomplish and what that outcome means for the business.

Sources

  1. The Ethics of Autonomous AI Agents for Offensive Security
  2. Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing

More in AI Offensive Tools