Research

Vulnerability Discovery with AI and Conventional Tools

Anthropic recently reported that its Project Glasswing partners had uncovered at least 129,000 verified software vulnerabilities. Palo Alto Networks reported 14,090 confirmed findings across 3,915 open-source projects in a two-month campaign. If model-assisted systems can find flaws at this scale, what role remains for conventional security testing tools?

A child peers through a magnifying glass at a weed growing from a crack in a stone pillar.

Found one. Now someone has to fix it.

Palo Alto’s NOVA system combines multiple models with specialised analysis tools and independent validation in clean environments. Anthropic’s total comes from partial partner reports with differing triage practices. Neither report compares those results with a similarly resourced programme of conventional security testing, so the totals cannot tell a team which existing tools it could safely retire.

To decide what to keep in a testing programme, we need to distinguish the jobs those tools do. Static application security testing, or SAST, examines code without running the target application. Dynamic application security testing, or DAST, commonly refers to testing a running web application through its external interfaces. Fuzzing repeatedly executes a target with generated or mutated inputs, looking for failures. The examples below also consider where a large language model, or LLM, could help, and how a discovery can become a lasting check.

The methods overlap, because fuzzing is also dynamic testing. Web and API fuzzing can form part of DAST, while a fuzzer aimed at a library function tests execution inside a process. The separate fuzzing section examines how input generation, coverage feedback and failure detection work together.

What the finding mix shows

Palo Alto assigned about 4% of its findings to memory and calculation issues, rising to 8% when resource-management and denial-of-service issues were included. The remaining categories included access-control failures, path traversal, code injection, server-side request forgery (SSRF) and prototype pollution. That is a broader range of failures than a crash-only campaign is designed to recognise.

Those percentages do not measure what existing tools missed. Path traversal and code injection can be investigated with static data flow queries or targeted runtime tests. Stateful fuzzing can explore workflows, and an application-specific assertion can detect an unauthorised action that causes no crash. A vulnerability’s category does not determine which technique can find it.

SAST follows the input through the code

SAST ranges from matching syntax to tracing how data moves through a program. CodeQL’s data flow analysis can follow values between functions and through object properties, while taint tracking follows attacker influence through transformations. A query can therefore investigate whether input from a request can reach a sensitive operation.

Palo Alto also describes NOVA’s discovery as largely static analysis, tracing external input towards dangerous operations. Its model-based review and CodeQL’s program analysis are different ways of investigating those relationships.

Consider a Python service that looks up customers in SQLite. The HTTP handler passes a query-string value to the following function without first constraining it to an integer. The connection is supplied by the application, and the function inserts the value directly into SQL.

def find_customer(db, customer_id):
    sql = f"SELECT id, name FROM customers WHERE id = {customer_id}"
    return db.execute(sql).fetchall()

The source is the request parameter, the sensitive operation is db.execute, and the formatted string carries attacker influence between them. A value such as 1 OR 1=1 changes the condition so that it selects every row. A suitable taint analysis can flag the path without sending that request to a live service. It still needs to establish where the argument comes from and whether an earlier check constrains it.

For SQLite, binding the parameter keeps the supplied value out of the SQL syntax.

def find_customer(db, customer_id):
    sql = "SELECT id, name FROM customers WHERE id = ?"
    return db.execute(sql, (customer_id,)).fetchall()

Python’s SQLite interface passes the value separately from the statement, so the same string no longer becomes an extra SQL condition. That addresses this injection route, while permission to read a particular customer remains a separate question.

Static analysis can inspect code that a running test never reaches. Once the failure pattern is understood, a query can search for related cases across a codebase and run again as the software changes. That is useful when several handlers share an internal database wrapper.

The limits come from the program model and the question encoded in the query. Consider an application that passes requests through an internal query builder before reaching the database. If the analyser does not understand that wrapper, it may miss the connection. Modelling every unknown function as passing through everything can create the opposite problem, producing paths that are irrelevant or impossible in practice.

DAST tests what the application actually permits

A suspicious code path may be blocked by authentication middleware or deployment settings. DAST exercises the running application to find out what can actually be reached under the conditions tested.

An active scanner such as ZAP applies known attacks to selected targets. That remains useful for finding familiar weaknesses through real requests. Its coverage still depends on reaching the right routes with the necessary accounts and application state. A scan that spends its time at the login page has little to say about a privileged workflow behind it.

Some failures require an understanding of the application’s rules. Suppose a service requires a request to be approved by someone other than its creator. A useful dynamic test creates a request and attempts approval using the same identity, then checks the stored result. In the following Python test, creator and auditor are authenticated HTTP clients connected to an isolated test instance. The fixture gives the auditor permission to read the request, and the API contract specifies a 403 response for forbidden approval.

created = creator.post("/requests", json={"amount": 100})
assert created.status_code == 201
request_id = created.json()["id"]

response = creator.post(f"/requests/{request_id}/approve")
assert response.status_code == 403

stored = auditor.get(f"/requests/{request_id}")
assert stored.status_code == 200
assert stored.json()["state"] == "pending"

The final assertion matters because a rejection response alone does not establish that nothing changed. An implementation could update the request before discovering the permission error. This test assumes synchronous processing, so an asynchronous service would also need it to wait for the operation to settle.

An integration test like this checks authorisation through the running API. Generic DAST scanners do not automatically know the application’s approval policy. As OWASP’s business logic guidance explains, making that rule explicit allows it to be tested automatically.

Fuzzing searches beyond the first example

A coverage-guided fuzzer such as libFuzzer mutates inputs and retains those that reach new code, using them as the basis for further variations. It can explore combinations that would be tedious to construct manually, provided the test setup gets those inputs to the relevant behaviour.

Consider a small binary format whose first byte declares how many payload bytes follow. This deliberately flawed C parser calculates a checksum using that length without checking whether the supplied buffer contains enough bytes. The final function is the entry point through which libFuzzer supplies each candidate input.

#include <stddef.h>
#include <stdint.h>

static uint8_t checksum(const uint8_t *data, size_t size) {
    if (size == 0) return 0;
    uint8_t sum = 0;
    for (size_t i = 1; i <= data[0]; ++i)
        sum ^= data[i];
    return sum;
}

int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
    volatile uint8_t result = checksum(data, size);
    (void)result;
    return 0;
}

The one-byte input 0x01 claims a payload byte that does not exist. Reading data[1] therefore crosses the buffer boundary. In a build instrumented with AddressSanitizer, that invalid access can produce a report even if the program would otherwise continue running. Coverage feedback guides the fuzzer’s search, while the sanitizer detects the memory error. The volatile result keeps the checksum observable so the compiler cannot simply discard the calculation as unused.

For this format, the missing condition is data[0] <= size - 1, checked after establishing that size is non-zero. A static analyser might also identify the missing bounds check. Fuzzing adds a concrete triggering input and can keep searching neighbouring lengths and payloads after the repair. The input demonstrates a memory error in this parser, while its effect in a product would still depend on how the parser is reached and used.

The small example puts the missing check directly in reach of the generated input. Complex formats may need enough structure to pass initial parsing, and stateful services may require several successful operations before a later request becomes interesting. More processing power cannot compensate for a campaign that repeatedly fails before reaching the code under investigation.

Google reported 26 vulnerabilities found using AI-generated or enhanced fuzz targets in 2024, including CVE-2024-9143 in OpenSSL. The team said that, as far as it could tell, the OpenSSL flaw would not have been discoverable with the existing human-written targets. It also reported a cJSON bug found by an AI-generated target in a function that already had a human-written harness. The model helped expose behaviour to testing, and fuzzing supplied the executions that revealed the defect.

Reaching more code is only part of the problem. The test also needs a way to recognise failure, often called an oracle. A runtime memory detector can report an out-of-bounds access, while an assertion can check that a rejected operation leaves protected state unchanged. A crash-only test has no reason to complain when an unauthorised request succeeds cleanly. Changing that requires a different check, even if the input generation stays the same.

Fuzzing also extends beyond malformed files and memory corruption. Microsoft’s RESTler generates sequences of API requests, using dependencies between operations to reach deeper service states. A resource may have to be created before it can be modified or deleted. Security questions involving the order of operations can therefore be tested systematically, provided the test knows which outcomes are forbidden.

An LLM-assisted investigation

NOVA’s reported workflow runs discovery, proof-of-concept construction and validation autonomously until final human review. A team can also use models for individual parts of an investigation. In the example below, the model’s proposed job is to identify a security assumption and help construct a test for it.

Anthropic’s property-based testing research describes an agent that inferred properties from Python code and documentation, wrote tests in Hypothesis and ran them. It found bugs in popular Python packages, showing one way a model can propose what a test should check. The study covered software bugs generally, rather than security vulnerabilities alone.

As an illustration, consider a service that checks permission when a user submits work, then places the operation on a queue. A separate worker executes it later. If the policy requires permission to remain valid at execution time, revoking the user’s role between submission and execution should prevent the protected action. A worker that relies entirely on the earlier decision may violate that rule, even though the submission endpoint authenticated the user correctly.

Some systems intentionally authorise work at submission, so the investigation has to establish which contract applies before treating delayed execution as a vulnerability.

An LLM could trace the submission handler, queue message and worker to identify where permission is checked. To test whether the worker relies on a stale decision, the following export regression keeps it paused until explicitly invoked. Access is revoked through the normal application service before the worker runs.

def test_revoked_user_cannot_export(app, worker, output_store):
    user, dataset = app.create_authorised_export_fixture()
    job = app.submit_export(user, dataset)
    app.revoke_access(user, dataset)

    worker.run(job.id)

    assert app.job_status(job.id) == "denied"
    assert not output_store.exists(job.id)

The test preserves a valid submission and changes the user’s permission before execution. It also needs a positive case in which an authorised user’s export completes, otherwise a worker that rejects every job could appear secure. The researcher must check that the setup preserves normal application behaviour and that the asserted policy is correct.

The investigation could then use static analysis to locate other consumers of the stored permission decision, dynamic testing to exercise the sequence through public interfaces, and stateful fuzzing to explore related orderings and retries.

The model can also be wrong about any of those steps. It may overlook a downstream permission check, assume that an attacker controls an internal field, or write a test that bypasses the normal application setup. A plausible explanation needs to survive examination of its prerequisites. Agreement from another model may help challenge the argument, but does not establish that those prerequisites hold.

In the property-based testing study, 56% of 50 manually selected reports from an initial set of 984 were judged valid bugs, and 32% were both valid and worth reporting. After the researchers introduced ranking, their top-scoring reports had validity and reportability rates of 86% and 81%. Generating reports was only the start of the work needed to select useful findings.

Keep the result after the research ends

A useful finding identifies a security failure under conditions the maintainer can understand and verify. That may come from a reproducible test or sufficiently strong code and configuration evidence. A crash in an artificial setup does not by itself establish a reachable attack, just as a static warning does not establish that the suspected path is feasible.

The fix should leave behind more than a report. A new regression can protect the business rule, a static query can search for the same mistake elsewhere, and a retained fuzz input can keep exercising the failure. These are ways to turn an expensive investigation into checks that the development process can continue to run.

Anthropic has acknowledged that verifying, prioritising and fixing findings remains difficult, despite the number of vulnerabilities uncovered. Increasing the supply of candidates is useful only if teams can establish which matter and carry the repairs through.

A security team has to judge the combination against its own software. A previously missed failure that can be reproduced, repaired and checked again is a better reason to change the testing programme than the size of another organisation’s discovery total.

Subscribe by email