BugBase automates black-box pentesting. Read the labs

Automated black-box pentesting is having a moment, and the pitch is genuinely good: your app gets probed continuously instead of once a year by a firm that bills in weeks. BugBase shipped its version this week. The product is interesting. The benchmark it published is the part that needs a careful read.

What actually happened

BugBase announced Pentest Copilot Enterprise on September 2 — an autonomous red-teaming platform that tests systems from the outside, with no source-code access, while holding authenticated context so it can reach past the login page. Specialized agents run in parallel across changing states, identities, and workflows, executing iterative attacks across roughly 100 vulnerability types including authentication, authorization, injection, and business-logic flaws. It drives real Chromium browsers rather than raw HTTP. It is publicly available now, and Help Net Security listed it among the week's infosec launches.

The stated problem is real: regulated companies often cannot hand source code to an outside pentest firm, and manual black-box assessments run for months and happen quarterly at best.

Now the numbers. BugBase reports 100% coverage of defined scope on OWASP Juice Shop and Broken Crystals, plus completion of the GOAD, NHA, and DRACARYS Active Directory labs. Those are vendor-published results on intentionally vulnerable training targets — applications built to be solved, with known answers. That is a legitimate smoke test and it is not evidence about your codebase. We are reporting the claim; we have not seen independent validation of it.

Why automated pentesting matters for your business

If you run a commerce site, the honest baseline is that you have never had a real pentest. Continuous automated probing at a software price beats that comfortably, and it is worth a look.

Two conditions before you buy. First, scope it as a floor, not a ceiling: these tools are strong on the enumerable classes — injection, auth bypass, exposed endpoints, misconfiguration — and weak on the business-logic flaws that actually cost money, like a discount code that stacks or a refund path that skips inventory. A human still finds those. Second, run any vendor's tool against your own staging environment before signing, and count what it finds against issues you already know about. That test costs you an afternoon and tells you more than any lab benchmark.

And whatever you buy, assume the same class of tooling is already pointed at you. Agentic probing got cheap for attackers at the same moment it got cheap for defenders.

Sources: BugBase press release, Help Net Security.