Are we secure? The question the annual pen test simply can’t answer

Written by Gregg Meade

  • News

Attackers now use AI to move at machine speed. Organisations who test once a year are bringing a photograph to a film shoot. 

In cyber security, we are all trying to answer a single question: are we secure? 

Anyone who is responsible for the continuity and protection of an organisation’s digital estate will recognise the uncertainty. It also explains why the ‘traditional’ annual penetration test has endured for so long. It is the closest most organisations get to an honest answer. 

The ritual of the annual penetration test. 

We are all familiar with the cycle: you spend weeks preparing your environment and agreeing scope. A talented expert spends a period of time assessing a deep yet narrow aspect of your estate. A report follows, and your team then works out what its findings mean for the parts of the estate nobody could test, and which risks your organisation should care about. 

By the time that work is done, you have usually been through two or three patch cycles. The system has moved on, the report has started to blur, and the next snapshot is twelve months away. 

Of course, none of this is a criticism of pen testers or traditional static pen testing – the traditional test is still excellent and still something you surely want. The problem is the model. It is narrow by design and infrequent because of the cost, and it can’t give you a running answer to that all-important question “are we secure?” 

A photograph and a video.

“A penetration test is a photograph, and your attack surface is a video.” – Robbie Petrie, Red Team Lead at KHIPU Networks. 

Between two annual tests, an estate changes constantly. New servers spin up, and new infrastructure and endpoints arrive. Staff join, leave and move roles. A third party gets a new integration, and someone amends a firewall rule to fix an urgent problem. None of it appears in the report you were handed months ago. 

Many organisations run their test for audit purposes, and that is a legitimate reason. A compliance certificate tells an auditor that you were tested on a given date, yet it can’t tell you whether the change made last Tuesday opened a door. 

Attackers have stopped waiting. 

The other half of the problem sits on the other side of the virtual fence. Hostile actors are already using AI to increase the speed and scale of their attacks, and to lower the barrier to entry. Techniques that once needed skilled operators are becoming faster and more accessible. 

In a recent anecdote, Dan Bird, Field CTO at Horizon3, explained how the NodeZero platform compromised a US defence industrial base supplier in about 76 seconds. If you can’t spot and stop an AI-driven attacker in that window, he argued, you’re already too late. His view is that the future of cyber conflict is “AI on AI, with humans by exception”, because machines are simply faster! 

So, with all of this in mind, what are the consequence for cyber estate defenders? An attacker who takes one pass at your environment and then leaves it alone for a year is a thing of the past. If threat actors can test your estate continuously, an annual defensive test leaves a wide gap. 

What “autonomous penetration testing” actually changes. 

Autonomous penetration testing is a change of tempo and reach, and it complements the human expert. Contrasting the two directly: the traditional test is deep and narrow, while autonomous offensive testing brings speed, scale and thoroughness. You can test the whole environment, or one part of it at high resolution, weekly, daily or quarterly, depending on how much certainty you want. 

It also works differently from a scanner in one very important respect. It attempts the exploitation itself. Ultimately, the only way to know whether a system is exploitable is to try to exploit it. 

The model behind the platform is simple: hack, fix, verify. 

  • Hack. Autonomously attack your environment as an adversary would, and find what is genuinely exploitable. 
  • Fix. Prioritise what matters and get a specific remediation for each proven weakness. 
  • Verify. Re-test to confirm the fix worked and record the result. 

Repeat that loop and something valuable accumulates. You build a series of data points showing your risk falling over time, which is a far better board-level story than a single annual PDF. 

The human layer still matters. 

Adopting offensive security at scale changes how an organisation operates. When you remove the bottleneck in finding problems, the choke point moves elsewhere, usually to remediation and change management. Of course, a tool is only as good as the human on the end of it. 

That is why KHIPU runs autonomous penetration testing as a managed service. Our Red Team interprets the results, and hands back prioritised, plain-English direction on what to do next. The service runs to your organisation’s cadence: one-off, always-on, external weekly, internal fortnightly, authenticated, unauthenticated, or both for comparison. 

Where to start… 

If your last penetration test was a year ago, the honest question is: what has changed since? KHIPU offers two low-friction ways to find out: 

  • External asset discovery. KHIPU maps your internet-facing attack surface using only your IP addresses, with no tooling to deploy. Expect to find open ports and domains you didn’t know were in scope. 
  • Internal segmentation scan. A small virtual machine scans your internal network to reveal open hosts and ports and test how well it is really segmented. 

Book your next step today!