Executive Summary

This brief provides an overview on the different value propositions of penetration testing for organisations.
Single point in time external pentesting by human experts can support compliance requirements, but provides only a limited view of overall security and resilience.
Strengthening security and resilience requires regular, automated testing at scale, combining external and internal exposure.

Our Adversarial Resilience & Testing (AR&T) services stress-test organisational,

executive, and systemic exposure against realistic adversary scenarios.

Contact us to engage AR&T when exposure intersects with consequence.

Introduction

A penetration test (pen-test) is a structured and controlled exercise that illustrates how a threat actor could exploit vulnerabilities in systems and the potential impact on the business if they did.

A vulnerability assessment can identify technical flaws or vulnerabilities. Penetration testing goes beyond this to assess whether the identified issues can lead to business disruption, reputational damage, or financial and data loss; it determines whether a threat actor could achieve its objectives, testing outcomes and consequences, rather than just vulnerabilities. By doing so, it provides clarity on exposure and identifies areas for improvement.

The motive for conducting penetration testing can originate from several sources. These can include mandatory requirements imposed by regulators (such as the new DORA regulations), contractual obligations specified by major clients, or even internal initiatives, such as an IT head seeking to demonstrate the effectiveness of existing controls.

Timing is important; penetration testing is often requested at moments of business risk, increased scrutiny, or major change. It can become especially important during corporate events such as investment or acquisition, signs of hostile interest targeting the organisation, or when disputes or regulatory pressure are increasing.

Knowing when not to use penetration testing is equally vital. Organisations with a low level of cyber security maturity will benefit more from addressing basic security issues than from conducting penetration testing. Improving upon weak security fundamentals, addressing poor asset visibility, and patching critical vulnerabilities should be the priority. This ensures that any readily apparent findings have been addressed, preventing the engagement of expensive penetration testers to merely reiterate the output of an automated tool.

Penetration testing is a useful activity for security assurance; however, the value comes from selecting the right form of testing for the right purpose and at the right time. Understanding why it is being commissioned and what question it is intended to answer, whether that be for compliance or for resilience. If the requirement is compliance, penetration testing can provide structured and defensible evidence that security controls have been assessed in a practical manner. If the requirement is strengthened security and resilience then the focus should be on automated penetration testing.

The following graphic outlines the different categories of penetration testing.

The following graphic outlines the common stages of penetration testing.

Pentesting for compliance

A penetration test is often engaged to meet specific compliance requirements; whether these relate to regulation, contractual agreements, or internal governance. Penetration testing in this use case provides an independent assessment, producing evidence-based findings on defined applications, systems, and controls.

When used as a compliance mechanism, penetration testing can provide a useful ‘point in time’ view on the effectiveness of existing security controls. This supports auditing and assurance. Value is also found in penetration testing providing a baseline for remediation of found vulnerabilities and in retesting proving control maturity. Penetration tests conducted for compliance purposes should be tightly scoped, covering only the required URLs, IP addresses, or applications and can include elements of internal threat.

Traditionally, penetration testing was conducted primarily by human penetration testers with ‘hands on keyboard’. This approach has important limitations.

Human testers will often pursue the most accessible route first, and focus their efforts on that path. If a compromise is achieved in the early stages of a test, human testers tend to move on to the next phase of the test, rather than systematically exploring other possible attack paths.

Testing is also limited to the time and day it is being carried out on, and is specific to the tester’s skills and what they can find in a defined period. It does not exhaust what an adversary may find and it does not guarantee that every weakness has been found.

This approach is still valid for compliance-based penetration testing (although as we will argue, it is less useful for testing aimed at genuine security and resilience).

Pentesting for Security and Resilience

Tightly scoped, human-led penetration testing is well suited to compliance and targeted assurance. However, it is less appropriate for continuously changing conditions, and ongoing resilience. With business environments changing rapidly, more frequent software releases, and changing attack surfaces, a quarterly or yearly test is no longer sufficient. Ongoing automated penetration testing is more effective in driving genuine improvements in security and resilience.

Recent advances in AI have made automated penetration testing more effective. Automated testing enables organisations to maintain ongoing ‘find and fix’ cycles, where automated systems continuously identify vulnerabilities and provide recommendations for mitigations. This approach is more likely to keep pace with new attacker tactics and techniques. The result is a material reduction in risk, versus the point-in-time view of traditional penetration testing.

A structured and repeatable assurance approach gives organisations clearer insight into their security posture beyond point-in-time testing. Combining adversarial testing, attack simulation, and structural exposure analysis provides a positive operational, regulatory, reputational, and valuation impact.

Continuous monitoring of attack paths and control effectiveness is more suitable for improving security and resilience. Routine checking and attack surface monitoring is becoming easier with the incorporation of AI-based testing and automation, making scaling, efficiency, and compatibility of security assessments a more straightforward task.

Recommendations

Conduct penetration testing in response to clear requirements

Organisations derive the most value from penetration testing by deploying it at the right stage of their security maturity journey.
Vulnerability scanning and basic cyber hygiene should be higher priorities than penetration testing for organisations with low security maturity.

The form of penetration testing adopted should be shaped by organisational goals

A point-in-time penetration testing programme can address compliance requirements.
Genuine improvements in security and resilience is likely to require continuous automated testing.

Penetration testing does not replace the fundamentals

Penetration testing is not always the right tool for the job; it cannot provide assurance in isolation.
Penetration testing delivers the greatest value when conducted as part of a structured assurance programme.

Contact us

Secured is a UK-based organisation that provides strategic advisory services to organisations concerned about threats to the security of research, innovation, and investment.

Our security practitioners help entities secure their intellectual property, build operational and financial resilience, and cultivate a positive organisational security culture.

We provide research on the national security implications of emerging technologies as part of our scientific and technical intelligence assessment capability.

Secured is part of Tyburn St Raphael Ltd, a boutique security consultancy.

info@tyburn-str.com

hello@secured-research.com

Compliance or resilience: an executive’s guide to penetration testing