Risk Assessment, Vulnerability Assessment and Pen Testing

Why are they so confusing, and what you need to know!

Almost weekly, a customer contacts me asking for a penetration test. However, what they really want is a vulnerability scan. Or, they ask for a vulnerability scan, but what they need is a Risk Assessment.

I don't know how the mix-up started, but I know it's not the customer's fault. More often than not, when I ask a customer for a copy of their Risk Assessment, what I get are Vulnerability Scan results. Who is calling a Vulnerability Scan a Risk Assessment? Most of the time, it is an IT (Information Technology) or IS (Information Security) vendor that should know better.

Now, unless someone is a CISM (Certified Information Security Manager)*, I can't be confident that the term Risk Assessment will be accurate.

There is some overlap between all three types of assessments. Risk assessments must include the outcome summaries of any vulnerably assessments or Pen Tests. Pen Tests are a highly specific subset of vulnerability scans that both emulate and exploit forms of attack against an organization.

In IS, clear and accurate communication is essential. Let’s do a quick overview of what the basic differences are between these terms.

Risk Assessment Overview

• Not completed with scanning software, and the output is not a list of vulnerabilities. 
  Conducted by IS professionals, not IT professionals.
• Top management, as well as IT management, use the output.
• IT-centric, but also has non-IT elements (e.g., physical security, polices, third party risk, and even some HR topics such as risk from disgruntled employees sharing intellectual property or IT configurations.)
• Centers on critical business practices at risk.
• Outputs a list of prioritized actions based on the organization's “risk appetite.”

Vulnerability Assessments

• Uses scanning software like Nessus or OSINT (Open-Source Intelligence) tools.
• Performed by either IT professionals or IS professionals.
• Primarily used by IT, with summarized results reported to management.
• IT-centric and consists primarily of network-based scans of internal and external networks.
• Centers around misconfigured equipment such as firewalls, operating systems, and IoT (Internet of Things) devices.
• Actionable items include patching operating systems, updating firewall policy, network reconfiguration, and pattern or behavior-based security software upgrades.

Penetration Tests (Pen Testing)

• Use a wide variety of highly specialized tools or procedures depending on the scope of the engagement.
• Done by a subset of IS professionals with specialized training in thwarting and bypassing security systems, countermeasures, and poorly trained employees.
• Results reported to IT management and may be reported to outside agencies if they require industry or government testing. Reporting might only go to upper management if part of a "black box" test. 
• Centered on suspected vulnerabilities within a predefined scope. May include non-IT elements such as physical security, third-party risk, and human-factor vulnerabilities. 
• Actionable items specific to areas where there was successful penetration of technology controls or employee policies. Like Vulnerability Assessments, they might include patching operating systems, updating firewall policies, network reconfiguration, or, like a Risk Assessment, might involve policies or employee training.

Commonalities between Risk Assessments and Vulnerability Assessments

• Can be best done by or in cooperation with independent 3rd party security firms. 
• Typically done in the open with full cooperation from IT and people in other organizational roles or departments.

Commonalities between Risk Assessments and Pen Tests 

• Require specially trained and experienced professionals and go beyond the use of scanning tools. 
• Nearly always done by independent 3rd party firms.
• Likely to include substantial human-factor topics and scope.

Other fun facts

• Pen Testing gets all the Hollywood cred from hacker activities in shows and movies, so this tends to be what the average person is first exposed to.
• Risk Assessments and Vulnerability Assessments are both critically important to organizations of all sizes, all you have to do is read the news headlines to see how the issues have escalated.
• With little training beyond typical IT knowledge and experience, anyone can use an inexpensive off-the-shelf tool for a Vulnerability Assessment. However, correctly interpreting the results and ensuring there are no gaps does require IS experience.
• An independently conducted, or at least independently managed, Vulnerability Assessment is more likely to find gaps unknown to the IT department

Risk Assessments: Understanding the Most Misunderstood Assessment

Why is risk important?

Risk management is the process of ensuring that the impact of threats exploiting vulnerabilities is within acceptable limits at an acceptable cost and aligned with organizational goals. It is the fundamental function of IS and provides rationale and justification for virtually all IS activities.

The most critical prerequisite to a successful Risk Assessment is understanding the organization. If your organization has an ISO level BCP (Business Continuity Plan), you should already have a Risk Assessment. However, more than 50 percent of the time, when an organization says they have a BCP, it actually doesn't. They likely have a Disaster Recovery Plan, Incident Recovery Plan, or maybe some cheap template downloaded from a non-reputable source.

A formal Risk Assessment (typically based on NIST 800-53/ISO 27001) will be geared toward large organizations and include ALE (Annualized Loss Expectancy), ranked risk levels and factors based on asset values.

Cadre Information Security has an Informal Risk Assessment that is based on NIST. Ours is in plain language and does not include ALE. But it does give a SMB (Small Medium Business) the major actionable benefits of a formal Risk Assessment for a fraction of the cost and time.

Output from a Risk Assessment: Stop guessing about what security products you need and how much to spend.

Every IS control used by the organization, from firewalls to anti-malware software, should have its existence, scope, and cost justified by risk. Fit for Use and Fit for Purpose Control Evaluation ensures you can tie controls back to the risks they mitigate, eliminating any guessing about adequacy, rationale, or appropriate costs and responsibilities for those controls.

A good Risk Assessment offers clear guidance to enhance organizational security. In addition to formal outputs, Risk Assessments provide overall rationale for the following:

• IS policy
• IS controls
• Security Awareness program design
• Security alignment with business processes
• Justification for cybersecurity investments with data-driven findings
• Actionable insights for the Board, C-suite, HR, IT, and legal counsel
• Organization-wide buy-in and reduction of security silos.

Security is part of everyone’s role, not just IT; evidence of Due Diligence and Due Care

Boards of Directors, senior executives, and oversight frameworks often need evidence of Due Diligence and or Due Care to meet policy or regulatory requirements, avoid possible litigation issues, or meet responsibilities codified in the organization by laws.

What you don’t know CAN hurt you!

Gap analyses, comparing risks against controls and other mitigations gives you the map needed to find gaps in not only physical controls, but also policies and training.

Executive Briefings, templates for reports, and clear management of core business practices.

Executives need to know the risks and mitigations for which they are responsible. A risk-to-mitigation briefing is one of the most efficient ways to document and communicate this information.

Cybersecurity insurance, 3rd party, and security self-reporting.

Without a Risk Assessment, you might fail the requirements of a security questionnaire from your insurance company, a supplier, or a partner.

Red Flags

Have you been sold a Risk Assessment that is not a Risk Assessment?

Here are some red flags that might indicate you don’t have an actual Risk Assessment:

1. No list of risks with the ability to assign a reaction (Mitigate, Avoid, Transfer, Accept) Risk Assessments to be complete must allow you to decide what you are going to do about each risk. This should be in plain language so that the management of the organization can make clear and accurate decisions about each risk. These decisions are made by all leadership, not just IT. 
2. You have a list of operating systems missing patches. A Risk Assessment might state that you have out of date and unpatched operating systems, but it would not be in granular detail, except for an additional document such as an appendix. Remember that a Risk Assessment needs to be able to be read by non-technical decision makers.
3. No risks outside of IT equipment. A Risk Assessment has a scope that covers the entire organization. It will include polices, human factor issues, and relationships with other organizations, such as 3rd party interactions. 
4. The assessment was done only with software tools. A Risk Assessment must include an understanding of polices, workflows, training, risk appetite, business processes, and other factors that require interviewing key stakeholders in your organization.

 Summary

Risk Assessment, Vulnerability Assessment, and Pen Testing are all key elements of any good business practice, and they become more important every day. Understanding these terms and the qualities, benefits, and ramifications of each of these assessments is essential for good policy and organizational security management.

I hope this article has been enlightening and will prove useful!

To learn more about Informal Risk Assessments, here is a concise guide. Click to learn more about Cadre Assessments.

Please feel free to contact me at Tim.OConnor@Cadre.net, read my past and future articles at or follow me on LinkedIn.

Key Terms:

Business Continuity Plan: An ISO framework plan to help an organization survive a major incidence, such as a cyberattack and not go out of business or suffer undue damage to business practices or reputation. This will include a DRP (Disaster Recovery Plan), BIA (Business Impact Analysis), and Risk Assessment, but it is not a DRP, BIA, or Risk Assessment.

Business Impact Analysis: Measures the impact from the loss of business processes over time.

Risk: The likelihood of a threat exploiting a vulnerability that could impact a business function.

Risk Assessment: Finds risks to critical business processes and measures impact and likelihood.

Security Awareness: Understanding the potential for people to deliberately or accidentally steal, damage, or misuse data.

Vulnerability: Weakness or flaw, MUST be exploitable.

* Not to be confused with CISSP, Certified Information Systems Security Professional.

AI did not generate this article.

In fact, this article was drafted on a grizzly old 1982 Apple LISA computer, which introduced the most profound advancement in personal computer history.

No part of this material may be used or reproduced in any manner for the purpose of training artificial intelligence.