The terms “audits,” “assessments” and “testing” (AAT) have both legal and common-usage distinctions and are now essential tasks in cybersecurity. These terms fit into multiple compliance, regulatory and security hygiene
usage scenarios and have become required functions.
This piece details the key definitions to consider regarding these terms, and helps you to understand their contextual meaning, use them properly and get the organization standardized on their legal interpretations.
Audits, Assessments and Testing Defined
AAT terms are often used very imprecisely between various industries and organizations. The definitions below cover the key distinctions between these terms but be sure to also check with your GRC and legal teams to understand key distinctions for
your organization and get authoritative answers and legal advice for use in your organization’s context. The 3 terms are defined in detail below:
What is an Audit?
An audit is usually understood in very precise terms:
- It is performed by an “auditor,” who by virtue of training, certification (as a Certified Public Accountant, for example) or accreditation (e.g., as an ISO 27001 certifying auditor) is authorized to make authoritative statements in accordance
with whatever standard is being audited against.
- It is performed against a set of well-defined criteria, requirements, standards or controls set by law, regulation or contracts.
- Third parties can rely on its accuracy. This is especially true, for example, of financial audits (which have large information security elements) under the Sarbanes-Oxley Act (SOX) and Service Organization Control (SOC) audits, such as SOC Type 2 reports
relevant to cloud service providers.
The terms “audit” and “auditor” should be reserved for use only in those situations where they are appropriate. Most companies have an internal audit department that employs auditors who perform their duties within the company,
often to help ensure the company is ready to receive an audit from an external entity (SOX, SOC 2, etc.).
READ: Why SOC Audits Are Key Drivers of Business Competitiveness
What is an Assessment?
“Assessment” is a broader term than “audit.” It usually has a descriptive adjective ahead of it – e.g., risk assessment, maturity assessment, compliance assessment – that explains what it is intended to accomplish.
In each case, performing an assessment requires setting the context for the evaluation itself.
For example, in performing a risk assessment, you first need to establish the risk framework, such as measuring software or hardware against NIST standards or Center for Internet Security (CIS) Top 20 controls.
A compliance assessment, as the term implies, measures compliance against a set of:
- Contractual requirements, such as those prescribed by the Payment Card Industry Data Security Standard (PCI DSS).
- Regulatory requirements, such as those prescribed by the Health Insurance Portability and Accountability Act (HIPAA).
- Internal requirements, such as those contained within the company’s information security policies and standards.
Usually, a compliance assessment has no legal bearing. It is advisory in nature, telling the subject of the assessment how well it meets the relevant requirements. This is very different from an audit, which is performed by qualified auditors and (usually)
has legal bearing.
For example, a company may internally assess its compliance with PCI DSS, but that is only useful for deciding whether the company is ready for a formal audit by a qualified security assessor (QSA). The “A” in QSA stands for “assessor,”
but it is just another term for “auditor.” PCI gives QSAs the authority to determine whether the company meets PCI DSS, usually documented in a Report on Compliance (ROC).
A maturity assessment measures the level of maturity of a security program using the Software Engineering Institute’s (SEI’s) Capability Maturity Model for Software or using the ISO/IEC 21827:2008 System Security Engineering - Capability Maturity
Model (SSE-CMM).
What is Testing?
“Testing” is the broadest term of the three, with some of its elements performed as part of audits and assessments. To illustrate its breadth, some items that are tested include:
- Software code. Testing here involves static testing (simply reviewing code written by someone else) or dynamic testing (evaluating code while it executes). The testing may be focused on establishing the code meets business, security or functional requirements,
or that it produces results consistent with empirical data. For example, in the context of Food and Drug Administration (FDA) regulations governing code running on medical devices like blood analyzers and infusion pumps, “testing” is a
term used as part of Computer System Validation (CSV), where it is important to confirm the code operates in a predictable fashion – i.e., has no surprises in its execution.
- Hardware. Here, the focus is on how hardware behaves and whether it meets relevant business, security and functional requirements. This is especially important with programmable logic controllers (PLCs) and supervisory control and data acquisition (SCADA)
devices.
- Controls. Usually an audit “tests” whether controls are working properly, are appropriately designed and cover the scope of what the audit is supposed to address. For example:
- A SOC 2 audit is performed against a set of controls (called Trust Service Principles) established by the Committee of Sponsoring Organizations (COSO).
- A financial audit is performed to the Statement on Standards for Attestation Engagements (SSAE) Number 18 rules and controls set by the American Institute of Certified Public Accountants (AICPA).
- A SOX audit is required of publicly traded companies annually by SOX and it is performed to rules and controls set by the Public Company Accounting Oversight Board (PCAOB). Other than being exceptionally stringent, a SOX audit tests assertions made by
the CEO and CFO of the company about the financial condition of the company, and if those assertions are determined to be false, SOX permits those parties to be prosecuted and imprisoned if found guilty of falsehoods or negligence.
In each of these cases, “testing” of the controls is performed and, from an information security perspective, involves elements like user account management, privilege management, role definitions, separation-of-duties considerations, and
so on, involving financial management and reporting systems. Testing usually involves actions like taking the list of users who have accounts on the system and verifying a statistically significant sample of those users still deserve the privileges
they have, are still employed, have not changed jobs and are not violating separation-of-duties rules.
- System or network architecture. How the network or a system is architected can be tested against good business practices as set forth by frameworks like COBIT or ITIL. Testing in this case involves taking the existing architecture and measuring it against
what those frameworks require/encourage as good business practice. One could also call this an “assessment” that includes “testing” as one of its elements, which illustrates the fluid nature of the boundary between these terms.
Best Practices for Audits, Assessments and Testing Terminology Use
To ensure everyone understands these terms the same context:
- Be careful, consistent and specific. People often speak of an “audit” when they really mean an “assessment.” This can lead to program and legal confusion as to what the review (audit or assessment) is really intended to accomplish.
- Ensure requirements are clear. The standard or requirements against which an audit, assessment or testing is done should be clearly stated and understood by the recipient of the results. An audit against SOX requirements has very different substance and
meaning from an audit against PCI DSS requirements.
- Use the right people for the task. Ensure whoever is doing the audit, assessment or testing is trained and capable of doing the job and, in the case of an audit, is appropriately certified or accredited. If they aren’t, the results may be inaccurate
or unusable with other parties (i.e., the parties relying on the results may not accept them because the person doing the work is not suitably certified or accredited). By analogy, you need a lawyer to do legal work; if the person rendering a legal
opinion is not a lawyer, the opinion is unlikely to be trustworthy and certainly will carry no weight in court or with regulators.
- Focus on security issues. Finally, determine exactly what audits are required by law, regulations or contracts, and ensure the information security elements relevant to them are properly performed so the audits will not fail owing to security issues.
Preparing for such audits likely involves assessments and testing to ensure everything is completed appropriately.
Although reasonable efforts will be made to ensure the completeness and accuracy of the information contained in our blog posts, no liability can be accepted by IANS or our Faculty members for the results of any actions taken by individuals or firms in
connection with such information, opinions, or advice.