Compliance scan: the often overlooked pillar of security

In addition to vulnerability scanning, an important - although often neglected - security activity is the conduct of various technical compliance checks (compliance scans). In most cases, these checks serve to verify compliance with some predefined framework (CIS Benchmarks, DISA STIG, Microsoft Security Baselines, etc.). In practice, this primarily means the automated comparison of the configuration settings of the endpoints with the recommendations and controls of the selected framework. The frequency of running scans varies depending on the maturity of the organization, but in general it can be stated that since the various hardening and other relevant regulations of an organization change less frequently, compliance scans are also run less frequently compared to vulnerability scans, where new vulnerabilities can appear even from day to day. Another reason for running scans less frequently is the “high noise level” of the scan results. There are many false positives, many known discrepancies, and justified remediation activities can also take much longer.

All of the most well-known vendors (Tenable, Rapid7, Qualys) have some kind of compliance scan solution, and I used Qualys’ Policy Audit module to illustrate the topic.

The vendor documentation highlights the following key capabilities of the product:

Accurate prioritization of compliance risks:

Coordinate risk management based on the following aspects:

Audit Impact – Address misconfigurations that directly impact compliance outcomes first.

Business Criticality – Focus on assets that are critical to the business.

Control Severity – Prioritize deficiencies based on risk exposure and compliance weight.

Ransomware Exposure – Reduce risks associated with exploitable configuration errors.

End-to-end compliance and remediation automation:

ITSM Integration – Automate ticket management with ServiceNow and other ITSM systems.

Smart Alerts – Instantly escalate compliance issues to the appropriate teams.

Audit Remediation Integration – Predefined and custom scripts quickly remediate deficiencies.

Audit-Ready Reports – Generate compliance-based reports on demand.

Continuous Compliance – CI/CD-based scanning ensures environments are continuously protected.

Ensure compliance visibility and successful audits:

Eliminate compliance blind spots – Continuously monitor configurations.

Audit with confidence – Validate controls using dashboards and reports.

Coverage of 90+ regulations – Pre-mapped controls for faster compliance.

Custom Reports – Provide on-demand insights for audits.

If you have a Qualys account, you can request a 30-day trial version of the Policy Audit module, which allows you to freely test the product. The trial version is identical in functionality to the full final version. The license is requested from the Qualys interface, which may be followed by a short email validation and ad hoc assistance.

Our lab environment consisted of an on-prem scanner appliance and a Windows Server 2022 virtual machine in addition to the Qualys cloud. During the scan, we used the CIS Benchmark for Microsoft Server 2022 Standalone, v2.0.0 framework.

The Center for Internet Security (CIS) is an American non-profit organization that was founded in October 2000. According to its mission statement, CIS’s task is to help people, businesses, and government agencies protect themselves against increasingly widespread cyber threats.

The CIS Benchmarks are developed jointly by members of the Consensus Community and CIS SecureSuite (the latter is a group of CIS members with access to additional tools and resources). The Consensus Community is composed of IT security experts who support the global Internet community with their knowledge and experience. CIS SecureSuite members are made up of organizations of all sizes and types, including government agencies, higher education institutions, non-profit organizations, IT auditors and consultants, security software vendors, and other organizations. CIS Benchmarks and other tools provided free of charge by CIS allow IT professionals to create reports that compare the security status of their systems against a consensus-based standard.

Before running a scan, there are a lot of configuration settings for the user. The most important step is to select the appropriate policy. This is done under the “Policies” tab of the PA module.

We can create our own framework that meets the needs of the company, or we can choose an existing framework. In the latter case, click on the “Import from Library” option. Here, browsing in a large library, we select the one that suits us (in my case, this is the already mentioned CIS Benchmark for Microsoft Server 2022 Stand-alone, v2.0.0):

In the “Assets” tab of the PA module, in the “Address Management” section, we add the necessary IP addresses:

Another important step is to configure the “Option Profile”. Here I have expanded the profile called “Compliance” with the necessary framework:

In order to conduct a successful scan, it is important to enable authentication for the scanner appliance:

If this has not been set up previously in connection with the use of the VMDR module, it is necessary to create a technical user on the target server and enter the login details under the PA “Scans – Authentication” tab:

Once we have done all this, let’s run an OnDemand scan:

After the scan has run, let’s definitely check whether the authentication was successful:

Then let’s move on to evaluating the results:

The report lists in detail which settings of the framework are appropriate and which ones require configuration changes. If we need more information, we clicked on a specific result:

Of course, similar to the VMDR module, we can also export the result in the form of a report. The presented lab clearly illustrates that compliance scans play an important role in assessing and maintaining the security status of organizations. While vulnerability scans focus on identifying currently exploitable errors, compliance checks measure the configuration compliance of systems in relation to a predefined framework based on industry recommendations.

The test performed using the Qualys Policy Audit module highlighted that a default system configuration can deviate from the CIS Benchmark recommendations in many ways. Identifying and prioritizing such deviations allows for increasing security maturity and also contributes to meeting regulatory and audit requirements.

Overall, it can be said that compliance scans are essential elements of modern information security practice, especially in automated and continuously monitored environments.