If you have looked for a free way to check your Microsoft 365 security configuration, you have almost certainly found CISA's ScubaGear. It is a genuinely good tool, and if you are doing point-in-time hardening you should use it. This article explains what ScubaGear does well, where it stops, and where a continuously-monitored SaaS platform like TenantVigil picks up.
Competitor capabilities described here were verified against CISA's public repository and documentation as of July 2026. ScubaGear moves quickly — always check the current release.
What ScubaGear is (and what it does well)
ScubaGear is an open-source assessment tool published by the U.S. Cybersecurity and Infrastructure Security Agency (CISA) as part of its Secure Cloud Business Applications (SCuBA) project. It is distributed for free under the Creative Commons Zero license, and it has been downloaded over 160,000 times on the PowerShell Gallery (as of July 2026).
It is legitimately strong in several ways:
- Broad service coverage. ScubaGear assesses seven M365 services: Entra ID, the Microsoft Defender security suite, Exchange Online, SharePoint, Teams, Power Platform, and Power BI. That Power Platform and Power BI coverage is real, and it is wider than many commercial tools — including TenantVigil, which today focuses on the six core services below.
- A rigorous policy engine. It works by running PowerShell against the M365 APIs, then evaluating the results with Open Policy Agent (OPA) against versioned Rego policies. This is a transparent, auditable approach.
- Authoritative baselines. The SCuBA baselines are government-authored and mapped to NIST SP 800-53 and MITRE ATT&CK.
- Portable output. It produces HTML, JSON, and CSV reports you can archive or feed into other systems.
If your goal is "run a thorough, free, one-time check of my tenant against a respected baseline," ScubaGear is an excellent choice, full stop.
Where ScubaGear stops
ScubaGear itself is a point-in-time assessment tool — it evaluates your tenant's current configuration rather than watching it continuously. (For automated, repeated runs, CISA maintains a separate project, ScubaConnect, that orchestrates ScubaGear in the cloud.) That boundary creates a few practical gaps for a small business or MSP:
- It runs when you run it. Security configuration drifts constantly — an MFA exclusion added at 2 a.m., a SharePoint sharing policy loosened "just for a project," a Conditional Access policy disabled and never re-enabled. A point-in-time scan from three weeks ago cannot tell you any of that happened. You would need to remember to re-run it, and then diff the results yourself.
- It requires PowerShell, modules, and setup. ScubaGear runs on a machine you configure, with the right PowerShell modules and permissions. That is fine for an engineer; it is a real barrier for the IT generalist at a 40-person company who is not a security specialist.
- The output is technical. ScubaGear tells you which policy passed or failed against the baseline. It does not translate "MS.AAD.3.1 failed" into "employees can sign in without a second verification step, which is how most account takeovers start."
- No cross-framework compliance mapping or evidence packaging. The SCuBA baselines map to NIST 800-53 and MITRE ATT&CK, but ScubaGear does not produce GDPR, NIS2, HIPAA, ISO 27001, or SOC 2 views, and it does not package your findings into the timestamped, exportable evidence reports you would hand an auditor or insurer at renewal.
None of these are criticisms of ScubaGear — they are simply outside what a free, point-in-time CLI tool set out to do.
How TenantVigil is different
TenantVigil is a hosted, always-on platform rather than a script you run. The differences that matter:
| ScubaGear | TenantVigil | |
|---|---|---|
| Cost | Free (CC0) | Start free; paid continuous monitoring |
| How you run it | PowerShell + OPA on your own machine | Read-only Microsoft consent, then scans, no agents |
| Timing | Point-in-time | Scheduled scans + drift diffs |
| Findings | Pass/fail vs. baseline | Plain-English business impact, then the technical detail and available source evidence, with unread and manual checks identified |
| Services | 7 (incl. Power Platform, Power BI) | 6 core: Entra ID, Exchange, Defender, SharePoint/OneDrive, Teams, Intune |
| Checks | SCuBA baselines | 90+ checks mapped to CIS and CISA SCuBA baselines |
| Compliance | NIST 800-53, MITRE ATT&CK | 7 framework mappings: CIS Microsoft 365 Benchmark, CISA SCuBA, SOC 2, ISO 27001, HIPAA, NIS2, GDPR |
| Insurance evidence | Not built in | Timestamped, exportable evidence reports with available source evidence, with unread and manual checks identified |
| MSP reporting | Not built in | Multi-tenant, branded recurring service reports |
A few specifics worth being precise about:
- Continuous, not point-in-time. TenantVigil re-scans on a schedule and records drift when a passing control starts failing. It also ingests Microsoft service advisories and Message Center changes.
- Read-only and no infrastructure. Onboarding is a Microsoft consent for read-only Graph scopes. There is nothing to install and no PowerShell to maintain.
- License-aware. TenantVigil detects your actual M365 license tier and reports licence-dependent checks it could not read as not applicable or unknown, never omitting them, so you can see what was and was not checked.
- Built for evidence. Every finding carries the available source evidence, with unread and manual checks identified. Reports come in an executive and a technical format — branded for MSPs.
Which should you use?
In short, they are complementary. If you want a free, deep, government-authored point-in-time check — including Power Platform and Power BI — run ScubaGear. If you need that posture watched continuously, translated into business risk, mapped to the compliance frameworks your auditor and insurer care about, and delivered without maintaining any tooling, that is the gap TenantVigil fills.
The user who runs ScubaGear today and thinks "great — now how do I keep it this way, and prove it?" is exactly who TenantVigil is for.
Start free to check one Microsoft 365 tenant — no agents and no PowerShell. The Free plan requires a Microsoft work-account sign-in and administrator consent and covers one tenant.