If you have looked for a free way to check your Microsoft 365 security configuration, you have almost certainly found CISA's ScubaGear. It is a genuinely good tool, and if you are doing point-in-time hardening you should use it. This article explains, honestly, what ScubaGear does well, where it stops, and where a continuously-monitored SaaS platform like TenantVigil picks up.
Competitor capabilities described here were verified against CISA's public repository and documentation as of July 2026. ScubaGear moves quickly — always check the current release.
What ScubaGear is (and what it does well)
ScubaGear is an open-source assessment tool published by the U.S. Cybersecurity and Infrastructure Security Agency (CISA) as part of its Secure Cloud Business Applications (SCuBA) project. It is distributed for free under the Creative Commons Zero license, and it has been downloaded over 160,000 times on the PowerShell Gallery (as of July 2026).
It is legitimately strong in several ways:
- Broad service coverage. ScubaGear assesses seven M365 services: Entra ID, the Microsoft Defender security suite, Exchange Online, SharePoint, Teams, Power Platform, and Power BI. That Power Platform and Power BI coverage is real, and it is wider than many commercial tools — including TenantVigil, which today focuses on the six core services below.
- A rigorous policy engine. It works by running PowerShell against the M365 APIs, then evaluating the results with Open Policy Agent (OPA) against versioned Rego policies. This is a transparent, auditable approach.
- Authoritative baselines. The SCuBA baselines are government-authored and mapped to NIST SP 800-53 and MITRE ATT&CK.
- Portable output. It produces HTML, JSON, and CSV reports you can archive or feed into other systems.
If your goal is "run a thorough, free, one-time check of my tenant against a respected baseline," ScubaGear is an excellent choice, full stop.
Where ScubaGear stops
ScubaGear itself is a point-in-time assessment tool — it evaluates your tenant's current configuration rather than watching it continuously. (For automated, repeated runs, CISA maintains a separate project, ScubaConnect, that orchestrates ScubaGear in the cloud.) That boundary creates a few practical gaps for a small business or MSP:
- It runs when you run it. Security configuration drifts constantly — an MFA exclusion added at 2 a.m., a SharePoint sharing policy loosened "just for a project," a Conditional Access policy disabled and never re-enabled. A point-in-time scan from three weeks ago cannot tell you any of that happened. You would need to remember to re-run it, and then diff the results yourself.
- It requires PowerShell, modules, and setup. ScubaGear runs on a machine you configure, with the right PowerShell modules and permissions. That is fine for an engineer; it is a real barrier for the IT generalist at a 40-person company who is not a security specialist.
- The output is technical. ScubaGear tells you which policy passed or failed against the baseline. It does not translate "MS.AAD.3.1 failed" into "employees can sign in without a second verification step, which is how most account takeovers start."
- No cross-framework compliance mapping or evidence packaging. The SCuBA baselines map to NIST 800-53 and MITRE ATT&CK, but ScubaGear does not produce GDPR, NIS2, HIPAA, ISO 27001, or SOC 2 views, and it does not package your findings into the timestamped, exportable evidence reports you would hand an auditor or insurer at renewal.
None of these are criticisms of ScubaGear — they are simply outside what a free, point-in-time CLI tool set out to do.
How TenantVigil is different
TenantVigil is a hosted, always-on platform rather than a script you run. The differences that matter:
| ScubaGear | TenantVigil | |
|---|---|---|
| Cost | Free (CC0) | Free assessment; paid continuous monitoring |
| How you run it | PowerShell + OPA on your own machine | Read-only OAuth consent, ~2-minute scan, no agents |
| Timing | Point-in-time | Scheduled scans + drift alerts (email, Slack, Teams) |
| Findings | Pass/fail vs. baseline | Plain-English business impact, then the technical detail and Graph evidence |
| Services | 7 (incl. Power Platform, Power BI) | 6 core: Entra ID, Exchange, Defender, SharePoint/OneDrive, Teams, Intune |
| Checks | SCuBA baselines | 90+ checks across CIS and CISA SCuBA baselines |
| Compliance | NIST 800-53, MITRE ATT&CK | 5 compliance views: GDPR, NIS2, HIPAA, ISO 27001, SOC 2 |
| Insurance evidence | Not built in | Timestamped, exportable evidence reports (raw Graph proof per finding) |
| MSP reporting | Not built in | Multi-tenant, white-label PDF reports |
A few specifics worth being precise about:
- Continuous, not point-in-time. TenantVigil re-scans on a schedule and alerts you the moment a control that was passing starts failing — in plain English, not a control ID.
- Read-only and no infrastructure. Onboarding is a Microsoft OAuth consent for read-only Graph scopes. There is nothing to install and no PowerShell to maintain.
- License-aware. TenantVigil detects your actual M365 license tier and only reports controls your license can satisfy, so you do not chase fixes you cannot make.
- Built for evidence. Every finding carries the raw Graph response that proved it, and reports come in an executive and a technical format — white-labeled for MSPs.
Which should you use?
Honestly? They are complementary. If you want a free, deep, government-authored point-in-time check — including Power Platform and Power BI — run ScubaGear. If you need that posture watched continuously, translated into business risk, mapped to the compliance frameworks your auditor and insurer care about, and delivered without maintaining any tooling, that is the gap TenantVigil fills.
The user who runs ScubaGear today and thinks "great — now how do I keep it this way, and prove it?" is exactly who TenantVigil is for.
Run a free, read-only TenantVigil assessment of your Microsoft 365 tenant in under two minutes at tenantvigil.com/assess — no account, no agents, no PowerShell.