8 Common Mistakes in a Microsoft 365 Security Review
Plenty of organisations do a Microsoft 365 security review once a year, tick the box and feel better. Months later, they discover that the same weaknesses are still there. A phishing incident exposes an old exclusion, or an insurer asks a question that the review never covered, or a new IT manager finds a pile of findings with no owners.
The problem is rarely a lack of effort. It is the way reviews are scoped, run and followed up. The same eight mistakes appear again and again, in small firms and large ones. They are easy to make, because each feels reasonable at the time.
This guide walks through them one by one, with a plain explanation of why each happens and what to do instead. Whether you are about to run your first Microsoft 365 security review or you have done several, you may recognise a few. At the end, there is a short repeatable routine you can adopt so that the next review builds on the last instead of starting from scratch.
Mistake 1: Treating Secure Score as the whole review
Why it happens: Secure Score is right there in the Defender portal. It gives you a number, a list of actions and a feeling of progress. It is tempting to work through the list, watch the percentage rise and call the job done.
Why it is a problem: Microsoft itself explains that Secure Score does not measure the likelihood of a breach. A completed action may only apply to some users. A policy might be enabled with a large exclusion group. A recommendation may not match your business at all. A tenant can score well and still have admin sprawl, loose sharing and risky apps.
What to do instead: Use Secure Score as a starting point. For each action, ask four questions. Which users and apps does it cover? Who is excluded and why? Does it fit our licences and work patterns? What evidence shows it works? Treat the score as a prompt for those questions, not as the answer.
Mistake 2: Setting a scope that leaves gaps
Why it happens: Scope is usually decided in a rush. "Check Exchange and Teams" sounds fine in a kickoff call. Later, nobody remembers that a second tenant exists, that on-premises Active Directory feeds identity, or that a third-party mail filter sits in front of Exchange.
Why it is a problem: Attackers do not respect your scope. If the weak point is a forgotten tenant or a connector from an old migration, a review that ignored it gives false comfort. An incomplete scope also makes findings hard to compare year on year.
What to do instead: Write the scope down. List tenants, domains, workloads, hybrid components, evidence period and the audience for the report. Include a short "not covered" section so everyone knows where the boundaries sit. Revisit the scope whenever you merge, migrate or add major services
Mistake 3: Checking policies without checking who they cover
Why it happens: It is natural to review a list of policies by name and status. "Require MFA for all users: On." Tick.
Why it is a problem: A policy name says what it was meant to do, not what it does. The real coverage depends on inclusions, exclusions, conditions, licensing and how policies combine. A group of 30 excluded users, left over from a project in 2022, can undo a control that looks perfect in the list.
What to do instead: Review effect, not labels. For each key control, such as MFA, device compliance or sharing restrictions, identify the target population, the exclusions, the owner of each exclusion and the evidence that the control applies in real sign-ins. Use report-only mode and sign-in logs to test. Where an exception must exist, write down the reason and set a review date.
Mistake 4: Ignoring guests, service accounts and apps
Why it happens: Reviews focus on employees because employees are the people you know. Guests, service accounts and third-party apps feel like background noise.
Why it is a problem: Non-employee access often grows quietly. Guest accounts from finished projects keep their access. Service accounts run with permanent passwords and no MFA. Apps hold permissions to read mail and files long after the person who approved them has left. Attackers value these paths because nobody is watching them.
What to do instead: Add three explicit workstreams. First, a guest review that lists who they are, who invited them and what they can reach. Second, a service account review that identifies each account, its owner, its purpose and how it authenticates. Third, an application review that covers enterprise apps, app registrations and consent grants. Remove what has no owner and restrict what is too broad.
Mistake 5: Mixing up a missing record with a missing control
Why it happens: During a review, you often cannot find evidence for something. A log is not available. A report shows no data. It feels natural to write "not implemented."
Why it is a problem: If you record a missing log as a missing control, you create false findings, waste time on fixes that were not needed and lose credibility with your own team. The reverse is also dangerous: assuming a control is working because you saw no alerts, when alerting was never switched on.
What to do instead: Use three clear labels in your notes and report: "observed" (we saw the evidence), "not observed" (we looked and the control is absent), and "unable to verify" (the evidence was not available, and here is why). Those three labels save arguments later and make the report more honest and more useful.
Mistake 6: Recommending fixes that do not fit your licences or your business
Why it happens: Benchmarks and scanners produce recommendations without knowing what you own or how you work. Reviewers sometimes forward the output more or less as it is.
Why it is a problem: A recommendation that needs Entra ID P2, Defender for Office 365 Plan 2 or an advanced auditing licence is useless if you do not have it and have no budget for it. A recommendation that blocks external sharing outright may bring a sales team to a halt. When people see impractical advice, they stop trusting the whole report.
What to do instead: Start with a licence inventory, by user group. Sort recommendations into three buckets: fixes you can make with what you already own, fixes that need a licence change, and fixes that are process or people matters. Check each against how the business actually works, and talk to department heads before you change something that affects them. Put the "use what you own" bucket first. It is usually where the quickest wins live.
Mistake 7: Ending the review at the report
Why it happens: Writing the report feels like the finish line. The team is tired, the project budget is spent and day-to-day work is waiting.
Why it is a problem: A report with no owners, dates or follow-up is a snapshot that goes stale. Findings sit unresolved, nobody confirms fixes work and the same issues turn up next year. This is often the biggest reason people say reviews "do not change anything."
What to do instead: Build the follow-up into the engagement. Each finding needs an owner, a priority, a target date, a dependency check and a closure test. Hold a short remediation meeting a week after delivery, and a progress check every month. Schedule a revalidation, where someone confirms the fixes actually work, ideally between 60 and 90 days later. Track it in whatever tool your team already uses, whether that is a ticket system or a shared spreadsheet.
Mistake 8: Reporting for the auditor, not the reader
Why it happens: Technical teams write reports for other technical people. They are full of settings, policy names and acronyms. The business owner or finance director who needs to approve a budget struggles to see what matters.
Why it is a problem: If decision-makers cannot understand the risk in plain terms, they will not fund the fix. If the technical team cannot see the exact setting and evidence, they will not know what to change. Reporting gaps leave both groups unhappy.
What to do instead: Write at two levels. Give leaders a one-page summary: the top risks in plain language, what they could cost in business terms, what is already good and what it will take to fix the worst issues. Give the technical team a detailed section with the observed condition, evidence, recommended change, owner and validation step for each finding. Avoid jargon where a simpler word works, and define the terms you cannot avoid.
Audit fatigue: the quiet ninth problem
One more issue deserves a mention, because it sits behind several of the mistakes above. Audit fatigue happens when teams are asked for evidence repeatedly, by insurers, customers, regulators and internal reviewers, and each request starts from zero. People get tired, answers get sloppier and reviews feel like a tax.
The cure is to collect and keep evidence as a normal habit. Store exports, screenshots and policy documents in an organised place with dates. Maintain a living record of exceptions and their owners. When the next request arrives, much of the answer is already prepared
A repeatable review routine
Instead of one big annual event, many organisations do better with a rhythm. Here is a simple pattern that works for most small and mid-sized businesses:
Monthly (30 minutes):
- Check Secure Score history for unexplained changes.
- Review new admin role assignments and new enterprise apps.
- Look at alerts for new inbox rules and forwarding.
Quarterly (half a day):
- Review Conditional Access policies and exclusions.
- Run a guest and external sharing report.
- Review privileged roles and service accounts.
- Check open findings and update their status.
Annually (a few days, or an independent engagement):
- Run a full Microsoft 365 security review across identity, email, collaboration, devices, apps and logging.
- Compare results with last year's.
- Update the scope, licence inventory and risk ranking.
- Refresh evidence for insurers and customers.
After major changes:
- Run a focused review after a merger, a migration, a major licence change, a security incident or before a Copilot rollout.
Keep it light enough that people will actually do it. A modest routine followed consistently beats an ambitious one that lapses after a quarter.
Questions to ask at the end of any review
Before you close a review, run through a quick check:
- Does the report say exactly what was covered and what was not?
- Can we show evidence for each major finding?
- Does each finding have an owner and a date?
- Do the recommendations match our licences and our way of working?
- Can a non-technical leader understand the top three risks?
- When will we check that the fixes worked?
- Where is the evidence stored, and who can access it?
If any answer is "no" or "not sure," you have found your next action.
Frequently asked questions
How long should a Microsoft 365 security review take?
It depends on the scope. A focused review of identity and email may take a few days. A multi-tenant or hybrid review takes longer. The preparation and follow-up often matter more than the fieldwork itself.
Who should be involved from our side?
At least the person accountable for risk, such as an IT director or owner, plus the administrators of identity, email, SharePoint and devices. Include a business representative who can speak to how teams work.
Should we review everything at once?
Not necessarily. If time is short, start with identity and email, since they carry the highest risk for most businesses. Add collaboration, devices and logging in later phases.
What if the review finds a lot of problems?
That is normal, and it is better to find them than not. Prioritise by risk and ease, fix the top items first and be open about what will take longer. A long list with a clear order of work is progress.
Do we need an outside reviewer?
Not always. Internal reviews are valuable, particularly for routine checks. An independent reviewer helps when you want a fresh perspective, formal evidence for insurers or customers, or specialist skills your team lacks.
Final thoughts
The mistakes above are common because they are natural. People trust the number on the dashboard, rush the scope, read policy names instead of effects, and stop at the report. The fixes are not complicated: write the scope down, check who is covered, look at guests and apps, label evidence honestly, fit advice to licences, plan the follow-up and write for the reader.
If your last review ended with a PDF and little else, treat that as useful information about the process, not about your team. Adopt a simple routine, assign owners and book the revalidation. If you would like an independent partner to help, Atlant Security runs evidence-led reviews with practical remediation guidance. Either way, the goal is a tenant that is easier to understand and safer to run next quarter than it is today.