Start with the data the model can receive, retrieve and return. Model preference comes later.
Microsoft's 2026 Data Security Index gives useful context, with an important boundary. The study is based on a survey of more than 1,700 security leaders across ten markets and interviews with security leaders. Microsoft reports that 32% of surveyed organizations' data security incidents involved generative AI tools. It also reports that 47% of surveyed organizations were implementing controls focused on generative AI workloads.
These figures describe the surveyed organizations. They do not establish that 32% of all breaches worldwide involve AI, or that 47% of organizations have finished an AI security program.
In previous Microsoft data-security and security-operations work, I learned to start with the data path. Which service is used? Which identity calls it? What data can that identity reach? Where are prompts and responses logged? Who can revoke access?
Use the survey as a prompt for inspection
The report identifies three priorities: unify data security, manage AI-powered productivity securely and use AI to improve security operations. Microsoft also says more than 80% of surveyed organizations were implementing or developing data security posture management strategies.
The practical interpretation is narrower:
- Find the AI services and identities already in use.
- Find sensitive data that is too broadly accessible.
- Set a policy for each data path.
- Test prevention, logging and response.
Buying another dashboard before completing those four steps usually creates a new place to look without resolving ownership.
1. Inventory sanctioned and unsanctioned AI use
Use Defender for Cloud Apps or an equivalent cloud access security broker to identify AI services. Record more than the application name:
| Field | Why it matters |
|---|---|
| Service and tenant | Separates an approved enterprise tenant from a personal account |
| User or workload identity | Gives the access path an owner |
| Data source | Shows whether the service can reach mail, files, code or business systems |
| Authentication method | Identifies OAuth grants, API keys and shared credentials |
| Upload and download path | Tells DLP where enforcement is possible |
| Business purpose | Supports a decision to allow, restrict or remove the service |
| Review date | Prevents an experiment from becoming permanent by neglect |
Defender XDR advanced hunting can help turn observed cloud application events into a review queue. This is a starter query, not a portable production rule. Application names and available fields vary by tenant and connector, so validate the result before scheduling it.
CloudAppEvents
| where Timestamp > ago(7d)
| where Application has_any (
"ChatGPT", "Claude", "Gemini", "Copilot", "Perplexity"
)
| summarize
Events = count(),
Users = dcount(AccountObjectId)
by Application, bin(Timestamp, 1d)
| order by Events desc
This counts observed events. It does not measure uploaded bytes or prove that sensitive data moved. That distinction matters. A useful inventory query should not pretend to be an exfiltration detector.
2. Fix the permissions that AI makes easier to search
Microsoft 365 Copilot works within the requesting user's permissions. That is useful, but it also makes old sharing mistakes easier to exploit. A broadly shared SharePoint site remains broadly shared when Copilot is added.
Use Purview Data Security Posture Management to prioritize:
- sensitive files available to large internal groups;
- unlabeled or incorrectly labeled content;
- stale external sharing links;
- sensitive locations with no clear business owner;
- high-risk users or workflows with access to several sensitive repositories.
Then fix the permission or data placement. A prompt policy cannot repair an inherited access-control mistake.
3. Apply DLP to a defined AI workflow
A DLP rule needs a clear enforcement point and a controlled test. Start with one sensitive information type and one supported AI path.
For each rule, record:
- the sensitive information type or label;
- the AI application or workload in scope;
- whether the action is audit, warn, block with override or block;
- the exception owner;
- the event and alert expected in Purview or Defender XDR;
- the test data and expected result;
- the review date.
Do not copy an email DLP policy into every AI workflow and assume it works. Prompt, browser upload, API and retrieval paths can expose different enforcement points. Verify the exact path used in your tenant.
A sensible policy split is often:
- Allow and monitor: approved enterprise services with contractual controls, identity governance and logging.
- Allow with restrictions: services where Conditional Access App Control or another session control can prevent sensitive uploads.
- Block: services with no approved purpose, no accountable owner or no acceptable data handling terms.
The decision should follow the data path. It should not follow a generic list of fashionable or unfashionable model vendors.
4. Rehearse an AI data incident
An alert is useful only if someone can act on it. A short playbook should answer four questions.
Confirm
- What data or label triggered the event?
- Was content sent, retrieved or only selected?
- Which user, workload and AI service were involved?
- Are prompts, responses and access logs available?
Contain
- Revoke the affected user session or workload token when justified.
- Disable the specific OAuth grant, connector or application path.
- Preserve audit records before making broad changes.
- Block further transfer of the affected data class if the scope is still unclear.
Scope and recover
- Identify the exact records exposed and their owners.
- Check whether the service retained prompts or used them for training under the applicable terms.
- Correct the underlying file permissions and DLP rule.
- Restore access through an approved workflow rather than leaving a permanent emergency block.
Decide and learn
- Assign legal, privacy and communications decisions to named owners.
- Record why the existing control did not prevent or detect the event.
- Add the failed path to the inventory and the next table-top exercise.
I would measure this playbook by evidence, not by a target such as "contained in four hours" without operational context. The test is whether the team can find the event, revoke the right access, preserve the facts and make a defensible notification decision.
Where Security Copilot fits
Microsoft reports that 82% of organizations in its study had developed plans to embed generative AI into data security operations. That is another survey finding, not proof that every planned deployment is mature.
Security Copilot can help summarize an event, retrieve related records and draft an incident note. Keep the analyst accountable for scope, severity and containment. Generated text should reduce lookup and writing time. It should not become the authority for a high-impact response decision.
What I would do next
Start with one week of observed AI application use. Pick the highest-risk data path. Confirm its owner, identity, data access, DLP coverage and revocation method. Then run a controlled test and a table-top exercise.
That sequence is less dramatic than declaring AI a new class of breach. It produces evidence about the environment you are responsible for.
Sources
- Microsoft Security Blog: New Microsoft Data Security Index report explores secure AI adoption to protect sensitive data
- Microsoft Security: 2026 Data Security Index: Unifying Data Protection and AI Innovation