Investigation
An incident was triggered after several users of the VSTO add-in encountered a "You are offline" message in the taskpane. The engineering team started investigating the issue on July 3, 2025, at 7:36 AM CET.During the investigation, the team quickly narrowed down the problem to a recent code change affecting the online check API endpoints. By reviewing deployment logs and release timelines, engineers pinpointed the specific release where an unintended modification to API pathing was introduced. This change resulted in the add-in’s inability to verify online status for certain users. By 7:47 AM CET, the root cause had been confirmed. Testing of several fixes began immediately in preview and test environments to ensure a safe restoration of service.
Mitigation
At 10:21 AM CET on July 3, 2025, engineers initiated deployment of a fix by reverting the affected API endpoints to their previous state. The fix was first validated in non-production environments before being rolled out to production clusters.
Resolution
By 10:40 AM CET on July 3, 2025, the corrective change had been fully deployed to all clusters. The engineering team closely monitored the platform, confirming that the VSTO add-in returned to normal functionality and that users were no longer seeing the "You are offline" message. No further issues were observed.
Post-Incident Actions
To prevent similar incidents in the future, the engineering team has taken the following actions:Added a smoke test specifically for critical unauthenticated APIs, such as the online check API.Ongoing monitoring will continue, and additional reviews of deployment and release processes are being planned to strengthen safeguards around API changes.
Impact and Scope
This incident impacted users across multiple tenants in all production environments utilizing the VSTO add-in. However, the issue was limited to a subset of users. Most users were not affected, as a fallback mechanism intervened and bypassed the online check, allowing the taskpane to load successfully.
Service was fully restored following the deployment of the corrective fix.