Investigation
On July 18, 2025, at 8:06 AM CET, an issue began affecting the AI Assistant system within the Hive environment. The problem was detected at 9:45 AM CET, when administrators reported that they could no longer access the prompt templates page in the Library app or manage these templates within the admin area.
The engineering team confirmed that the incident was isolated to the West Europe (Production 0) cluster and by 10:45 AM CET, had identified that recent changes made during a migration to a new Azure AI Search service had resulted in certain configuration settings not being properly loaded, which prevented access to prompt templates for affected users.
Mitigation
At 11:17 AM CET, the engineering team released an update to ensure that the application could reliably access and handle configuration settings, even if some values were temporarily unavailable. Due to the rapid response and containment measures from the engineering team, the impact was limited solely to the West Europe (Production 0) cluster. No other environments or tenants were affected.
Resolution
By 11:17 AM CET on July 18, 2025, the changes had been applied, restoring access to the prompt templates page in the Library app and enabling administrators to manage templates in the admin area for all affected tenants in the West Europe (Production 0) cluster. The team closely monitored the system to confirm full restoration of normal functionality, and no further issues were reported.
Post-Incident Actions
To prevent similar incidents in the future, the following steps have been initiated:
Impact and Scope
This incident affected only tenants on the West Europe (Production 0) cluster, specifically preventing access to and management of prompt templates in the Library app and admin area. The rapid response ensured that the issue was detected and resolved before any other clusters or users were impacted.
We sincerely apologize for the disruption caused by this incident. Maintaining a reliable and seamless experience for our customers is our highest priority, and we are taking immediate steps to further strengthen our processes and prevent similar issues in the future.