As part of our security protocol, we conducted a regeneration of the access credentials for our main storage account. Unfortunately, this action resulted in the loss of access for our preview engine, which relies on accessing the storage to store generated previews. Consequently, the preview engine was unable to save the generated previews during this period.
The incident came to light on the 27th of July at 16:29 PM when users reported the absence of saved previews. Our engineering team swiftly responded to the issue and resolved it by releasing a new version of the component responsible for generating previews. This updated version included the new security credentials and was deployed at 17:59 PM, successfully restoring the preview generation functionality.
In the future, we will take extra precautionary measures to avoid similar incidents. These measures include changing the security method used by the preview component to access our storage account, adding comprehensive documentation covering all steps of the process, implementing pre-change validation procedures, and deploying robust monitoring systems to detect any potential access-related anomalies promptly.