@@ -8,7 +8,7 @@ Related issue: https://gitlab.developers.cam.ac.uk/uis/devops/devhub/docs/issues
# Summary
On 28/03/2019 at approximately 16:15 UTC a full backup of the GitLab database, repositories and Docker image registry from an earlier time was restored to the production instance instead of a test instance created for this purpose. An unrelated load-spike issue masked early warning signs that this had occurred until several minutes later when the loss was noticed. Confusion over the timeline of events meant that the existence of a later backup was not discovered until approximately an hour after the original incident and after database histories had diverged.
On 28/03/2019 at approximately 14:15 UTC a full backup of the GitLab database, repositories and Docker image registry from an earlier time was restored to the production instance instead of a test instance created for this purpose. An unrelated load-spike issue masked early warning signs that this had occurred until several minutes later when the loss was noticed. Confusion over the timeline of events meant that the existence of a later backup was not discovered until approximately an hour after the original incident and after database histories had diverged.
Users were notified of the data loss and a parallel instance of GitLab with our most recent backup was provided to users. Future mitigating actions were determined and are in the process of being carried out.
