postmortem-ir-1-undeployed-app-service-response-slow.md

Open Notebook in Datadog

1h Oct 24, 7:54 pm - Oct 24, 8:54 pm

Postmortem IR-1: Undeployed App Service Response Slow

Created by Ryan MacLean MacLean on Oct 24

INCIDENT PROPERTIES
Status	resolved
Severity	SEV-5
Started	Oct 25, 2021 03:39 am UTC
Commander	Ryan MacLean MacLean
Incident Overview	IR-1

You can generate a postmortem from any resolved incident with these fields pre-filled, along with incident metadata and timeline.

What Happened?

Impact on Customers

All clients impacted!.
This lasted for 15 minutes, beginning at 8:39 pm .

Why Did it Happen?

Root Cause

Testing

Timeline

---

Oct 24, 2021 at 8:39 pm

Ryan MacLean MacLean

Avg of synthetics.http.download.time over check_id:mm8-sgf-q4k,step_id:ixj-d2n-dbb

---

Oct 24, 2021 at 8:40 pm

Chat started by Ryan MacLean MacLean
https://chat.aka.msVideo call started https://video.teams.ms

---

Oct 24, 2021 at 8:41 pm

Customer impact updated by Ryan MacLean MacLean

Scope: All clients impacted!
Started at: 10/24/2021 8:39 pm

---

Oct 24, 2021 at 8:41 pm

Ryan MacLean MacLean

Update Still working on it!

---

Oct 24, 2021 at 8:42 pm

Field updated by Ryan MacLean MacLean

Other Docs: Sharepoint

---

Oct 24, 2021 at 8:42 pm

Task created by Ryan MacLean MacLean
Please try deploying the current branch to the app service @all

---

Oct 24, 2021 at 8:43 pm

Task updated by Ryan MacLean MacLean
Please try deploying the current branch to the app service @all
Assignees: Ryan MacLean MacLean
Due date: 10/25/2021

---

Oct 24, 2021 at 8:43 pm

Field updated by Ryan MacLean MacLean

Notified: Ryan MacLean MacLean

---

Oct 24, 2021 at 8:50 pm

Fields updated by Ryan MacLean MacLean

Root Cause: Testing
Detection Method: Monitor

---

Oct 24, 2021 at 8:53 pm

Ryan MacLean MacLean

Link to notebook for incident -
https://us3.datadoghq.com/notebook/648/azure-incident-response?view=view-mode

---

Oct 24, 2021 at 8:54 pm

Incident set to resolved by Ryan MacLean MacLean
Customer impact updated
Ended at: 10/24/2021 8:54 pm

---

How do we prevent it in the future?

Action Items

• Please try deploying the current branch to the app service @all
  Created by: Ryan MacLean MacLean