Fusing Puzzle Pieces Together

It's always interesting to see how complexity manifests itself, especially in the corporate world.  Take a relatively straightforward process for creating after-hours on-call schedules.  If we assume that we can't contact a specific team for a specific type of outage automatically because our alerting is currently too rudimentary, then it's necessary to either rotate through teams or select one team for initial triage.  

We decide to opt for the second option (one team for all triage) in order to avoid labor law nightmares.  But, we notice that the team we've selected is an expert in a particular class of problem, so we assign a certain type of alert to page that team under certain conditions. 

We then decide to use a different team that has less technical system knowledge, but knows the product domain better, for alerts on all other conditions.  If the issue proves to be too technical, then this team can contact the team mentioned above.

Now, we've got two escalation paths that need to be encoded in our escalation software, which is already notoriously tricky to use.  Throw in a misconfiguration for the escalation software that has no team on-call during boundary hours, and a misplaced assumption about why those hours are missing, and now you've got another gap to cover.

This is where it helps to have a fresh perspective, even if it's from a point of complete ignorance.  Then you can unmask the problem that led to the missing coverage hours.  You can also recognize that of the two teams responsible for after-hours on-call, one handles pretty much all issues and contacts the other team if they can't resolve a more technical concern.  So, only that team needs to be on-call.  And, voilá, your bezier curve complexity is now a straight line.

Nothing about this is immediately obvious when you're enmeshed in the details, but it helps, whenever possible, to break free of the going assumptions and ask why, even if it's a process that's been in place for ages.  Maybe especially because it's a process that has been in place for ages.

Until next time, my human and robot friends.