Product development operational reviews

At Zmags we have a quarterly and yearly planning cycle. It works really well, it’s well managed and it ensures that everybody at senior management level is on the same page. Part of the quarterly planning session is operational reviews of all functional areas of the organization. The original framework that we employ for the operational reviews was an adaptation of a sales department operational review kindly provided by the good people at OpenView. I’ve used it for a while and it’s been tweaked a fair bit along the way. I recently had a chance to see the draft operating review framework specifically for product development that Igor Altman at OpenView is creating (and as usual it’s really good).
Seeing Igor’s work got me thinking about how I do operational review presentations and the purpose of them. The way I see it, the purpose of the operational review is to analyze the inner workings and activities of a department or functional area with the purpose of identifying problems and potential improvements.
This is all well and good but the problem here is that at executive and board level only few people have (and should imho have) complete domain knowledge. This means the presenter will be presenting a specific view port into the problem domain and is forced to translate the specifics into commonly understandable terms.
So the tradeoff becomes presenting a too high level view that everybody understands but does not facilitate the identification of improvements or presenting a too detailed view that isn’t understood and causes unnecessary concern and pushes focus towards the KPI’s instead of their meaning.
In addition to the overall purpose of the review I find that the good ones also serve the following purposes:

• Provide executives with an overview of the past and planned operations of the department so that they can plan for upcoming cross departmental activities
• Provide knowledge and ensure that planned operations of the reviewed department is in line with the overall company strategic themes
• Identify areas of possible cross departmental synergy (given the above bullet there should be plenty)
• Instill confidence that you know what you’re doing (if that is the case) and that they don’t need to worry about the things you don’t draw their attention to
  

In general I try to make my review documents substantially shorter and put only a few bullets in writing on every slide. I think it provides a substantial advantage to start my presentation at a slightly higher level and then dive into the underlying KPI’s whenever necessary. In other words, if there really isn’t a quality problem I wouldn’t dive deep into low level quality metrics unless asked and in that case I would have those KPIs in a separate presentation. It (I hope) gives my colleagues the confidence that no matter what KPI they feel like drilling into, I know and have the data and that allows me to fast forward through some of the areas in which I have a strong believe that a lengthy discussion would be futile.
In summary I tend to stick to these basic rules

• Provide reasonably few metrics that highlight problem areas (I provide 5-10)
• Have the underlying KPI’s for each top level metric ready but don’t show them unless necessary (I try to have additional KPI’s ready in a separate presentation or dive right into the reporting systems)
• Prefer meaningful graphs over tables
• Weird domain specific acronyms that I don’t know, makes me feel like a jerk. I assume this is a general thing so I try to avoid them
• Highlight risks
• Listen! – This is your chance to pick the brains of the most expensive coworkers in your company. Don’t waste the opportunity
• Be exceptionally honest (this is your chance to tell them that the multiple-inheritance multi-apartment threaded COM object based architecture that your architect came up with wont be supported in the next version of windows (btw: the version for which the next release is targeted at))