Evaluation of a project with future improvement of performance as a main goal seems easy at a first glance. Nevertheless, the above short decision-making-process could lead to very mismatching results.
Although ending a project is not always an easy thing to do, it is quite clear when it happens (when no one is working anymore for the project ), but the starting point of a project is a little bit more fuzzy, because projects can be prepared in a different way, depending on the customer demands, time to market constraints, prior knowledge about the product to be developed, the maturity of the project organization etc.
That means that projects start under different conditions, assumptions and interests and consequently it is very hard to compare them.
Project start is not a certain moment in time
It is quite helpful to see the start-up of a project as a process and not as a specific date in time. Organizations are free to define the duration of this start-up process in order to reach the level of information required to reduce financial and other risks.
Every organization if free to decide when to book hours on the project and talk about project cost and it is practical to do so, after a certain degree of detailed information about the objectives of the project has been agreed or all involved parties are clear about assumptions and other parameters.
Let us consider two extreme cases:
- Case 1: Members of the future project team are involved in the inception of the project idea. Requirements are consolidated during a period of two months and at some point there is the point where the project teams starts building. After this build phase the team delivers and the customer is satisfied.
- Case 2: There are no detailed requirements, but only an idea about the deliverables. The team starts building and along the way requirements are specified.
I believe that is is impossible to compare the performance of the two different projects. But even if we had only Case 1-projects it is impossible, or proven by practical experience to capture all the requirements and ideas at the beginning of the project. That means that somewhere in every Case-1 project there is a Case-2 hidden and vice versa.
What usually happens is that organizations do not take an inception of conceptual phase into consideration when it comes to calculating their performance KPIs. Having Case-1 projects it seems easy then to compare, but there is no practical way at the “Beginning of a product life cycle” to be able to define in detail the parameters of one or more projects required to bring a specific product to life. Meaning that there always be a factor that either you include or exclude in your calculation.
That all leads me to the conclusion that when you want to compare projects, you should be looking at the process not the work or the results during the project. What the team produced is not as important as how they did it when it comes to lessons-learned and comparisons. If you only look at KPIs we will be getting non comparable results.