Measuring a Process

If your organization is using defined processes for project and organizational tasks, you might be ready for some measures to objectively see how well they are being implemented. Measurement data can help an organization identify strengths, weaknesses and provide historical data for future planning. In this article, we give example measures for some common processes used within a project.

If you are using CMMI and implementing Generic Practice (GP) 3.2, the ideas listed below are examples of what could be measured to obtain insight into specific process implementations. Measurements are not required for GP 3.2, they are just one option. Tailor these examples to fit your needs or use them as a starting point to generate your own measurements.

In the examples, the term Earned Value is used. Some organizations use Earned Value Management to track project work. Each project task is assigned a value, and when that task is complete, the project records that value to indicate how much of the total project is complete. When process steps such as design, verification, risk and project tracking are included in a project’s schedule, the completion of these steps is reflected in the total earned value for the project. When these process steps are skipped, or become stalled, the EVM system reflects these problems.

[Definition: # = Number of]

Requirements Management
Planned/actual effort/schedule to perform the process
# Requirements, growth, volatility over time
# Non-compliances for this process

Project Planning and Management
Planned/actual effort/schedule to perform the process
Final Earned Value number (Cost Performance Index, Schedule Performance Index) or planned vs. actual data showing how well planning and tracking went
# Non-compliances for this process

Risk Management
Planned/actual effort/schedule to perform the process
# Risks over time (increasing or decreasing)
# Risks realized or averted due to risk management
# Risks open vs. closed
# Non-compliances for this process

Configuration Management
Planned/actual effort/schedule to perform the process
# CM-related defects
# Percentage of bad builds
# Non-compliances for this process

Process and Product Assurance
Planned/actual effort/schedule to perform the process
Earned Value of the process assurance activities (i.e., whether all assurance activities were performed, and whether they were under/over budget)
# Requests for help vs. #unfrequented (or scheduled) audits (indicating how sought after the Quality Assurance group is)
# Non-compliances for this process

Measurement and Analysis
Planned/actual effort/schedule to perform the process
# Measures collected but unused
# Non-compliances for this process

Requirements Development
Planned/actual effort/schedule to perform the process
# Derived requirements (indicating the amount of initial ambiguity/completeness)
# Requirements, growth, volatility over time
# Defects in requirements documents, or defect density
# Non-compliances for this process

Design and Implementation
Planned/actual effort/schedule to perform the process
# Defects in design and code/drawings, or defect density
# Output over time (e.g., code/changes/design per engineer-week)
# Non-compliances for this process

Product Integration
Planned/actual effort/schedule to perform the process
# Defects found or defect density
Build audit results (e.g., #bad builds)
# Non-compliances for this process

Verification
Planned/actual effort/schedule to perform the process
# Defects found in verification or defect density
# Non-compliances for this process

Validation
Planned/actual effort/schedule to perform the process
# Test cases
# Defects found in validation, defect density, or test pass/fail percentage
# Actual test cycles vs. planned
# Rate of defect find vs. defect resolution
# Non-compliances for this process

Decision Analysis and Resolution
Planned/Actual Effort to perform the process
Earned Value for DAR tasks
# DARs performed per project
# Non-compliances for this process

Process Improvement
Planned/actual effort/schedule to perform the process
Earned Value for improvement tasks
Adoption of processes over time (e.g., #processes or practices used, measured via a mini-assessment)
# Defects found in process assets
# Non-compliances for this process

Training
Planned/actual effort/schedule to perform the process
# Training hours planned vs. received
Training evaluation scores
# Non-compliances for this process

[Thanks to Ed, Bruce, Pat, Christof, Michael and John for the review.]