What is a Benchmark?
Definitions
In SPR terminology, baseline and benchmark are similar but distinct
activities. Figuratively, a baseline is a "line in the sand"
for an organization whereby it measures important performance characteristics
for future reference.
A benchmark is best understood by way of the original derivation
of the word itself. Tradesmen engaged in repetitive tasks, such
as sawing lumber to consistent lengths, often placed notches on
their workbenches to indicate placement of boards prior to cutting.
Literally, a benchmark became a standard for comparison and an indicator
of past success.
Applying these concepts to software organizations, SPR uses its
expertise in measurement to assist clients in establishing meaningful
baselines. In addition, SPR draws on its extensive industry knowledge
base, derived from more than 9000 completed software projects of
all types, as a reference point from which to compare ("benchmark")
client baseline data. It is the actual organizational data that
forms the baseline and the comparative analysis against industry
data that constitutes the benchmark.
A Line in the Sand
SPR baseline engagements typically begin by establishing a point-in-time
inventory and size for a relevant group of software applications
and/or projects. Usually, software baselines also include the other
"hard" data (schedules, costs, effort, defects) associated
with an inventory. If a client has an established measurement capability,
SPR can audit and validate the accumulated information. In the absence
of a formalized measurement program, SPR can collect and measure
the necessary data itself (with the active participation of the
client organization, if possible).
SPR also collects "softer" attribute data for each application
and each project included in a particular study through a detailed
interview process. The interview covers five fundamental areas of
interest specific to each application and project:
| • |
Personnel |
| • |
Process |
| • |
Product |
| • |
Technology |
| • |
Environment |
In addition to the application size and other hard data collected
in setting the initial baseline and the attribute data gathered
in team interviews, specific information regarding age, complexity,
stability, staffing, levels of documentation, and work breakdown
structure (life-cycle components) are collected. We also collect
information regarding user satisfaction, enterprise goal alignment,
and organizational perceptions of risk and value.
Comparison to Industry Benchmarks
The final step entails a comparison of related project and application
data against industry average results, best-in-class results (usually
derived from the top 5% to 10% of companies within a specific industry),
or other subsets of the SPR knowledge base that may reveal useful
insights (e.g., "middleware" to "middleware").
Using SPR's proprietary analysis tool, SPR
KnowledgePLAN®, we then model against the selected portions
of the knowledge base in order to identify and quantify opportunities
for improvement based on the known characteristics of comparable
projects. With this approach, SPR can show how you compare in the
areas that matter most: quality, productivity, and time to market.
Whether it's a comprehensive baseline and benchmark or a quick
productivity snapshot, Software Productivity Research has the knowledge,
experience, and industry data to enhance understanding of development
organizations and to provide the basis for measurable gain.
|
