Measure AI coding productivity by the cost and speed of verified outcomes, not by the volume of generated code. A useful scorecard tracks time to a user-valid result, first-pass acceptance, rework, escaped defects, human attention, and total cost. These measures reveal whether AI is improving delivery or merely increasing activity.
Choose the outcome unit first
Use a unit a user or operator can recognize: a completed onboarding step, a resolved production defect, a published integration, or a validated experiment. “Prompts sent,” “commits made,” and “lines generated” are activity counters. They can rise while the product stands still.
For each task, write the completion condition before work begins. If the outcome cannot be tested or observed, improve the task definition before measuring the agent.
Track six practical measures
- Outcome cycle time: elapsed time from an accepted task to verified completion.
- First-pass acceptance: share of outcomes that pass agreed checks without a repair loop.
- Rework rate: time spent correcting, reverting, or narrowing agent output.
- Escaped defects: problems discovered after the result was accepted or shipped.
- Human attention: active review, prompting, diagnosis, and coordination time.
- Cost per verified outcome: model, tool, infrastructure, and review cost divided by accepted results.
Do not collapse everything into one vanity score. A faster cycle with more escaped defects is a tradeoff to investigate, not an automatic win.
Establish a small baseline
Pick one recurring task class and compare several outcomes before and after introducing an agent workflow. Keep task difficulty and completion rules similar. A solo developer rarely has enough volume for statistical certainty, so use the numbers as decision evidence, not as universal benchmarks.
OpenAI’s GPT-5.6 efficiency report illustrates how model evaluations can compare output quality, tokens, and latency under controlled workloads. Your own project needs the same discipline at a smaller scale. The DORA AI-assisted development report provides broader evidence that adoption effects are mediated by delivery practices rather than by tool access alone.
Count verification as part of the work
An agent result is not productive until someone can trust it. Include test execution, visual or runtime checks, external delivery confirmation, and repair work in cycle time and cost. Excluding verification rewards fast drafts and hides the burden transferred to review.
The verify-before-shipping guide shows how to make that evidence repeatable.
Review trends by failure mode
Once a week or after a small batch, ask:
- Which task types complete on the first pass?
- Where does review time exceed implementation time?
- Which missing context creates repeat rework?
- Which checks catch defects before shipping?
- Where is the agent slower or riskier than manual work?
Change the workflow only where the evidence points. Better task briefs, narrower tools, stronger repository instructions, or a smaller task may matter more than switching models.
SoloMap supports this outcome-based view by keeping the planned step, completion criteria, run evidence, and handoff in the project. You can compare what was promised with what was actually verified without using code volume as a proxy. The SoloMap method treats evidence-backed completion as the unit of progress, while the build, sell, learn, improve loop connects delivery metrics to product learning.
Frequently asked questions
Is lines of code ever a useful AI productivity metric?
It can describe output size, but not value or correctness. Use it for review planning, never as the primary measure of productivity.
How many tasks do I need before comparing workflows?
Start with five to ten similar outcomes and label the result as directional. Increase the sample before making an expensive or irreversible decision.
Should agent waiting time count in cycle time?
Yes for end-to-end delivery. Track active human attention separately so you can see whether elapsed time rose while your own focused time fell.