Story ideas: Software practices

The typing argument is usually settled by taste. Controlled experiments, a check of real JavaScript bugs, and a failed replication of the most cited repository study show what types do and do not buy.

Tension: Fans say types catch bugs before users do; skeptics say most bugs are not type errors and dynamic languages are faster to write. Both have some evidence.

Other titles: Static vs Dynamic Typing: What the Experiments Show · The Language-Bug Study That Fell Apart

Could chain with: decision:10x-developers-and-productivity · decision:continuous-delivery

Readiness

Ready Coherent (0.75), central conclusion supported, 4 essential findings verified, no pending review.

Coherence

0.75 (review): One question, do static types pay off: errors caught before running, how much faster the fix is, the repository-scale bug claims that fell apart, and the prototyping speed argued for dynamic typing. The runtime performance claim is a separate question with only a qualified, implementation-bound verdict, so it is pruned; the in-editor feedback claim is folded in from its own candidate.

Why it ranks here

editorial 0.79 = mean of coherence 0.75 (review), relevance 0.79 (10 sources, 9 channels), curiosity 0.7 (decision, a study overturned the guess), stakes 0.7 (lean: pro), depth 1 (6 studies, 4 claims, tension); evidence 0.75 = strongest Berger et al. 2019 (languages reproduction) at 0.75 (direct) across 1 independent document; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Controlled experiments on type-error fixing, a measure of how many real bugs a type checker would catch, the replication of the repository-scale study, and development-time comparisons.

• Berger et al. 2019 (languages reproduction): right (0.75, direct)
• Kleinschmager et al. 2012 (static types): right (0.6, direct)
• Hanenberg et al. 2014 (static types): right (0.6, direct)
• Gao, Bird & Barr 2017 (types vs JS bugs): right (0.5, direct)
• Kleinschmager et al. 2012 (static types): right (0.65, partial)
• Hanenberg et al. 2014 (static types): right (0.65, partial)
• Prechelt 2000 (seven languages): partly (0.45, partial)
• Hanenberg et al. 2014 (static types): partly (0.3, direct)
• Ray et al. 2014 (languages on GitHub): partly (0.2, partial)

Research gaps

• Ray et al. 2014 (languages on GitHub) is context only: observational bug-fix counts from a corpus whose TypeScript sample Berger et al. later showed was mislabelled; it cannot decide what a compiler catches

Premise and arc

In Kleinschmager et al.'s controlled experiment the Java version of a type-error task failed to compile while the Groovy version only failed at run time, and the static side won on time: 78 against 149 seconds and 44 against 246 seconds on the two type-error tasks. Hanenberg et al.'s journal version found every indicator favoured static types on those tasks because the compiler names the location while a run-time failure can appear far from the cause. Those experiments used a compiler, not an in-editor checker; an editor shows the same message sooner, but that was not what was tested. Gao, Bird and Barr annotated 400 fixed public JavaScript bugs and the type checker flagged 15% of them, so most bugs are not type errors. Ray et al.'s 729-project GitHub study, which linked languages to defect rates, did not survive Berger et al.'s reanalysis: mislabelled files, duplicated projects, 36% false-positive bug labels and no multiple-testing correction left 4 of 11 associations with very small effects. On prototyping, Prechelt's scripts took a median 3.1 hours against 10.0 for C, C++ and Java, but on one problem, with self-reported script times, and the author credits built-in associative arrays rather than typing; Hanenberg's own series has tasks going each way.

hook

Every few months the static-versus-dynamic argument flares up again, usually citing a big study that says language choice changes your bug rate.

conflict

Types clearly catch some errors earlier, but how many bugs are type errors, and does the extra ceremony slow you down?

evidence

The type-error experiments and their timings, the 15% of real JavaScript bugs a checker would have flagged, the failed replication of the GitHub study, and the prototyping comparison with its caveats.

resolution

Static types find type errors sooner and point at them, which makes those fixes faster; they catch a minority of real bugs; the repository-scale claims about whole languages did not hold up; and the speed advantage for dynamic languages is narrow and confounded.

takeaway

Choose static types for the faster, located feedback on type errors, not for a promise of fewer bugs overall.

Claims

• Static typing catches type errors at compile time before code reaches users
• In-editor type errors give faster feedback than running the program and tracing a crash.
• Large-scale repository studies claiming some languages produce fewer bugs did not survive replication.
• Dynamic typing is faster for prototyping and rapid iteration

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

TDD is sold as the discipline that makes code better, and plenty of teams require it. Controlled experiments with students and professionals checked whether the test-first order is what makes the difference.

Tension: Advocates say test-first forces better design and fewer bugs. The experiments compared it with writing tests in small steps straight after the code and could not tell the two apart.

Other titles: Test First or Test After: Does the Order Matter? · What Happened When Researchers Put TDD to the Test

Then answers: Does high test coverage mean your tests will catch your bugs?

Readiness

Ready Coherent (0.9), central conclusion supported, 1 essential finding verified, no pending review.

Coherence

0.90 (review): One question: does writing the test first produce better code or faster work than testing in small steps afterwards? Three independent studies answer it.

Why it ranks here

editorial 0.66 = mean of coherence 0.9 (review), relevance 0.46 (3 sources, 3 channels), curiosity 0.8 (surprise, a study overturned the guess), stakes 0.4 (lean: calibrating), depth 0.75 (3 studies, 1 claim, tension); evidence 0.78 = strongest Fucci et al. 2016 (TDD replication) at 0.6 (direct) across 3 independent documents; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Controlled comparisons of TDD against a disciplined test-last practice on quality and productivity, plus a review of the wider literature.

• Fucci et al. 2016 (TDD replication): right (0.6, direct)
• Fucci et al. 2017 (TDD dissection): right (0.55, direct)
• Karac & Turhan 2018 (TDD review): right (0.5, direct)

Premise and arc

Fucci et al. 2016 ran a blind-analysis crossover replication with 21 graduate students and found no difference between TDD and incremental test-last in tests written, external quality or productivity, with negligible effect sizes. Fucci et al. 2017 analysed 82 task sessions by 39 professionals and found that the test-first order dropped out of both the quality and the productivity models; the authors conclude test-first and test-last could be equally effective when done in equally small, steady cycles. Karac and Turhan's 2018 review of 15 years of studies finds no convincing evidence that TDD consistently beats other iterative methods. The limits: the comparison is with a disciplined test-last practice, not with writing no tests; the tasks were short; and long-term maintenance was not measured.

hook

Many teams treat writing the test first as the rule that separates good engineers from the rest.

conflict

If test-first is what makes code better, an experiment that changes only the order should show it.

evidence

The 2016 replication with graduate students, the 2017 study of professionals where order dropped out of the models, and the 2018 review of 15 years of results.

resolution

Against testing in small steps right after the code, test-first made no measurable difference to quality or productivity; the short cycle, not the order, is what the studies point to.

takeaway

Write tests in small, steady steps; whether each test comes first or second is a matter of taste, not quality.

Claims

• Test-driven development produces no better quality or productivity than other disciplined testing approaches.

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

The 10x programmer shapes hiring, pay and how people judge themselves. A reanalysis of 61 experiments traces the famous figure to its source and measures the typical gap.

Tension: Everyone has met a programmer who seems ten times faster, and the 28:1 figure is quoted as proof. The data behind it are thin, and the typical gap is far smaller.

Other titles: Where the 10x Programmer Number Really Comes From · How Much Faster Is the Fastest Programmer, Really?

Could chain with: decision:static-vs-dynamic-typing · decision:hours-interruptions-and-offices

Readiness

Ready Coherent (0.8), central conclusion qualified with accepted qualifications, 3 essential findings verified, no pending review.

Coherence

0.80 (review): One question, are 10x programmers real: the spread between programmers, whether 10x is the typical gap, and where the 28:1 legend came from. The hiring claim is a management prescription no study here checks, and its wording is flagged, so it is pruned.

Why it ranks here

editorial 0.75 = mean of coherence 0.8 (review), relevance 0.76 (9 sources, 8 channels), curiosity 0.7 (decision, a study overturned the guess), stakes 0.5 (lean: skeptic), depth 1 (4 studies, 3 claims, tension); evidence 0.63 = strongest Prechelt 1999 (28:1 legend) at 0.55 (direct) across 2 independent documents; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Work-time distributions from many controlled programming experiments, the original source of the 28:1 figure, and other examples of repeated figures that did not hold up.

• Prechelt 1999 (28:1 legend): partly (0.55, direct)
• Prechelt 2000 (seven languages): partly (0.35, direct)
• Prechelt 1999 (28:1 legend): partly (0.5, direct)
• Dybå & Dingsøyr 2008 (agile review): partly (0.4, direct)
• Berger et al. 2019 (languages reproduction): partly (0.35, direct)

Premise and arc

Prechelt's 1999 reanalysis of 61 experiments with 614 subjects finds that the slowest quarter of programmers typically takes 2 to 3 times as long as the fastest quarter, rarely more than 4, and that best-against-worst ratios above 10 are rare. So differences between people are real and larger than most experimental effects, but ten times is the extreme pairing, not the typical gap. The famous 28:1 came from Grant and Sackman's debugging experiment (170 hours against 6); Prechelt's notes put it at 14:1 once two methodological mistakes are corrected, from 12 subjects, and it kept being quoted after it was debunked in 1981. Prechelt's 2000 study shows the same picture on one task: bad-to-good work-time ratios of 1.5 to 3.2 within a language, with a few outliers far above the median. The claim that the spread is mostly a person's own task-to-task noise is not measured in these studies; they show the spread is smaller than the legend, not why. The subjects were mostly students, which may understate the range among professionals. Berger et al.'s reanalysis of the language-and-defects study is a second example of a widely cited figure that did not survive checking; two examples show it happens, not how often.

hook

The 10x programmer is quoted in hiring posts and conference talks as settled science.

conflict

If the gap were ten to one, experiments would show it routinely. Where did the number come from?

evidence

The 61-experiment reanalysis and its typical quarter-to-quarter ratio, the 1960s 28:1 figure and its correction, the within-language spread from a second study, and the language-defect study as another legend that fell apart.

resolution

Some programmers are several times faster than others on a given task; ten times is the best against the worst, and the 28:1 that made the idea famous was a flawed figure from 12 people.

takeaway

Expect a two-to-three-fold spread between typical fast and slow programmers, and ask for the source when someone quotes a bigger number.

Claims

• Some programmers are vastly more productive than their typical peers
• The 10x spread between fastest and slowest programmers is mostly noise: a programmer's own task-to-task variation is as large as the variation between programmers.
• Many widely repeated software engineering facts trace back to sources that are missing, measure something else, or are methodologically flawed

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Coverage is the number teams put on dashboards and gate merges on. A large experiment checked whether a higher number means a suite actually catches more faults.

Tension: Coverage feels like a measure of safety, but it only counts lines that ran, not whether any test would notice them being wrong.

Other titles: Why 90% Coverage Can Still Miss Your Bugs · What Makes a Test Suite Worth Trusting?

Part of: surprise:test-driven-development-produces-better-quality, as a follow-up question; not produced on its own.

Readiness

Ready Coherent (0.7), central conclusion supported, 3 essential findings verified, no pending review.

Coherence

0.70 (review): Pruned to one question, can you trust a test suite because of its coverage number: coverage against fault-finding, what a trusted suite buys, and one way suites lose trust. The TDD claims are their own story (chained ahead of this one) and the two test-first beliefs are unchecked.

Why it ranks here

editorial 0.68 = mean of coherence 0.7 (review), relevance 0.76 (9 sources, 7 channels), curiosity 0.5 (decision), stakes 0.7 (lean: pro), depth 0.72 (2 studies, 3 claims); evidence 0.7 = strongest Inozemtseva & Holmes 2014 (coverage) at 0.7 (direct) across 1 independent document; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: A controlled measure of coverage against fault detection with suite size held fixed, and evidence on what a trusted suite does for delivery.

• Inozemtseva & Holmes 2014 (coverage): right (0.7, direct)
• DORA State of DevOps 2019: partly (0.35, partial)

Research gaps

• No study strong enough to decide: "A trustworthy automated test suite lets developers change and deploy code without fear"
• Unchecked: "Writing a test per method and mocking every dependency couples tests to implementation so refactoring breaks them"

Premise and arc

Inozemtseva and Holmes generated 31,000 test suites for five large Java systems and measured how many mutants (artificially injected faults) each suite caught. Once suite size was controlled, coverage correlated only low to moderately with that effectiveness, and the strength varied between systems, so a high coverage number does not tell you a suite is effective. What a trusted suite is for is less well measured: DORA's 2019 survey finds automated testing predicts continuous integration, which predicts continuous delivery, and describes developers trusting a passing suite, but it measures neither fear nor confidence and rests on self-reports. The claim that a test per method with every dependency mocked makes refactoring break the tests is made by two sources and checked by no study in the corpus; present it as practitioner experience.

hook

Your dashboard says 90% coverage. Would your tests notice if half those lines were wrong?

conflict

Coverage counts lines that ran; it says nothing about whether a test checked what they did.

evidence

The 31,000-suite experiment with size held fixed, then DORA's survey link from automated testing to delivery, then the practitioner warning about over-mocked tests.

resolution

Coverage is a weak signal of fault-finding once you account for how many tests there are; a suite earns trust by checking behaviour, and the confidence it gives is claimed more than measured.

takeaway

Treat low coverage as a warning and high coverage as no guarantee; judge tests by whether they would fail when the behaviour breaks.

Claims

• Line coverage shows code was executed, not that the tests check its behaviour
• A trustworthy automated test suite lets developers change and deploy code without fear
• Writing a test per method and mocking every dependency couples tests to implementation so refactoring breaks them

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Teams hunt bugs with complexity metrics, coverage and churn. Microsoft's own data found that measures of who worked on the code predicted failures better than any of them.

Tension: You would expect the code to be the best guide to where its bugs are. In Windows, the organisation behind the code was a better one.

Other titles: Why Too Many Cooks Means More Bugs · The Best Bug Predictor at Microsoft Was Not in the Code

Part of: decision:clean-code-and-technical-debt, as a follow-up question; not produced on its own.

Readiness

Ready Coherent (0.85), central conclusion qualified with accepted qualifications, 1 essential finding verified, no pending review.

Coherence

0.85 (review): One question: does who works on the code predict its defects better than measures of the code itself?

Why it ranks here

editorial 0.57 = mean of coherence 0.85 (review), relevance 0.23 (1 source, 1 channel), curiosity 0.8 (surprise, a study overturned the guess), stakes 0.3 (lean: neutral), depth 0.65 (3 studies, 1 claim, tension); evidence 0.78 = strongest Nagappan et al. 2008 (org structure) at 0.6 (direct) across 3 independent documents; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Head-to-head prediction of defect-prone components from organisational and code metrics, and replications outside one company.

• Nagappan et al. 2008 (org structure): right (0.6, direct)
• Bird et al. 2011 (code ownership): right (0.5, partial)
• McIntosh et al. 2014 (review coverage): right (0.4, partial)

Premise and arc

Nagappan et al. built defect predictors for 3,404 Windows Vista binaries. Eight organisational metrics, such as how many engineers and teams touched a binary, predicted failure-prone binaries with 86.2% precision and 84.0% recall, against 78.6 and 79.9 for churn, 79.3 and 66.0 for complexity, 74.4 and 69.9 for dependencies, 83.8 and 54.4 for coverage, and 73.8 and 62.9 for pre-release bugs. That is one system at one company. Bird et al. found that in Vista and Windows 7 the number of minor contributors to a binary correlated with failures more strongly than any other metric Microsoft collects. McIntosh et al. found the same ownership measures added explanatory power to every defect model in three open-source systems, Qt, VTK and ITK, while size mattered in two and complexity in none. These are predictions and correlations; they show where bugs concentrate, not that reorganising removes them.

hook

If you had to guess which part of a huge codebase would break next, you would look at the code.

conflict

Microsoft tested that instinct against a different signal: who had worked on each part.

evidence

The Vista prediction results side by side, the minor-contributor finding in Vista and Windows 7, and the open-source check in Qt, VTK and ITK.

resolution

Organisational measures predicted failure-prone parts better than code measures in Windows, and ownership mattered in open-source projects too; the evidence is strongest at one company and shows prediction, not cause.

takeaway

When choosing where to review and test hardest, look at code many people touch and few own.

Claims

• Organisational structure metrics predict defect-prone components better than code metrics

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Release freezes and change boards exist because shipping less often feels safer. Six years of DORA survey data checked whether speed and stability really trade off.

Tension: The intuition is a trade-off: every deploy is a chance to break production. The survey finds the fastest teams are also the most stable.

Other titles: Speed or Stability: Do You Really Have to Choose? · Why the Teams That Deploy Most Break the Least

Could chain with: decision:static-vs-dynamic-typing · decision:10x-developers-and-productivity

Readiness

Ready Coherent (0.75), central conclusion supported, 2 essential findings verified, no pending review.

Coherence

0.75 (review): The group mixed delivery outcomes with branching mechanics; pruned to one question, does shipping more often mean breaking more, with trunk-based development as the practice the same survey ties to both. The long-lived-branch claim rests on a 0.3 observation about open source that measures no merge conflicts, and the feature-flag claim is unchecked.

Why it ranks here

editorial 0.7 = mean of coherence 0.75 (review), relevance 0.68 (7 sources, 7 channels), curiosity 0.7 (decision, a study overturned the guess), stakes 0.7 (lean: pro), depth 0.68 (1 study, 2 claims, tension); evidence 0.5 = strongest DORA State of DevOps 2019 at 0.5 (direct) across 1 independent document; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Delivery speed and stability measured together across many teams, and the practices that go with high performance on both.

• DORA State of DevOps 2019: right (0.5, direct)
• DORA State of DevOps 2019: partly (0.4, direct)

Research gaps

• No study strong enough to decide: "Trunk-based development with one small change per release increases release frequency and makes rollbacks easier"

Premise and arc

DORA's 2019 State of DevOps report says that for six years running its research has found speed and stability enable each other: a cluster analysis of the four delivery measures (deployment frequency, lead time, change failure rate and time to restore) finds the highest performers significantly better on all four and the lowest performers worse on all four, rather than trading throughput for stability. The same report finds trunk-based development, with frequent check-in to trunk and deployment to production, predictive of those outcomes; it does not measure whether one small change per release makes rollbacks easier. This is a self-selected survey of practitioners analysed as correlations from one research programme: it shows the two go together in the teams surveyed, not that deploying more often by itself makes a team more stable.

hook

Your team ships once a month because every release is scary. Would shipping every day make it scarier?

conflict

Each deploy is a chance to break production, so fewer deploys should mean fewer failures. The survey data say otherwise.

evidence

DORA's cluster analysis of the four delivery measures, then its finding that trunk-based development with frequent deploys goes with high performance.

resolution

In the teams surveyed, the fastest are also the most stable; speed and stability move together, though the survey shows association, not that speed causes stability.

takeaway

Do not slow down releases to buy safety; small, frequent changes to trunk go with both speed and stability.

Claims

• Software delivery speed and stability move together rather than trading off; the best teams get both.
• Trunk-based development with one small change per release increases release frequency and makes rollbacks easier

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Every team argues about whether cleaning up code is worth the time. A study of 39 codebases put numbers on how much more trouble the worst files cause.

Tension: Clean-code advocates say messy code is where bugs and delays live; skeptics say it is aesthetics. The data back the defect half strongly and the time half more modestly.

Other titles: How Much Do Bad Files Actually Slow You Down? · Technical Debt, Measured

Then answers: Does the way a team is organised predict bugs better than the code itself?

Readiness

Ready Coherent (0.85), central conclusion qualified with accepted qualifications, 1 essential finding verified, no pending review.

Coherence

0.85 (review): The group mixed four questions (switch-versus-polymorphism speed, code health, modular design, rewrites); pruned to the only one a study checks, does unhealthy code cost more in defects and change time. The other claims are unchecked beliefs, two with wording flags.

Why it ranks here

editorial 0.63 = mean of coherence 0.85 (review), relevance 0.46 (3 sources, 3 channels), curiosity 0.5 (decision), stakes 0.7 (lean: pro), depth 0.62 (2 studies, 1 claim, tension); evidence 0.57 = strongest Tornhill & Borg 2022 (Code Red) at 0.5 (direct) across 2 independent documents; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Defect counts and task times for files graded by code quality across many codebases, and an independent check of the defect link.

• Tornhill & Borg 2022 (Code Red): partly (0.5, direct)
• Buse & Weimer 2010 (readability): partly (0.35, direct)

Premise and arc

Tornhill and Borg's 2022 study of 30,737 files in 39 proprietary codebases found that files graded Alert (low code health) had 15 times the reported defects of Healthy files, and tasks in them took 124% longer on average, about 2.2 times, so many times more defects holds but several times longer overstates the average time cost. It is observational, drawn from CodeScene customers and written by the vendor, and the authors leave the direction of cause open. Buse and Weimer's 2010 study of 15 open-source Java projects independently found that functions scored less readable were more likely to carry static-analysis warnings, change in the next release and be touched by bug fixes; it measures nothing about time.

hook

Your manager asks why the team needs a sprint to clean up code that already works.

conflict

Is messy code an engineering cost or a matter of taste?

evidence

Defects and task times by code-health grade across 39 codebases, then the readability study's link between hard-to-read functions and bug fixes.

resolution

The worst files carry many times the defects and take about twice as long to change; that is a correlation from vendor data, but an independent study points the same way on defects.

takeaway

Spend clean-up effort on the unhealthy files you change most often, where the defect and time cost concentrates.

Claims

• Code in unhealthy files has many times more defects and takes several times longer to change than healthy code

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Logging estimates against actuals is the standard advice for getting better at estimating. Studies of training programmes built on exactly that measured whether accuracy improved.

Tension: Feedback improves most skills, so comparing your guesses with outcomes should calibrate you. For time estimates, the programmes that did this found no gain in accuracy.

Other titles: Why Estimate-vs-Actual Reviews Do Not Fix Your Estimates · Can You Train Yourself to Estimate Better?

Part of: surprise:relative-size-estimation-more-accurate-time, as a follow-up question; not produced on its own.

Readiness

Ready Coherent (0.9), central conclusion qualified with accepted qualifications, 1 essential finding verified, no pending review.

Coherence

0.90 (review): One question: does recording estimates and comparing them with actuals make your future estimates more accurate?

Why it ranks here

editorial 0.63 = mean of coherence 0.9 (review), relevance 0.23 (1 source, 1 channel), curiosity 0.8 (surprise, a study overturned the guess), stakes 0.7 (lean: pro), depth 0.52 (2 studies, 1 claim, tension); evidence 0.57 = strongest Halkjelsvik & Jørgensen 2018 (Time Predictions) at 0.5 (direct) across 2 independent documents; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Evaluations of estimation training that uses outcome feedback, with untrained controls, and evidence on what feedback does improve.

• Halkjelsvik & Jørgensen 2018 (Time Predictions): wrong (0.5, direct)
• Jørgensen 2004 (expert estimation): partly (0.45, partial)

Research gaps

• Jørgensen 2004 (expert estimation) differs: The only software study found outcome feedback (estimate vs actual) did not improve accuracy, consistent with other domains, though it does improve assessment of estimate uncertainty; task-relation feedback is needed for accuracy gains.

Premise and arc

Halkjelsvik and Jørgensen's 2018 review reports that independent evaluations of the Personal Software Process, which has people record time predictions, compare them with actual time and analyse the deviations, found no improvement in prediction accuracy, and that a feedback-and-lessons-learned programme likewise did no better than untrained controls. Jørgensen's 2004 review finds the only software study of outcome feedback showed no accuracy gain either, consistent with other fields, though such feedback does improve how well people judge the uncertainty of their estimates, and feedback about how tasks relate is what is needed for accuracy. The same 2018 review notes that more experience doing a task generally does make predictions more accurate. So estimating can improve, but recording estimate against actual is not the route the evidence supports for accuracy.

hook

Every retrospective compares what you estimated with what it took. Is that making anyone better at estimating?

conflict

If estimation is a skill, feedback should train it; the training programmes built on that idea were measured.

evidence

The Personal Software Process evaluations, the lessons-learned programme against controls, and the 2004 review's finding on outcome feedback and uncertainty.

resolution

Comparing estimates with actuals did not improve accuracy in these studies, though it can sharpen how uncertain you say you are; experience with the task is what improved accuracy.

takeaway

Keep the log if it helps you set ranges, but do not expect it to make point estimates more accurate.

Claims

• Estimation is a learnable skill: recording estimates and comparing them to outcomes calibrates future estimates

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

The NoEstimates movement says estimating is wasted effort because estimates are always wrong. A review of time-prediction research measured how wrong they are, and where.

Tension: Software is known for overruns, so estimates look useless. The review finds them off by a fifth to a third on average, not biased overall, and too low mainly on larger projects.

Other titles: Are Software Estimates Really That Bad? · NoEstimates: Is Estimating a Waste of Time?

Part of: surprise:relative-size-estimation-more-accurate-time, as a follow-up question; not produced on its own.

Readiness

Ready Coherent (0.75), central conclusion qualified with accepted qualifications, 3 essential findings verified, no pending review.

Coherence

0.75 (review): Pruned to one question, are software estimates bad enough to stop making them: how far off they are, where they go wrong (larger work), and the NoEstimates claim that the time is wasted, which is folded in from its own candidate. The deadline-and-overtime claim is a different question (what pressure does to the work) and does not measure overtime; the fixed-budget-increment claim is unchecked.

Why it ranks here

editorial 0.67 = mean of coherence 0.75 (review), relevance 0.68 (7 sources, 7 channels), curiosity 0.7 (decision, a study overturned the guess), stakes 0.5 (lean: skeptic), depth 0.73 (1 study, 3 claims, tension); evidence 0.5 = strongest Halkjelsvik & Jørgensen 2018 (Time Predictions) at 0.5 (direct) across 1 independent document; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Measured accuracy and bias of time predictions in software, split by task size, and evidence on what replaces an estimate if you stop.

• Halkjelsvik & Jørgensen 2018 (Time Predictions): partly (0.5, direct)
• Halkjelsvik & Jørgensen 2018 (Time Predictions): partly (0.5, partial)
• Halkjelsvik & Jørgensen 2018 (Time Predictions): wrong (0.4, direct)

Premise and arc

Halkjelsvik and Jørgensen's 2018 review of time-prediction research finds the average error of time predictions, across domains, is around 20 to 30%. In software it did not find a general tendency to underestimate: the median bias is dominated by many small tasks that were predicted without bias or even too pessimistically. Only for projects of at least 100 work hours does the expected overrun appear, with a median of about 20%, so the claim that most projects run late holds for larger projects, not for software work in general. That is a review of studies, not one measurement, and it gives the median and average, not how often individual projects are badly over. On NoEstimates, the authors call the suggestion that people stop giving time predictions before a project starts unrealistic, while agreeing that extrapolating from early progress is a good method; they note it is itself a simple prediction model, and few models beat it. That is the reviewers' judgement, not a test of teams that stopped estimating.

hook

Your estimates were wrong again. Some teams have decided to stop making them altogether.

conflict

If estimates are always wrong, the time spent on them is wasted; if they are roughly right, dropping them throws away a useful tool.

evidence

The review's 20 to 30% average error, the missing overall bias with small tasks, the 20% median overrun on larger projects, and what the reviewers say about NoEstimates.

resolution

Estimates are imperfect rather than useless: roughly a fifth to a third off, unbiased on small tasks and too low on large ones. Measuring early progress and extrapolating, the method NoEstimates favours, is itself a good estimate.

takeaway

Keep estimating, but break large work into smaller pieces and update the forecast from how fast the first pieces actually went.

Claims

• Most software projects run late or over budget, with few finishing early.
• Long-horizon task estimates are inaccurate and managers should not rely on them
• Time spent estimating is wasted and better spent building

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Companies tear down walls to get people collaborating, and developers pay for it in focus. Two headquarters measured with sensor badges what actually happened to face-to-face contact.

Tension: The open office is sold as a trade: less quiet, more collaboration. The measurements suggest you can lose the collaboration as well.

Other titles: What Happened When Two Companies Tore Down Their Cubicles · The Open Office Was Meant to Bring People Together. Did It?

Could chain with: decision:10x-developers-and-productivity · decision:static-vs-dynamic-typing

Readiness

Ready Coherent (0.8), central conclusion supported, 2 essential findings verified, no pending review.

Coherence

0.80 (review): The group mixed three questions (crunch hours, meetings and focus blocks, open offices); pruned to the one with a direct study, do open-plan offices get people talking. Crunch and meeting claims rest on qualified evidence below 0.5 and the sleep belief has a wording flag.

Why it ranks here

editorial 0.53 = mean of coherence 0.8 (review), relevance 0.46 (3 sources, 3 channels), curiosity 0.5 (decision), stakes 0.5 (lean: skeptic), depth 0.38 (1 study, 2 claims); evidence 0.65 = strongest Bernstein & Turban 2018 (open offices) at 0.65 (direct) across 1 independent document; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Measured face-to-face and electronic interaction before and after a move to open plan, and evidence on distraction and focus.

• Bernstein & Turban 2018 (open offices): right (0.65, direct)
• Bernstein & Turban 2018 (open offices): partly (0.2, direct)

Research gaps

• No study strong enough to decide: "Open-plan offices increase noise and visual distraction that impairs focused work"

Premise and arc

Bernstein and Turban followed two Fortune 500 headquarters that moved from cubicles to open floors. Sociometric badges and server logs showed face-to-face time fell 72% at the first firm (from about 5.8 to 1.7 hours per person per day) and 67 to 71% at the second, while email sent rose 56% and instant messages 67%. It is a quasi-experiment without a control group, with 52 and 100 participants, in corporate offices rather than software teams. The noise and distraction complaint is not measured there: the paper cites earlier workplace research, mixed in its findings, that open offices reduce focus and privacy, and notes employees isolating themselves with large headphones, so that part is second-hand.

hook

The walls come down so the team will talk more. Then everyone puts on headphones.

conflict

Open plan is supposed to trade quiet for collaboration. Does the collaboration actually happen?

evidence

Badge data from the two headquarters before and after the move, the rise in email and messaging, and what the paper says about focus.

resolution

At both firms face-to-face contact fell by about two thirds or more and electronic messages rose; the cost to focus is widely reported but was not measured here.

takeaway

Do not expect an open floor to create conversation; if you want people to talk, design for that directly.

Claims

• Moving to open-plan offices reduces face-to-face interaction and shifts communication to email and messaging
• Open-plan offices increase noise and visual distraction that impairs focused work

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Keep your PRs small is the most repeated code-review advice. The data show part of it and leave the rest unmeasured.

Tension: Small changes are said to be reviewed faster, reviewed better and to break less; the evidence here shows the first only partly and the others indirectly.

Other titles: How Big Should a Pull Request Be? · What Google's Review Data Say About Change Size

What blocks it

• Reviewer: Sadowski et al. show only that small changes get first feedback sooner at Google, not that they merge faster; McIntosh et al. measure review rate, not review size; DORA ties batch size to failures only through approvals. Unblock with review-to-merge latency broken down by change size (for example Rigby and Bird's cross-project review data), a measured link between lines under review and defects found (for example the Cisco review study), and change size against change failure or restore time.

Readiness

Needs research Essential evidence is missing: Reviewer: Sadowski et al. show only that small changes get first feedback sooner at Google, not that they merge faster; McIntosh et al. measure review rate, not review size; DORA ties batch size to failures only through approvals. Unblock with review-to-merge latency broken down by change size (for example Rigby and Bird's cross-project review data), a measured link between lines under review and defects found (for example the Cisco review study), and change size against change failure or restore time.

Coherence

0.75 (review): The group mixed pairing and mobbing beliefs with change size; pruned to one question, should you keep changes small for review. The pairing and mob claims are unchecked beliefs and the pair-review claim rests on a 0.25 recommendation.

Why it ranks here

editorial 0.75 = mean of coherence 0.75 (review), relevance 0.79 (10 sources, 10 channels), curiosity 0.5 (decision), stakes 0.7 (lean: pro), depth 1 (3 studies, 3 claims, tension); evidence 0.33 = strongest Sadowski et al. 2018 (Google code review) at 0.55 (partial) across 1 independent document; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Review-to-merge time by change size, defect-finding by review size, and incident rates by change size.

• Sadowski et al. 2018 (Google code review): right (0.55, partial)
• McIntosh et al. 2014 (review coverage): partly (0.35, direct)
• Sadowski et al. 2018 (Google code review): partly (0.3, direct)
• DORA State of DevOps 2019: partly (0.4, partial)

Research gaps

• Reviewer: Sadowski et al. show only that small changes get first feedback sooner at Google, not that they merge faster; McIntosh et al. measure review rate, not review size; DORA ties batch size to failures only through approvals. Unblock with review-to-merge latency broken down by change size (for example Rigby and Bird's cross-project review data), a measured link between lines under review and defects found (for example the Cisco review study), and change size against change failure or restore time.
• No study strong enough to decide: "Reviewer defect-finding effectiveness drops sharply once a single review exceeds a few hundred lines"
• No study strong enough to decide: "Smaller changes mean less rework, fewer incidents and faster restoration"

Premise and arc

6 claims bear on this choice; studies decide 1 of them. Walk the viewer from the claims they have heard to the decision the evidence supports.

hook

The decision a viewer faces, in their terms.

conflict

The claims pulling in different directions: "Pairing or mobbing puts the best of each participant into the code and rarely the worst"; "Mob programming onboards new members quickly and spreads knowledge so the team does not depend on one person"; "Pair programming provides continuous code review without adding a separate review step that slows delivery"; "Small pull requests are picked up and merged faster"; "Reviewer defect-finding effectiveness drops sharply once a single review exceeds a few hundred lines"; "Smaller changes mean less rework, fewer incidents and faster restoration".

evidence

Each claim's verdict, strongest study first.

resolution

What the evidence settles and what it leaves open.

takeaway

A decision rule the viewer can apply.

Claims

• Small pull requests are picked up and merged faster
• Reviewer defect-finding effectiveness drops sharply once a single review exceeds a few hundred lines
• Smaller changes mean less rework, fewer incidents and faster restoration

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

The choice shapes a system for years and is usually made on conference talks. The evidence on outcomes is much thinner than the advice.

Tension: Microservices promise team independence; critics say they cost more and the boundaries are wrong at first. The studies here catalogue pains but do not measure outcomes.

Other titles: Do Microservices Pay Off for a New Project? · Why So Many Teams Regret Starting with Microservices

What blocks it

• Reviewer: The only sources are a review of practitioner reports (Soldani 2018) that lists pains without measuring cost or outcomes, a DORA footnote advising against premature decomposition with no data, and coordination findings (Bird 2011, Halkjelsvik 2018) that never test service boundaries. Unblock with a study that measures cost, delivery performance or failures for comparable systems built as microservices and as monoliths, or tracks greenfield microservice projects against monolith-first ones.

Readiness

Needs research Essential evidence is missing: Reviewer: The only sources are a review of practitioner reports (Soldani 2018) that lists pains without measuring cost or outcomes, a DORA footnote advising against premature decomposition with no data, and coordination findings (Bird 2011, Halkjelsvik 2018) that never test service boundaries. Unblock with a study that measures cost, delivery performance or failures for comparable systems built as microservices and as monoliths, or tracks greenfield microservice projects against monolith-first ones.

Coherence

0.70 (review): One question, should a new system start as microservices or a monolith: cost, coordination, boundary mistakes and the modular-monolith alternative. The hype claim is a motive, not part of the answer, so it is pruned.

Why it ranks here

editorial 0.75 = mean of coherence 0.7 (review), relevance 0.87 (13 sources, 9 channels), curiosity 0.5 (decision), stakes 0.7 (lean: pro), depth 1 (4 studies, 4 claims, tension); evidence 0.3 = strongest Soldani et al. 2018 (microservices review) at 0.3 (direct) across 1 independent document; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Measured cost, delivery or failure outcomes for comparable systems built as microservices and as monoliths, ideally greenfield against monolith-first.

• Soldani et al. 2018 (microservices review): partly (0.3, direct)
• Halkjelsvik & Jørgensen 2018 (Time Predictions): partly (0.4, direct)
• Bird et al. 2011 (code ownership): partly (0.3, direct)
• Soldani et al. 2018 (microservices review): partly (0.35, direct)
• DORA State of DevOps 2019: right (0.2, partial)

Research gaps

• Reviewer: The only sources are a review of practitioner reports (Soldani 2018) that lists pains without measuring cost or outcomes, a DORA footnote advising against premature decomposition with no data, and coordination findings (Bird 2011, Halkjelsvik 2018) that never test service boundaries. Unblock with a study that measures cost, delivery performance or failures for comparable systems built as microservices and as monoliths, or tracks greenfield microservice projects against monolith-first ones.
• No study strong enough to decide: "Microservices cost more in the short term and rarely save money"
• No study strong enough to decide: "Coordination cost grows faster than linearly with team count on a shared codebase, which service boundaries contain"
• Unchecked: "Keeping modules in one process with enforced boundaries gets most of the modularity benefit without the distribution cost"
• No study strong enough to decide: "Service boundaries are usually wrong on the first attempt because the domain is not yet understood, so systems built as microservices from scratch tend to end in trouble"

Premise and arc

5 claims bear on this choice; studies decide 0 of them. Walk the viewer from the claims they have heard to the decision the evidence supports.

hook

The decision a viewer faces, in their terms.

conflict

The claims pulling in different directions: "Microservices cost more in the short term and rarely save money"; "Hype drives teams to adopt microservices they do not need, adding cost and risk"; "Coordination cost grows faster than linearly with team count on a shared codebase, which service boundaries contain"; "Keeping modules in one process with enforced boundaries gets most of the modularity benefit without the distribution cost"; "Service boundaries are usually wrong on the first attempt because the domain is not yet understood, so systems built as microservices from scratch tend to end in trouble".

evidence

Each claim's verdict, strongest study first.

resolution

What the evidence settles and what it leaves open.

takeaway

A decision rule the viewer can apply.

Claims

• Microservices cost more in the short term and rarely save money
• Coordination cost grows faster than linearly with team count on a shared codebase, which service boundaries contain
• Keeping modules in one process with enforced boundaries gets most of the modularity benefit without the distribution cost
• Service boundaries are usually wrong on the first attempt because the domain is not yet understood, so systems built as microservices from scratch tend to end in trouble

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Teams switch to story points because hours estimates keep failing. The research on relative estimation says whether points fix the accuracy problem.

Tension: Points are pitched as the honest way to estimate uncertain work. The studies find no general accuracy gain, and a specific bias when tasks differ a lot in size.

Other titles: Story Points vs Hours: Which Lands Closer? · Does Estimating in Points Make You More Accurate?

Then answers: Does tracking your estimates against actuals make you a better estimator? → Are software estimates so unreliable that you should stop making them?

Readiness

Ready Coherent (0.9), central conclusion supported, 1 essential finding verified, no pending review.

Coherence

0.90 (review): One question: are relative-size estimates such as story points more accurate than estimates in hours?

Why it ranks here

editorial 0.6 = mean of coherence 0.9 (review), relevance 0.23 (1 source, 1 channel), curiosity 0.8 (surprise, a study overturned the guess), stakes 0.7 (lean: pro), depth 0.38 (1 study, 1 claim, tension); evidence 0.5 = strongest Halkjelsvik & Jørgensen 2018 (Time Predictions) at 0.5 (direct) across 1 independent document; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Comparisons of relative and absolute time predictions on accuracy, and evidence on when relative estimates go wrong.

• Halkjelsvik & Jørgensen 2018 (Time Predictions): wrong (0.5, direct)

Premise and arc

Halkjelsvik and Jørgensen's 2018 review of time-prediction research finds few studies comparing relative and absolute methods: relative predictions improved accuracy in some contexts and worsened it in others, and in general they do not seem more accurate. When tasks differ a lot in size, people judge them more similar than they are, so a task ten times larger than the reference tends to be underestimated more with relative methods. Relative methods may still simplify estimating and reduce the need to know who will do the work. The evidence base is thin and the review does not single out complex or uncertain work, so the honest statement is that the evidence does not show points to be more accurate, not that they are worse.

hook

Your team stopped estimating in hours because the hours were always wrong. Are the points any better?

conflict

Relative sizing is supposed to suit uncertain work; does it actually land closer to reality?

evidence

The review's summary of the relative-versus-absolute studies, and the size-assimilation bias it describes.

resolution

Relative estimates are not generally more accurate than hours, and they skew when a task is much bigger or smaller than the reference.

takeaway

Use points for convenience if you like, but do not expect them to fix accuracy, and compare tasks of similar size.

Claims

• Relative-size estimation is more accurate than time estimates for complex or uncertain work

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Teams moving to continuous delivery are told review gates will become the bottleneck. One company runs mandatory review on every change at enormous scale.

Tension: Critics say per-change gates cannot survive many deploys a day; Google does it, but Google is not a typical team.

Other titles: Does Mandatory Review Slow Down Fast-Shipping Teams? · How Google Reviews 20,000 Changes a Day

What blocks it

• Reviewer: The only counter-evidence is Sadowski et al.'s description of Google (0.4: one highly tuned company and its own tooling, not a pull-request workflow, with no throughput comparison), and DORA 2019 says per-change approval breaks down only for external boards, not teammate review. Unblock with a study measuring lead time or deployment frequency against mandatory per-change peer review across several continuous-delivery organisations, for example DORA data split by review practice.

Readiness

Needs research Essential evidence is missing: Reviewer: The only counter-evidence is Sadowski et al.'s description of Google (0.4: one highly tuned company and its own tooling, not a pull-request workflow, with no throughput comparison), and DORA 2019 says per-change approval breaks down only for external boards, not teammate review. Unblock with a study measuring lead time or deployment frequency against mandatory per-change peer review across several continuous-delivery organisations, for example DORA data split by review practice.

Coherence

0.80 (review): One question: can mandatory review of every change keep up with continuous delivery? Distinct from decision:pairing-and-code-review, which asks about change size.

Why it ranks here

editorial 0.57 = mean of coherence 0.8 (review), relevance 0.23 (1 source, 1 channel), curiosity 0.8 (surprise, a study overturned the guess), stakes 0.5 (lean: skeptic), depth 0.52 (2 studies, 1 claim, tension); evidence 0.46 = strongest Sadowski et al. 2018 (Google code review) at 0.4 (direct) across 2 independent documents; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Delivery throughput or lead time measured against per-change peer review in continuous-delivery teams, beyond one company.

• Sadowski et al. 2018 (Google code review): wrong (0.4, direct)
• DORA State of DevOps 2019: partly (0.3, direct)

Research gaps

• Reviewer: The only counter-evidence is Sadowski et al.'s description of Google (0.4: one highly tuned company and its own tooling, not a pull-request workflow, with no throughput comparison), and DORA 2019 says per-change approval breaks down only for external boards, not teammate review. Unblock with a study measuring lead time or deployment frequency against mandatory per-change peer review across several continuous-delivery organisations, for example DORA data split by review practice.
• DORA State of DevOps 2019 differs: DORA says approving each individual change by an external board is impossible in the continuous paradigm, but it recommends shifting to per-change peer review by a teammate during development, so the breakdown applies to external gates, not to peer review itself.

Premise and arc

Before checking, the best guess was "partly": Google and others sustain high deploy rates with change-based review; the claim holds only when review latency is high. Sadowski et al. 2018 (Google code review): More than 25,000 Google developers put about 20,000 changes a day through mandatory per-change review into one trunk, with median latency under 4 hours and developers confirming the process works well at scale; one highly tuned company, not a GitHub pull-request setup. 1 source take a position on it.

hook

State the expectation everyone shares, with the sources that voice it.

conflict

A study checked it and found something else.

evidence

Sadowski et al. 2018 (Google code review): More than 25,000 Google developers put about 20,000 changes a day through mandatory per-change review into one trunk, with median latency under 4 hours and developers confirming the process works well at scale; one highly tuned company, not a GitHub pull-request setup.

resolution

Why the expectation was wrong, or what the sources were missing.

takeaway

The corrected belief, with its limits.

Claims

• Per-change pull request gates break down under continuous delivery with many deployments a day

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Six sources say pairing and mobbing put each person's best into the code and filter out the worst, and teams pay for that belief in developer hours; nothing in the corpus measures it.

Tension: Advocates describe fewer mistakes and shared knowledge from experience; a pair costs two people's time, and whether the code is that much better is the open question.

Other titles: Pairing and Mobbing: Better Code or Just More People? · Two Heads at One Keyboard: Does It Pay?

What blocks it

• No study in the corpus checks the claim; the story is the search for evidence.

Readiness

Needs research Essential evidence is missing: No study in the corpus checks the claim; the story is the search for evidence.

Coherence

0.80 (review): One question: does code written by a pair or a mob take the best of each person? It was pruned from decision:pairing-and-code-review as unchecked and is its own question, about collaboration rather than change size.

Why it ranks here

editorial 0.6 = mean of coherence 0.8 (review), relevance 0.64 (6 sources, 4 channels), curiosity 0.5 (gap), stakes 0.7 (lean: pro), depth 0.35 (0 studies, 1 claim); evidence 0 = no decisive evidence; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Controlled comparisons of pairs or mobs against individuals on the same tasks, measuring defects, design quality and total effort.

• No decisive study yet.

Research gaps

• No study in the corpus checks the claim; the story is the search for evidence.
• Has anyone measured whether pairing or mobbing puts the best of each participant into the code and rarely the worst?
• Which study or dataset would settle it?

Premise and arc

6 sources assert "Pairing or mobbing puts the best of each participant into the code and rarely the worst", and no study in the corpus bears on it. The story is the search for evidence, not a verdict.

hook

How often this gets said, and how confidently.

conflict

Ask for the evidence and find none in the corpus.

evidence

What the sources offer instead (anecdote, authority, mechanism) and what a real test would need.

resolution

Absence of evidence is not evidence of absence: what would change the picture.

takeaway

How to treat the claim until it is checked.

Claims

• Pairing or mobbing puts the best of each participant into the code and rarely the worst

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.

Six sources defend TDD less for bug counts than for design, saying test-first forces decoupled, testable code; the TDD experiments in the corpus measured other outcomes.

Tension: If the order of tests does not change quality, as the experiments found, the design argument is the one left; nobody here has measured it.

Other titles: TDD and Code Design: Does Testability Come for Free? · Is Decoupled Code a Side Effect of Writing Tests First?

What blocks it

• No study in the corpus checks the claim; the story is the search for evidence.

Readiness

Needs research Essential evidence is missing: No study in the corpus checks the claim; the story is the search for evidence.

Coherence

0.80 (review): One question: does writing the test first produce more decoupled, testable design? Distinct from surprise:test-driven-development-produces-better-quality, whose studies measure external quality and productivity, not design structure.

Why it ranks here

editorial 0.6 = mean of coherence 0.8 (review), relevance 0.64 (6 sources, 6 channels), curiosity 0.5 (gap), stakes 0.7 (lean: pro), depth 0.35 (0 studies, 1 claim); evidence 0 = no decisive evidence; overall = 0.6 editorial + 0.4 evidence, scaled by originality. Heuristics from corpus counts and the audit, not audience data.

Evidence

Needed: Studies comparing coupling, cohesion or other internal design metrics of code written test-first against test-last, on comparable tasks.

• No decisive study yet.

Research gaps

• No study in the corpus checks the claim; the story is the search for evidence.
• Has anyone measured whether writing tests first forces the code to be decoupled and testable by construction?
• Which study or dataset would settle it?

Premise and arc

6 sources assert "Writing tests first forces the code to be decoupled and testable by construction", and no study in the corpus bears on it. The story is the search for evidence, not a verdict.

hook

How often this gets said, and how confidently.

conflict

Ask for the evidence and find none in the corpus.

evidence

What the sources offer instead (anecdote, authority, mechanism) and what a real test would need.

resolution

Absence of evidence is not evidence of absence: what would change the picture.

takeaway

How to treat the claim until it is checked.

Claims

• Writing tests first forces the code to be decoupled and testable by construction

Evidence audit

Status, then the publication decision: approved as written, qualify (needs the stated limitation), or blocked. "Review" means a person must confirm the passages support the exact wording. Each study verdict is marked direct, partial or contextual for this sentence; only direct and partial ones decide.