ali's garden

ai-assisted development frameworks: the ladder

Properties1
tagslearning, ai, software-engineering, harness-engineering

9 min read

Learning in progress

Part of the harness. Working notes, bullet structure intentional.

The cleaner frame: this reads as a ladder, but each philosophy is really a bet on one harness component - guides, sensors, or loop. See the map. The rung-by-rung detail below is the same set, laid out as a spectrum.

Research digest, July 2026. No single standard exists yet. Frameworks below are ordered by increasing structure and decreasing human implementation control.

The core spectrum (one line each)

  • Vibe coding: “make something like this”, inspect the product, not the code
  • Prompt-driven: “build this feature with these instructions”
  • Pseudocode-driven: “here is my algorithm, implement it”
  • Spec-driven (SDD): “here are requirements, design, acceptance criteria”
  • Test/contract-driven: “here are the properties that must hold”
  • Harness engineering: “here is the controlled environment the agent runs in”
  • Formal specification: “here is a mathematically verifiable correctness definition”

grouped by harness component

Each philosophy mainly feeds one part of the harness:

  • Guides (feed-forward): pseudocode, SPDD, spec-driven, memory / rules, and the spec half of formal
  • Sensors (feedback): test/contract-driven, evals, the proof half of formal
  • Loop (orchestration): human-on-the-loop, multi-agent
  • No harness: vibe coding, prompt-driven (judge by output)

The hybrid this digest lands on spans two components: pseudocode + light spec (guides) and tests (sensors). The rung-by-rung detail below is that same set as a spectrum.

1. Software 3.0 (Karpathy)

2. Vibe coding

  • Surrender implementation control, judge by “does it seem to work”
  • Good: prototypes, throwaway tools, UI experiments, learning
  • Bad: production, sensitive data, pipelines, anything maintained
  • Original definition (Willison): NOT all AI-assisted coding
  • Source: Willison, Not all AI-assisted programming is vibe coding
  • See it in practice:

3. Pseudocode-driven

  • Human owns the algorithm, AI translates to Spark/SQL/Python
  • No settled name: intent-driven, algorithm-first, implementation delegation
  • Weakness: pseudocode omits edge cases, architecture, correctness criteria
  • See it in practice (thin tooling - this rung has no serious canonical tool yet):

4. Structured prompt-driven (SPDD, Thoughtworks)

  • Prompts become version-controlled artifacts, not chat messages
  • Contains: business intent, context, constraints, review checkpoints
  • Flow: requirements → clarification → context → prompt → code → tests
  • Fits teams that want AI without restructuring their process
  • Source: Fowler/Thoughtworks, SPDD
  • See it in practice:
    • open-spdd - CLI that manages SPDD prompt/command templates across Cursor, Claude Code, Copilot
    • token-billing - the article’s companion repo: SPDD applied to a real Spring Boot feature

5. Spec-driven development (SDD)

  • Spec is source of truth; code is one generated realization
  • Spec contains: purpose, scenarios, requirements, I/O, rules, architecture, acceptance criteria
  • Spec Kit (GitHub): Spec → Plan → Tasks → Implement, markdown artifacts
  • Kiro (AWS): project files (structure/tech/product) + feature files (requirements/design/tasks)
  • OpenSpec: lighter, incremental, existing-codebase friendly
  • vs pseudocode: “what the system must do” vs “how I would solve it”
  • Sources: Fowler, Understanding SDD tools | Spec Kit | Kiro deep dive | OpenSpec
  • See it in practice (each ships runnable example specs):
    • GitHub Spec Kit - open-source SDD toolkit/CLI; works with Claude Code, Copilot, Gemini, Cursor
    • AWS Kiro - spec-first agentic IDE: prompts → requirements, design, sequenced tasks
    • OpenSpec - lightweight delta-spec framework; agree on the spec before code

6. Test/contract-driven agentic

  • Human specifies observable correctness, not implementation
  • Agent loop: write tests → implement → run → inspect failures → revise
  • Builds on TDD, BDD, design-by-contract, property-based testing
  • Key line: pseudocode = how you imagine solving it; tests = what must remain true
  • See it in practice:

7. Harness engineering

8. Human on the loop

9. Multi-agent orchestration

  • Separate agents: planning, architecture, implementation, testing, review, security
  • Example: OpenAI Symphony (project board as control plane, task-per-agent)
  • Human role shifts to: product owner + architect + eng manager + reviewer
  • Source: OpenAI Symphony
  • See it in practice:

10. NL → formal specification

  • Natural language → formal spec (e.g. Dafny) → verified implementation
  • Can prove invariants, forbidden states, spec conformance
  • Unsolved: ambiguous prose → correct formal requirements
  • Source: arXiv 2506.11874, Formalising requirements with LLMs
  • See it in practice:
    • Dafny - verification-aware language: write pre/post-conditions and invariants, the verifier proves them
    • Dafny (repo) - source, reference manual, and tutorials

Where this is heading

  • Not toward casual pseudocode for everyone
  • Toward authoring higher-level artifacts:
    • intent + specs + examples + constraints + tests + architecture decisions + feedback loops
  • AI compiles those artifacts into code

Practical working loop (lightweight hybrid: 3 + 5 + 6)

  1. Write the objective
  2. Define inputs and outputs
  3. Pseudocode the core logic
  4. List edge cases and leakage risks
  5. Define acceptance tests
  6. Ask AI for an implementation plan
  7. Review plan BEFORE code generation
  8. Generate code in small pieces
  9. Run tests, inspect key decisions

Synthesis notes

  • Central axis: WHO owns the reasoning at each rung
  • Pseudocode and tests are complements:
    • pseudocode protects algorithm ownership (skill retention)
    • tests protect correctness (safety net when AI implements)
  • SDD tooling war is unresolved; artifacts matter more than the tool
  • Harness engineering is the highest-leverage rung for production DS/ML work:
    • pipeline debugging history (silent poison caches, case-mismatch bugs) is exactly what harness feedback controls catch
  • The hypothesis-first / bounded-scope AI protocol already implements rungs 3 to 5
  • Plan-before-code discipline = step 7 of the working loop, already in practice
  • Gap to close: acceptance tests (step 5) rarely defined before implementation in current workflow
  • Formal specification is a research horizon, not actionable yet

Two-dimensional scoring

The three views below are interactive. In the radar, toggle any framework on or off to compare shapes. The proposed hybrid (violet, dashed) - pseudocode + acceptance tests + light spec - is plotted on all three as the target.

Framing A: process axes

  • X = human reasoning ownership (who designs the logic)
  • Y = verification rigor (how failures get caught automatically)
FrameworkOwnershipVerificationCluster
Vibe21Faith
Prompt-driven32Faith
Pseudocode83Craft
SPDD75Artifact
Spec-driven66Artifact
On the loop46Systems
Multi-agent37Systems
Test-driven58Systems
Harness59Systems
Formal spec710Rigor outlier
Proposed hybrid88Target
  • Faith cluster (low, low): fine for throwaways only
  • Craft cluster (high ownership, low verification): fragile-expert risk zone
  • Artifact cluster (balanced, paperwork-heavy)
  • Systems cluster (high verification, ownership drifts down)
  • Empty top-right quadrant (8+, 8+) is the real target; no existing framework occupies it
  • Cheapest approximation: pseudocode + acceptance tests → lands the hybrid at (8, 8)

Framing B: outcome axes (good-software attributes, ISO 25010 style)

  • X = correctness assurance (reliability, defect prevention)
  • Y = maintainability and comprehension (the cognitive-debt axis)
FrameworkCorrectnessMaintainabilityZone
Vibe21Poor software
Prompt-driven33Poor software
Multi-agent63Fragile-correct
On the loop64Fragile-correct
Formal spec105Fragile-correct
Test-driven85Balanced
SPDD66Balanced
Harness86Balanced
Spec-driven77Balanced
Pseudocode58Comprehension-first
Proposed hybrid (target)88Target
  • Key reversal from Framing A: multi-agent drops hard on maintainability (nobody comprehends the whole system)
  • Pseudocode rises: human owns the mental model, matching the scaffolded-group advantage in cognitive-debt research (39% vs 77% maintenance failure)
  • Formal spec is correct but hard to read: high X, mid Y
  • SDD scores best of the existing frameworks (7, 7): spec doubles as living documentation
  • Same conclusion from both framings: hybrid pseudocode + tests + light spec dominates any single philosophy
  • Personal position estimate: near (6, 8) today; adding pre-implementation acceptance tests moves toward (8, 8)

Framing C: six good-software dimensions (radar)

  • Axes: correctness, maintainability, velocity, skill retention, team scalability, ease of adoption
  • Vibe is a spike (velocity + ease), collapses on everything else
  • Pseudocode alone peaks on skill retention; weak on team scalability
  • Harness is the widest single framework but ease of adoption is its Achilles heel
  • No single framework fills the hexagon
  • The proposed hybrid (pseudocode + test-driven + light spec) covers nearly the full shape - that union is the practical working loop above

Radar values in Framing C are estimates (no source numbers exist for that view). The scatter coordinates in A and B are from the tables above.

Full source list

Graph View

Backlinks