# Do the Unvibed repairs look better?

Across 18 live command comparisons, both presentation orders preferred the kit on **13** and plain on **1**. **4** were tied or unresolved. The mean preference was **5.11/7**, where 4 is even.

## By command

| Command or screen | Pairs | Wins | Tied or unresolved | Losses | Mean preference |
| --- | ---: | ---: | ---: | ---: | ---: |
| /landing | 2 | 2 | 0 | 0 | 6.25 |
| /polish | 2 | 1 | 1 | 0 | 4.75 |
| /harden | 2 | 2 | 0 | 0 | 5.75 |
| /ship | 2 | 1 | 1 | 0 | 5 |
| /audit | 2 | 2 | 0 | 0 | 5.25 |
| /motion | 2 | 2 | 0 | 0 | 5.5 |
| /fix | 3 | 1 | 1 | 1 | 3.83 |
| /design-system | 3 | 2 | 1 | 0 | 5.17 |

## Where those results came from

The starting set contains all eight original before-demo routes and seven new synthetic screens: pricing, onboarding, an empty inbox, settings with a failed save, expired access, a dispatch table and a mobile feed. The demo inputs are captured DOM and CSS from those routes. Three additional assignments exercise /design-system on three reused starters.

Both arms use codex / gpt-6.1-sol and receive the same user task, records, screen source and output constraints. The treatment also receives the named procedure. On audit and ship tasks, both arms first write a review; a separate implementation call uses that review to produce a comparable screen. This measures the visible consequence of the review. Individual vendor trials reuse the same plain response as the matched task. Each skill receives one preassigned task and its pinned main instructions, with applicable local reference text where it fits the input budget.

Local Chrome renders each candidate at 1440 × 900 and 390 × 844. The vision judge sees the paired desktop images side by side and the paired phone images side by side, with neutral labels. Two independent calls reverse the left and right positions. The judge never sees skill names, candidate source or detector scores. Scores below are oriented toward the treatment: 1 strongly favors plain, 4 is even, and 7 strongly favors the procedure. A win or loss requires both orderings to agree on the direction. Ties and other unresolved comparisons are counted together.

The judge scores product-specificity, hierarchy, typography, spacing, color, copy, visible states and mobile layout. A separate deterministic detector counts observable generic patterns on a 0 to 100 scale, higher means more signals. It can mistake a deliberate design for a generic one. Visible states receive no score where evidence is insufficient.

| Command or screen | Pairs | Wins | Tied or unresolved | Losses | Mean preference |
| --- | ---: | ---: | ---: | ---: | ---: |
| existing demo | 8 | 7 | 1 | 0 | 5.56 |
| new screen | 7 | 4 | 2 | 1 | 4.57 |
| design-system | 3 | 2 | 1 | 0 | 5.17 |

## Visual dimensions

| Dimension | Mean kit-oriented preference |
| --- | ---: |
| Product specificity and generic signals | 4.39 |
| Visual hierarchy | 5.11 |
| Type scale | 4.36 |
| Spacing rhythm | 4.53 |
| Color intent | 4.11 |
| Copy specificity | 4.86 |
| Visible states | 4.82 |
| Mobile layout | 4.94 |

The deterministic detector averaged 8.17/100 on plain screens and 10.28/100 on kit screens. Higher scores mean more matched patterns. The judge did not see these scores.

## Procedure revision

The first pass used the original procedures on the 15 core assignments. We inspected the eight existing demo cases, retained every initial result and revised the procedures to keep the user’s deciding fact and its action visible together on both widths. The seven new screen cases were not used to select those changes. All changed kit arms were regenerated against their original plain controls. The extra design-system cases are reported separately.

| Command or screen | Pairs | Wins | Tied or unresolved | Losses | Mean preference |
| --- | ---: | ---: | ---: | ---: | ---: |
| Initial command procedures | 15 | 2 | 5 | 8 | 3.3 |
| Revised procedures, same 15 cases | 15 | 11 | 3 | 1 | 5.1 |

The first revision on the 15 core cases recorded 11 wins, 3 unresolved and 1 loss. The later extraction change also affects /fix, so we regenerated its dependent cases for the final table.

The initial extraction procedure lost all three design-system visual comparisons (mean 2.17/7). We inspected those cases, tightened how it chooses roles for a real decision and reran all three. These are development cases. The original ballots remain in [the extraction revision record](https://unvibed.app/bench/design-initial.json).

## First confirmation cohort

Three new domains, briefs and record sets were reserved until the preceding procedures were locked. Their arms and procedures are frozen to that revision. They are outside the 18 comparisons above. Their plain controls were generated independently. These three comparisons offer a small check on transfer to new inputs.

| Command or screen | Pairs | Wins | Tied or unresolved | Losses | Mean preference |
| --- | ---: | ---: | ---: | ---: | ---: |
| Fresh confirmation | 3 | 2 | 1 | 0 | 5.17 |

**/landing, confirm-price.** win. The two treatment-oriented scores were 6 and 6.

**/harden, confirm-form.** Tied or unresolved. The two treatment-oriented scores were 2 and 5.

**/fix, confirm-sheet.** win. The two treatment-oriented scores were 6 and 6.

## Final confirmation cohort

The final change used only the existing demo evidence to address generic composition, color use and typography in /fix and /design-system. The theatre, ceramic and harbor cases were registered before that change. Neither arm was generated before the final procedures were frozen. They are separate from both earlier cohorts. [Preregistration](https://unvibed.app/bench/final-confirmation-lock.json), [pre-final results](https://unvibed.app/bench/previous-run.json).

| Command or screen | Pairs | Wins | Tied or unresolved | Losses | Mean preference |
| --- | ---: | ---: | ---: | ---: | ---: |
| Final confirmation | 3 | 1 | 1 | 1 | 3.5 |

**/fix, rail-callboard.** win. Scores: 5 and 5.

**/design-system, tile-log.** Tied or unresolved. Scores: 5 and 2.

**/fix, tide-board.** loss. Scores: 2 and 2.

## Inspect two comparisons

These are the first stable win and loss in task order, when present. Each image shows the actual 390px capture.

### tally-home · /landing

A studio owner wants to see whether this invoice tool can show what clients still owe. Make the opening screen and its next step convincing.

| Plain | With procedure |
| --- | --- |
| ![Plain mobile screen](https://unvibed.app/bench/tally-home/plain/mobile.png) | ![Procedure mobile screen](https://unvibed.app/bench/tally-home/kit-landing/mobile.png) |

Procedure-oriented scores: 7 and 6. [Read both judgments](https://unvibed.app/benchmark#tally-home-kit-landing).

### data-table · /fix

Finish a dispatch manifest for a dock worker locating the next outbound crate. Preserve every shipment and its destination.

| Plain | With procedure |
| --- | --- |
| ![Plain mobile screen](https://unvibed.app/bench/data-table/plain/mobile.png) | ![Procedure mobile screen](https://unvibed.app/bench/data-table/kit-fix/mobile.png) |

Procedure-oriented scores: 2 and 2. [Read both judgments](https://unvibed.app/benchmark#data-table-kit-fix).

## Individual public skills

The 36 trials produced 7 keep, 2 optional and 27 skip verdicts. These are decisions for the tested web task. Each skill was compared separately with its task’s plain control. See the per-skill results and reasons in [the full report](https://unvibed.app/benchmark).

## Limits

One generation per treatment and task, one judge model, two presentation orders. These are preference judgments about fixed synthetic screens. They do not measure checkout conversion, runtime safety, motion timing, production reliability or how all users would choose. Native-only and illustration skills were applied to the closest screen in a web-only benchmark; their results have limited relevance to native apps or original illustration work. Reused baselines and related screens mean the pairs are not independent statistical trials.

Reproduce the run using [the benchmark instructions](https://unvibed.app/benchmark). All individual judgments and rendered candidate files are linked from [the full report](https://unvibed.app/benchmark).