Repair loop and screenshots

A widget can fail in ways the model cannot see from its own output: a script throws, a library URL is blocked by the policy, or everything renders at zero height. This page covers the pieces that make those failures visible.

previewWidget

import { previewWidget } from "generative-frame";

const result = await previewWidget(code, { width: 680, appearance: "dark" });
// { ok, kind, width, height, blank, errors, console, screenshot?, screenshotError? }

previewWidget renders complete code in a hidden frame, waits for it to settle, inspects it, takes a screenshot, and disposes the frame. ok is true when there are no errors and the render is not blank. It needs a browser.

The preview frame stays inside the viewport, nearly transparent and behind the page, because Chrome stops animation frames in cross-origin frames outside the viewport, which would freeze script-driven charts mid-animation.

buildRepairFeedback

import { buildRepairFeedback } from "generative-frame/repair";

const feedback = buildRepairFeedback(result);
// feedback.text:
// The widget did not render cleanly. Fix the causes below.
//
// Errors (1):
// - error: Chart is not defined (line 14:5)
//
// Rendered height: 0px.

It takes a RenderReport and returns { ok, round, errors, console, blank, height?, screenshot?, text }. text lists errors with their locations, a blank-render note, an excerpt of console warnings and errors (the last 20 by default), and the rendered height. Options: round, ok (override the verdict), consoleLimit, and includeScreenshot (default true).

type RenderReport = {
  errors: WidgetError[];
  console: ConsoleEntry[];
  blank: boolean;
  height?: number;
  screenshot?: string;
};

A PreviewResult and the output of widget.inspect() (with height taken from size) both fit this shape. The assistant-ui toolkit uses the same text in its render reports.

repairLoop

import { previewWidget } from "generative-frame";
import { repairLoop } from "generative-frame/repair";

const { ok, code, report, rounds } = await repairLoop({
  generate: async ({ round, previousCode, feedback }) =>
    askModel({ previousCode, feedback: feedback?.text }),
  render: (code) => previewWidget(code),
  maxRounds: 3,
});

Each round calls generate, renders the result, and builds feedback for the next round. generate returns new code as a string, or { edits } with exact replacements on the previous round's code. Edits that do not apply count as a failed round with an error that says why.

The loop stops at the first accepted render. If none is accepted, it returns the attempt with the fewest errors (a blank render counts as one).

Screenshots

const { dataUrl, width, height } = await widget.screenshot({ scale: 2 });

A host page cannot capture a cross-origin frame, so the screenshot is taken inside the frame: the runtime clones the widget, inlines computed styles, serializes it into an SVG foreignObject, draws that onto a canvas, and returns a PNG data URL. scale defaults to the frame's device pixel ratio, capped at 2, and background defaults to --color-background. Canvas elements are copied as images.

It is best effort:

• Web fonts loaded by URL are not embedded, because fetching them would need network access the frame does not have by default. Fonts already inlined as data: URLs are.
• Cross-origin images served without CORS headers cannot be inlined.
• Pseudo-elements (::before, ::after) are not reproduced.
• Layout can drift by a few pixels from the live frame.
• The canvas side is capped at 4096 pixels; larger captures are scaled down.