Skip to content
All Benchmarks
Dataset & BenchmarkJune 2026

Image-to-Code: A Dataset & Benchmark for Static Web Page Replication

Image-to-CodeMultimodalDatasetFine-tuning

Image-to-Code (ITC) is the task of generating front-end code directly from visual designs. At its core is one fundamental problem: given a screenshot of a web page, produce HTML/CSS whose rendering is faithful to the original. This report introduces our dataset for that task, the evaluation protocol behind it, benchmark results for four frontier multimodal models, and ablation studies that measure what the data contributes in fine-tuning.

+16.5
Total-score gain over the base model after fine-tuning on 1,000 dataset samples (52.4 → 68.9)
52%
Replication success rate after fine-tuning, up from 9% zero-shot and double the 25% reached with public data
8
Evaluation metrics across 4 dimensions, with weights validated against human review scores

1. Background

Image-to-Code (ITC), the automatic generation of front-end code from visual design mockups, is becoming a key link in human–AI collaborative development. Within this space, single-page replication is the most fundamental and central task: given a screenshot of a web page, the model must produce HTML/CSS code whose visual rendering is highly faithful to the original. The task jointly stresses a model's visual understanding, layout reasoning, style restoration, and code-organization abilities. This dataset is intended for building high-quality (screenshot, code) paired data to train models for end-to-end generation from visual input to code output.

We constructed an evaluation set of 300 samples and, under a common scoring protocol, evaluated four mainstream multimodal models (GPT-5.4, Qwen-3.6-plus, Kimi-k2.6, and Doubao-seed-2.0), yielding the following results:

Left: score distribution by model (n=300, bin=5). Right: model scores by difficulty (sorted by difficulty score, with trend).

Note: the full analysis report appears in Section 5.1.

2. Data Design

2.1 Dataset Architecture

The dataset comprises two categories, real data and synthetic data, stored in a 4:6 mixture, totaling 3,000 samples. Real data is built by crawling code from live websites, then cleaning, filtering, and quality-assessing it before rendering screenshots (all rendered with the Playwright framework in the Chrome browser). Synthetic data is produced by a baseline model generating code from prompts, which is then rendered. The file structure is as follows:

DATAS/
├── case_001/                # Item 1 (real data)
│   ├── assets/              # Local images (present only for real data)
│   ├── question.png
│   ├── answer.html
│   └── analysis.json
├── case_002/                # Item 2 (synthetic data)
│   ├── question.png
│   ├── answer.html
│   └── analysis.json
└── case_003/                # Item 3; subsequent items follow the same structure

2.2 Data Types

  • Real data: simulates complex production scenarios, containing genuine business logic, multi-level nested components, and non-standard code. Its value lies in exposing a model's engineering weaknesses (component decomposition, commenting conventions, naming semantics), capabilities that cannot be trained from synthetic data alone. It serves as supplementary data to improve generalization to real-world scenarios.
  • Synthetic data: offers the advantages of batch production and a controllable design space; through a combinatorial matrix (scenario × tech stack × convention × color scheme) it systematically covers diversity, and its screenshot–code pairs are perfectly aligned, providing a high-certainty training signal. It serves as the main body of training data, ensuring the model learns the basic visual-to-code mapping.

The mixed regime avoids the inherent flaws of either type alone: pure real data is sparsely distributed and noisy, while pure synthetic data cannot cover engineering complexity.

2.3 Analysis JSON

Prior work such as Web2Code and Design2Code has shown that introducing a textual intermediate representation can raise visual fidelity (CLIP Score) by 5–8% and spatial-layout metrics (Position/Layout Similarity) by 10–15%. We introduce an analysis.json file and, after testing across samples of varying difficulty, reach the following conclusions:

  • Positive gains across all difficulty levels: with Analysis-assisted generation, the weighted total score for page replication rises at every difficulty tier, validating the effectiveness of the auxiliary mechanism.
  • Marked improvement on core metrics: CLIP Score (semantic similarity), Position Similarity, and Layout Similarity (spatial layout) improve substantially, indicating that the Analysis file effectively helps the model understand page semantics and layout.
  • Minor fluctuation on individual metrics: some samples show small declines in Block-Match and CW-SSIM, which is expected: the Analysis file is optimized for overall structure and semantic restoration rather than local block-level matching, and these fluctuations do not affect the overall assessment.
  • Value on complex samples: for hard samples (containing real images, with very poor color/layout restoration), the Analysis assistance still raises scores, indicating good adaptability to complex scenarios and an effective reduction in replication difficulty.

The field schema is shown below:

{
  "status": " ",
  "layout": {
    "method": " ",
    "viewport": " ",
    "sections": [
      { "tag": " ", "layout": " ", "height_ratio": }
    ],
    "nesting_depth": ,
    "total_elements":
  },
  "components": [
    {
      "name": " ",
      "tag": " ",
      "bbox": [ ],
      "children_count": ,
      "css_key": [ ]
    }
  ],
  "colors": [
    { "hex": " ", "role": " " }
  ],
  "typography": [
    { "tag": " ", "size_px": , "weight": , "color": " " }
  ],
  "spacing": {
    "section_gap_px": ,
    "content_padding_px": ,
    "card_gap_px": ,
    "base_unit_px":
  },
  "decorations": [
    "border-radius: ",
    "box-shadow: ",
    "border: "
  ],
  "css_strategy": [
    "display: ",
    "display: ",
    "position: ",
    "overflow: "
  ]
}
FieldTypeMeaning
status"ok" | "error"Whether the analysis succeeded
layout.methodstringTop-level page layout method
layout.viewportstringTarget viewport size
layout.sectionsarrayList of page skeleton sections; each contains the HTML tag, internal layout method, etc.
layout.nesting_depthintMaximum DOM nesting depth, reflecting page complexity
layout.total_elementsintTotal number of visible elements; guards against under-/over-generation
componentsarrayList of all visible components
components[].namestringDescriptive component name
components[].tagstringPrimary HTML tag
components[].bbox[x, y, w, h]Normalized coordinates (0.0–1.0), relative to the viewport
components[].children_countintNumber of direct child elements
components[].css_keystring[]The component's most critical CSS declarations, directly writable into style
colorsarrayColors used on the page; exact hex value + semantic role
typographyarrayExact numeric values for each text style (px/weight/color)
spacingobjectSpacing system; all values are integer px
decorationsstring[]Decorative CSS declarations (border-radius, box-shadow, border)
css_strategystring[]Key CSS layout/positioning capabilities used on the page

Note: a worked test-sample example appears in Section 5.2.

3. Difficulty Stratification

3.1 Evaluation Scheme

To ensure evaluation accuracy, the scheme combines the strength of metric-based computation in precisely quantifying local features with the strength of multimodal large language models (MLLMs) in multi-dimensional semantic analysis, using the latter to compensate for the weakness of purely computational metrics on global semantic information. We adopt the following two-layer evaluation architecture:

  • LLM evaluation layer: generates structured scores using GPT-5.4, based on a Rubrics scoring framework.
  • Scoring evaluation layer: scores along 4 dimensions (pixel, semantic, spatial layout, content) using 8 evaluation metrics (see table below).

The final output is a dimension-weighted composite score, used to determine each sample's difficulty score. The weighting formula is:

ScoreEval = 0.20·pixel + 0.25·semantic + 0.30·spatial + 0.25·content
NameEvaluation dimensionQC dimensionRole
CW-SSIMPixel-level similarityPixel restorationMeasures structural and textural similarity at the pixel level; relatively robust to brightness/contrast changes.
LPIPSPixel-level similarityPixel restorationUses perceptual features to measure the visual difference between two images, complementing traditional pixel metrics with human-perceptual consistency.
CLIP ScoreSemantic similaritySemantic consistencyMeasures the consistency of the generated and target images at high-level semantics (content understanding).
Block-MatchSpatial layoutSpatial layoutMeasures local-region layout correspondence via block-level matching.
Position SimilaritySpatial layoutSpatial layoutEvaluates whether major elements are positionally aligned within the page.
Layout SimilaritySpatial layoutSpatial layoutEvaluates whether the overall layout structure (block relationships, arrangement) is consistent.
Color SimilarityContent restorationColor consistencyEvaluates color distribution and dominant-tone consistency.
LLEMContent restorationContent restorationCompares HTML structure and interactive-element consistency, reflecting structural-semantic restoration of the page.

Note: validation of the evaluation scheme's effectiveness appears in Section 5.3.

3.2 Data Filtering

Based on the composite-score distribution of the full sample set (shown below), we establish a three-tier difficulty stratification:

Left: weighted-score distribution scatter. Right: success-normalized score distribution (Low 30% / Middle 60% / Top 10%); thresholds 0.60 and 0.80, mean 0.641, median 0.658.
  • Easy data (evaluation range [85%–100%]). Relative to the evaluation baseline, samples scoring above it are deemed easy and assumed to have very low training value.
  • Complex data, i.e., submitted data (evaluation range [60%–85%)). Samples in this band rarely show anomalies in generation accuracy or completeness; their issues lie mainly in spatial layout and color restoration. They fall within the training objective and exhibit good stability, and are assumed to be valid data to be packaged and submitted.
  • Problematic data (evaluation range [0%–60%)). For synthesized samples, scores below 60% mostly stem from rendering failures, truncated model generation, or malformed prompts (i.e., data-generation errors) rather than genuinely low image similarity. Data below 60% is therefore assumed problematic and is not retained.

Because model-synthesized prompts may exhibit issues such as component overflow or unreasonable layout, a manual quality-control step is added after score-based filtering to screen the samples and ensure data quality.

4. Model Training and Evaluation

4.1 Parameter Configuration

To validate the dataset's effectiveness, we use the following training configuration and design a set of ablation experiments to verify the effect:

  • Base model: Qwen3.5-9B-Base
  • GPU configuration: 4 × NVIDIA GeForce RTX 3090
  • Distributed training framework: DeepSpeed
  • Parameter-efficient fine-tuning strategy: QLoRA
  • Training hyperparameters:
    • Epochs: 5
    • Learning rate: 2×10⁻⁴
    • Random seed: 42
    • Warmup: 3%
    • Weight decay: 0
    • Per-device batch size: 1
    • Gradient accumulation steps: 4 (effective global batch size = 1 × 4 GPUs × 4 accumulation = 16)
    • Max gradient-norm clipping threshold: 1.0

4.2 Ablation Experiments

4.2.1 Training Convergence Validation

Before any specific ablation comparison, we first verify that the training process itself converges healthily. Training loss (cross-entropy) should decrease steadily and stabilize; eval loss should decrease in tandem, or at least not rebound markedly; and the gradient norm should stay within a reasonable range rather than oscillating violently. If training loss is falling while eval loss begins to rise, the model has entered an overfitting regime: memorizing the training set rather than acquiring generalization.

Training convergence over the full 300 steps with overfitting detection; best eval_loss = 0.1624 at step 120 (epoch ≈ 1.9).

From the overall trajectory of the loss curves, training divides cleanly into three phases:

  • Phase 1: rapid convergence (steps 10–120, ≈ first 2 epochs). Training loss falls steadily from 0.2517 to 0.1453 (−42.2%), while eval loss falls in step from 0.2231 to 0.1624 (−27.2%). The two curves track closely, indicating the model is learning generalizable features rather than merely memorizing training samples.
  • Phase 2: plateau oscillation (steps 130–190, ≈ epochs 2–3). Training loss continues to decline slowly to 0.1051, but eval loss hovers in the narrow 0.1634–0.1669 band and no longer follows training loss downward; the signals begin to diverge.
  • Phase 3: overfitting (steps 200–300, ≈ epochs 3–5). Training loss is pushed further down to 0.0501, while eval loss climbs back to 0.1958, a 20.6% rebound relative to the optimum (0.1624 at step 120). The train–eval gap widens from 0.1031 to 0.1457; the overfitting signature is unmistakable.
MetricValue
Initial train_loss0.2517
Final train_loss0.0501
Initial eval_loss0.2231
Best eval_loss0.1624
Final eval_loss0.1958
Overfitting rebound+20.6%
Best checkpointstep 120
Gradient-norm stability over the full 300 steps; Phase 1 mean = 0.116, Phase 3 mean = 0.205; best checkpoint at step 120.

On the gradient-norm side, the L2 gradient norm remained within 0.10–0.23 throughout training, with no sign of gradient explosion or vanishing. Notably, the gradient norm rises somewhat during the overfitting phase (after step 200), from ≈0.13 to 0.20–0.23, consistent with the model continuing to push down an already-small training loss and requiring larger updates to fit residual noise, which independently corroborates the overfitting diagnosis. The learning-rate schedule uses linear decay, with warmup over ≈9 steps (3% × 315 total steps); the peak learning rate of 2×10⁻⁴ is reached around step 10 and then decays smoothly to the end of training, behaving as expected.

Taken together, training exhibits healthy convergence within the first 2 epochs, with the model effectively acquiring the screenshot-to-HTML mapping. However, the full 5-epoch schedule carries clear overfitting risk, and the optimal model state occurs at step 120 (≈ epoch 1.9). All subsequent ablation evaluations are therefore based on the checkpoint with the lowest eval loss in each run, rather than the final weights at the end of training.

4.2.2 Data Scaling Ablation

Experimental design:

  • Fixed training configuration (QLoRA r=16, lr=2e-4, 5 epochs)
  • Data variable: training-data volume (300 / 500 / 1000 samples)
  • Objective: observe the effect of data volume on the final evaluation score and judge whether marginal returns diminish.
Training dataMean total scoreSuccess rateRule-based scoreLLM score
300 samples57.5123.0%72.4542.56
500 samples61.7432.0%74.5349.83
1000 samples68.9152.0%79.2258.54

From the available comparison, a 3.3× increase in data volume raises the total score from 57.51 to 68.91 (+11.40), lifts the success rate from 23.0% to 52.0% (+29 percentage points), and improves the LLM-judge score from 42.56 to 58.54 (+15.98). These gains are substantial; in particular, the more-than-doubling of the success rate indicates that 300 samples is far from the saturation point and that the model remains “data-hungry” at this scale.

4.2.3 Data Quality Ablation

Under an identical training configuration, we trained separately on the in-house dataset (1000 samples) and on a public, authoritative dataset (1000 samples randomly drawn, in three groups, from 2.96M public samples), and evaluated both on the same test set. Using the base model's zero-shot inference as a lower-bound baseline, this constitutes a three-tier “baseline / public / in-house” comparison.

Training/eval-loss curves: in-house data (left) vs. public data (right).

Compared with the in-house training-loss curve (left), the public-data curve (right) shows a cliff-like drop over steps 0–20 and is extremely flat thereafter, presumably because the public dataset is too small and too simple. Across the randomly sampled full set, the median HTML length is only ≈1.6K characters; the model essentially fits the data within the first epoch (25 steps), and the remaining 4 epochs revisit the same data with little incremental learning. Eval loss is nearly stationary from step 20 onward (≈0.068–0.072), indicating convergence by steps 20–30, with subsequent training amounting to detail-level fine-tuning.

Data-quality comparison:

Experiment dataMean total scoreSuccess rateRule-based scoreLLM scoreGain over baseline
Base model52.379.0%70.9533.80N/A
Public dataset62.8225.3%75.9349.73+10.45
In-house dataset68.9152.0%79.2258.54+16.54
Radar charts: dimension scores, per-metric scores, and LLM component scores for baseline / production (in-house) / authoritative (public) data.

Dimension-level comparison:

DimensionBase zero-shotPublic dataIn-house data
pixel0.74320.73540.7588
semantic0.82070.88760.9101
spatial0.63590.71880.7148
content0.65950.69860.7940

In terms of total score, the model trained on the in-house dataset (68.91) significantly outperforms the one trained on public data (62.82), a lead of 6.09 points, with a success rate of 52.0% versus 25.3%, more than double. Both improve substantially over the base zero-shot model (52.37), confirming that fine-tuning itself is effective; however, the in-house improvement (+16.54) far exceeds that of the public data (+10.45).

At the dimension level, the in-house data holds a clear advantage on the content (0.7940 vs 0.6986) and semantic (0.9101 vs 0.8876) dimensions, indicating higher annotation quality for text-content replication and semantic consistency. The spatial dimension is close between the two (0.7148 vs 0.7188), so layout-restoration ability is essentially on par; pixel is slightly higher for the in-house data (0.7588 vs 0.7354). Overall, the in-house dataset is of higher quality than the public dataset and yields markedly better training outcomes.

5. Experiments and Testing

5.1 Model Replication-Capability Analysis Report

5.1.1 Overall Performance

ModelMeanMedianStdMinMax
GPT-5.480.0283.2511.1321.4890.35
Qwen-3.6-plus72.6480.187.8243.0989.79
Doubao-seed-2.072.1079.166.5550.2689.17
Kimi-k2.670.5880.3212.2235.5389.99
  • GPT-5.4 has the highest mean (80.02), and its histogram is clearly right-shifted, with 80+ accounting for 73% (219/300) of cases.
  • Doubao has the smallest variance (std = 6.55) and is the most stable, with only 7 cases (2.3%) below 60, but also the lowest ceiling (max = 89.17).
  • Kimi has the largest variance (std = 12.22) and a pronounced long tail: its histogram shows ≈25 samples in the 30–55 range, indicating severe replication failures on some cases.
  • Qwen has a relatively concentrated distribution centered in the 65–80 range, appearing unimodal and right-skewed.

5.1.2 Difficulty-Stratified Performance

Mean score per model across difficulty tiers:

DifficultyGPT-5.4Qwen-3.6-plusDoubao-seed-2.0Kimi-k2.6
Easy (n=60)80.775.273.972.8
Medium (n=180)80.472.671.769.9
Hard (n=60)78.170.271.570.4
Max drop2.65.02.42.9

Key findings:

  • GPT leads at every difficulty tier and shows only a small decline from Easy to Hard (80.7 → 78.1, −2.6), indicating good robustness to difficulty.
  • Qwen has the largest drop from Easy to Hard (75.2 → 70.2, −5.0), with marked volatility.
  • Doubao shows small differences across tiers (73.9 → 71.7 → 71.5), exhibiting good stability but lacking high-score breakthroughs.
  • The trend lines show all four models recovering slightly at the tail of the Hard tier. Because the x-axis is sorted by per-case difficulty score, this suggests that original-page visual complexity correlates positively with replication difficulty as a trend, while still containing randomness.

5.1.3 Head-to-Head Comparison

Per-case win rate; a margin within ±0.5 points counts as a tie.

ComparisonGPT winTieGPT loss
GPT vs Qwen68.7%7.3%24.0%
GPT vs Kimi69.3%7.0%23.7%
GPT vs Doubao72.7%3.0%24.3%

GPT's win rate against each of the other three models is in the 68–73% range, but ≈24% of cases are still won by the other model, indicating that different models hold advantages on particular page types.

5.1.4 Stability and Risk

  • Low-score risk (<60) share: Kimi 10.3% > Qwen 4.0% > GPT 2.7% > Doubao 2.3%.
  • Kimi's number of low-score cases is 3–4× that of the other models, with a clear second peak in the 30–55 range on its histogram, possibly tied to systematic failure on certain page types (e.g., dynamic-interaction-heavy pages).
  • Although Doubao does not have the highest mean, it has the highest lower bound (min = 50.26), the lowest low-score share, and the best robustness.

5.2 Analysis-JSON Effectiveness Validation

Owing to space constraints, three test samples were selected at random for demonstration. The columns of each figure, left to right, are: the original (question), the plain replication, and the Analysis-assisted replication; all samples follow the same format. In the metric tables, replicatee.png denotes the plain replication and TTC.png the Analysis-assisted replication.

Test sample 1:

Sample 1: original / plain replication / Analysis-assisted replication.
Metricreplicatee.pngTTC.pngChange
CW-SSIM0.91970.9704+0.05
LPIPS0.78200.8857+0.10
CLIP Score0.57560.9527+0.38
Position Sim0.58180.8702+0.29
Layout Sim0.82100.9092+0.09
Block-Match0.88030.7079−0.17
Color Sim0.93790.9930+0.06
LLEM0.55300.6379+0.08
Weighted total0.72870.8764+0.15

Test sample 2:

Sample 2: original / plain replication / Analysis-assisted replication.
Metricreplicatee.pngTTC.pngChange
CW-SSIM0.73020.7197−0.01
LPIPS0.61100.6842+0.07
CLIP Score0.73110.7840+0.05
Position Similarity0.62270.6483+0.03
Layout Similarity0.75150.8845+0.13
Block-Match0.57600.7500+0.17
Color Similarity0.67790.6657−0.01
LLEM0.59340.59340
Weighted total0.67080.7220+0.05

Test sample 3:

Sample 3: original / plain replication / Analysis-assisted replication.
Metricreplicatee.pngTTC.pngChange
CW-SSIM0.74210.8524+0.11
LPIPS0.72150.7773+0.06
CLIP Score0.60450.7564+0.15
Position Similarity0.74130.7788+0.04
Layout Similarity0.88300.9075+0.02
Block-Match0.76460.6875−0.08
Color Similarity0.95700.9732+0.02
LLEM0.89710.89710
Weighted total0.76810.8232+0.06

5.3 Visual-Evaluation Scheme Validation

Rubrics scoring-framework stability. To assess scoring stability under the Rubrics framework, the same sample was evaluated 5 times independently. Score variance was small, consistent with expectations (results below).

Final-score trend across 5 independent evaluation runs, for four samples.

Composite-score fit validation.

  • Comparison baselines: human review scores, LLM review scores, and computed-metric scores.
  • Weight configurations: multiple metric-to-LLM weighting ratios (9:1, 8:2, 7:3, 6:4, 5:5, 4:6, 3:7, 2:8).

The figure below shows the numeric curves for human review, LLM review, and metric computation.

Per-sample score curves: computed metrics, LLM score, human score, and the weighted combinations.

Computed results:

metric : LLMMAERMSEBiasPearsonSpearman
8:24.58016.01642.04210.26390.3022
7:34.18015.69872.21860.35880.4066
6:43.98435.47302.39500.44600.5016
5:53.93945.35102.57140.51060.5161
4:64.01155.33972.74780.54670.5337
3:74.21285.43992.92420.55940.5323
2:84.45975.64563.11070.55740.5424
Pure LLM fit5.27096.32743.45350.53560.5012
Pure metric fit5.49436.87331.68930.09950.1483

Analysis:

  • Pure-LLM and pure-metric evaluation each achieve only a relatively low level of agreement with human evaluation.
  • The metric:LLM ratio of 5:5 yields the smallest error relative to human scores (figure below, left).
  • The metric:LLM ratio of 3:7 yields the highest correlation with human scores (figure below, right).
Left: fit under the 5:5 configuration. Right: fit under the 3:7 configuration.

Balancing error control against evaluation stability, the 5:5 weighting is adopted as the default computation scheme for the composite score.