1. Background
Image-to-Code (ITC), the automatic generation of front-end code from visual design mockups, is becoming a key link in human–AI collaborative development. Within this space, single-page replication is the most fundamental and central task: given a screenshot of a web page, the model must produce HTML/CSS code whose visual rendering is highly faithful to the original. The task jointly stresses a model's visual understanding, layout reasoning, style restoration, and code-organization abilities. This dataset is intended for building high-quality (screenshot, code) paired data to train models for end-to-end generation from visual input to code output.
We constructed an evaluation set of 300 samples and, under a common scoring protocol, evaluated four mainstream multimodal models (GPT-5.4, Qwen-3.6-plus, Kimi-k2.6, and Doubao-seed-2.0), yielding the following results:
Note: the full analysis report appears in Section 5.1.
2. Data Design
2.1 Dataset Architecture
The dataset comprises two categories, real data and synthetic data, stored in a 4:6 mixture, totaling 3,000 samples. Real data is built by crawling code from live websites, then cleaning, filtering, and quality-assessing it before rendering screenshots (all rendered with the Playwright framework in the Chrome browser). Synthetic data is produced by a baseline model generating code from prompts, which is then rendered. The file structure is as follows:
DATAS/
├── case_001/ # Item 1 (real data)
│ ├── assets/ # Local images (present only for real data)
│ ├── question.png
│ ├── answer.html
│ └── analysis.json
├── case_002/ # Item 2 (synthetic data)
│ ├── question.png
│ ├── answer.html
│ └── analysis.json
└── case_003/ # Item 3; subsequent items follow the same structure2.2 Data Types
- Real data: simulates complex production scenarios, containing genuine business logic, multi-level nested components, and non-standard code. Its value lies in exposing a model's engineering weaknesses (component decomposition, commenting conventions, naming semantics), capabilities that cannot be trained from synthetic data alone. It serves as supplementary data to improve generalization to real-world scenarios.
- Synthetic data: offers the advantages of batch production and a controllable design space; through a combinatorial matrix (scenario × tech stack × convention × color scheme) it systematically covers diversity, and its screenshot–code pairs are perfectly aligned, providing a high-certainty training signal. It serves as the main body of training data, ensuring the model learns the basic visual-to-code mapping.
The mixed regime avoids the inherent flaws of either type alone: pure real data is sparsely distributed and noisy, while pure synthetic data cannot cover engineering complexity.
2.3 Analysis JSON
Prior work such as Web2Code and Design2Code has shown that introducing a textual intermediate representation can raise visual fidelity (CLIP Score) by 5–8% and spatial-layout metrics (Position/Layout Similarity) by 10–15%. We introduce an analysis.json file and, after testing across samples of varying difficulty, reach the following conclusions:
- Positive gains across all difficulty levels: with Analysis-assisted generation, the weighted total score for page replication rises at every difficulty tier, validating the effectiveness of the auxiliary mechanism.
- Marked improvement on core metrics: CLIP Score (semantic similarity), Position Similarity, and Layout Similarity (spatial layout) improve substantially, indicating that the Analysis file effectively helps the model understand page semantics and layout.
- Minor fluctuation on individual metrics: some samples show small declines in Block-Match and CW-SSIM, which is expected: the Analysis file is optimized for overall structure and semantic restoration rather than local block-level matching, and these fluctuations do not affect the overall assessment.
- Value on complex samples: for hard samples (containing real images, with very poor color/layout restoration), the Analysis assistance still raises scores, indicating good adaptability to complex scenarios and an effective reduction in replication difficulty.
The field schema is shown below:
{
"status": " ",
"layout": {
"method": " ",
"viewport": " ",
"sections": [
{ "tag": " ", "layout": " ", "height_ratio": }
],
"nesting_depth": ,
"total_elements":
},
"components": [
{
"name": " ",
"tag": " ",
"bbox": [ ],
"children_count": ,
"css_key": [ ]
}
],
"colors": [
{ "hex": " ", "role": " " }
],
"typography": [
{ "tag": " ", "size_px": , "weight": , "color": " " }
],
"spacing": {
"section_gap_px": ,
"content_padding_px": ,
"card_gap_px": ,
"base_unit_px":
},
"decorations": [
"border-radius: ",
"box-shadow: ",
"border: "
],
"css_strategy": [
"display: ",
"display: ",
"position: ",
"overflow: "
]
}| Field | Type | Meaning |
|---|---|---|
| status | "ok" | "error" | Whether the analysis succeeded |
| layout.method | string | Top-level page layout method |
| layout.viewport | string | Target viewport size |
| layout.sections | array | List of page skeleton sections; each contains the HTML tag, internal layout method, etc. |
| layout.nesting_depth | int | Maximum DOM nesting depth, reflecting page complexity |
| layout.total_elements | int | Total number of visible elements; guards against under-/over-generation |
| components | array | List of all visible components |
| components[].name | string | Descriptive component name |
| components[].tag | string | Primary HTML tag |
| components[].bbox | [x, y, w, h] | Normalized coordinates (0.0–1.0), relative to the viewport |
| components[].children_count | int | Number of direct child elements |
| components[].css_key | string[] | The component's most critical CSS declarations, directly writable into style |
| colors | array | Colors used on the page; exact hex value + semantic role |
| typography | array | Exact numeric values for each text style (px/weight/color) |
| spacing | object | Spacing system; all values are integer px |
| decorations | string[] | Decorative CSS declarations (border-radius, box-shadow, border) |
| css_strategy | string[] | Key CSS layout/positioning capabilities used on the page |
Note: a worked test-sample example appears in Section 5.2.
3. Difficulty Stratification
3.1 Evaluation Scheme
To ensure evaluation accuracy, the scheme combines the strength of metric-based computation in precisely quantifying local features with the strength of multimodal large language models (MLLMs) in multi-dimensional semantic analysis, using the latter to compensate for the weakness of purely computational metrics on global semantic information. We adopt the following two-layer evaluation architecture:
- LLM evaluation layer: generates structured scores using GPT-5.4, based on a Rubrics scoring framework.
- Scoring evaluation layer: scores along 4 dimensions (pixel, semantic, spatial layout, content) using 8 evaluation metrics (see table below).
The final output is a dimension-weighted composite score, used to determine each sample's difficulty score. The weighting formula is:
ScoreEval = 0.20·pixel + 0.25·semantic + 0.30·spatial + 0.25·content| Name | Evaluation dimension | QC dimension | Role |
|---|---|---|---|
| CW-SSIM | Pixel-level similarity | Pixel restoration | Measures structural and textural similarity at the pixel level; relatively robust to brightness/contrast changes. |
| LPIPS | Pixel-level similarity | Pixel restoration | Uses perceptual features to measure the visual difference between two images, complementing traditional pixel metrics with human-perceptual consistency. |
| CLIP Score | Semantic similarity | Semantic consistency | Measures the consistency of the generated and target images at high-level semantics (content understanding). |
| Block-Match | Spatial layout | Spatial layout | Measures local-region layout correspondence via block-level matching. |
| Position Similarity | Spatial layout | Spatial layout | Evaluates whether major elements are positionally aligned within the page. |
| Layout Similarity | Spatial layout | Spatial layout | Evaluates whether the overall layout structure (block relationships, arrangement) is consistent. |
| Color Similarity | Content restoration | Color consistency | Evaluates color distribution and dominant-tone consistency. |
| LLEM | Content restoration | Content restoration | Compares HTML structure and interactive-element consistency, reflecting structural-semantic restoration of the page. |
Note: validation of the evaluation scheme's effectiveness appears in Section 5.3.
3.2 Data Filtering
Based on the composite-score distribution of the full sample set (shown below), we establish a three-tier difficulty stratification:
- Easy data (evaluation range [85%–100%]). Relative to the evaluation baseline, samples scoring above it are deemed easy and assumed to have very low training value.
- Complex data, i.e., submitted data (evaluation range [60%–85%)). Samples in this band rarely show anomalies in generation accuracy or completeness; their issues lie mainly in spatial layout and color restoration. They fall within the training objective and exhibit good stability, and are assumed to be valid data to be packaged and submitted.
- Problematic data (evaluation range [0%–60%)). For synthesized samples, scores below 60% mostly stem from rendering failures, truncated model generation, or malformed prompts (i.e., data-generation errors) rather than genuinely low image similarity. Data below 60% is therefore assumed problematic and is not retained.
Because model-synthesized prompts may exhibit issues such as component overflow or unreasonable layout, a manual quality-control step is added after score-based filtering to screen the samples and ensure data quality.
4. Model Training and Evaluation
4.1 Parameter Configuration
To validate the dataset's effectiveness, we use the following training configuration and design a set of ablation experiments to verify the effect:
- Base model: Qwen3.5-9B-Base
- GPU configuration: 4 × NVIDIA GeForce RTX 3090
- Distributed training framework: DeepSpeed
- Parameter-efficient fine-tuning strategy: QLoRA
- Training hyperparameters:
- Epochs: 5
- Learning rate: 2×10⁻⁴
- Random seed: 42
- Warmup: 3%
- Weight decay: 0
- Per-device batch size: 1
- Gradient accumulation steps: 4 (effective global batch size = 1 × 4 GPUs × 4 accumulation = 16)
- Max gradient-norm clipping threshold: 1.0
4.2 Ablation Experiments
4.2.1 Training Convergence Validation
Before any specific ablation comparison, we first verify that the training process itself converges healthily. Training loss (cross-entropy) should decrease steadily and stabilize; eval loss should decrease in tandem, or at least not rebound markedly; and the gradient norm should stay within a reasonable range rather than oscillating violently. If training loss is falling while eval loss begins to rise, the model has entered an overfitting regime: memorizing the training set rather than acquiring generalization.
From the overall trajectory of the loss curves, training divides cleanly into three phases:
- Phase 1: rapid convergence (steps 10–120, ≈ first 2 epochs). Training loss falls steadily from 0.2517 to 0.1453 (−42.2%), while eval loss falls in step from 0.2231 to 0.1624 (−27.2%). The two curves track closely, indicating the model is learning generalizable features rather than merely memorizing training samples.
- Phase 2: plateau oscillation (steps 130–190, ≈ epochs 2–3). Training loss continues to decline slowly to 0.1051, but eval loss hovers in the narrow 0.1634–0.1669 band and no longer follows training loss downward; the signals begin to diverge.
- Phase 3: overfitting (steps 200–300, ≈ epochs 3–5). Training loss is pushed further down to 0.0501, while eval loss climbs back to 0.1958, a 20.6% rebound relative to the optimum (0.1624 at step 120). The train–eval gap widens from 0.1031 to 0.1457; the overfitting signature is unmistakable.
| Metric | Value |
|---|---|
| Initial train_loss | 0.2517 |
| Final train_loss | 0.0501 |
| Initial eval_loss | 0.2231 |
| Best eval_loss | 0.1624 |
| Final eval_loss | 0.1958 |
| Overfitting rebound | +20.6% |
| Best checkpoint | step 120 |
On the gradient-norm side, the L2 gradient norm remained within 0.10–0.23 throughout training, with no sign of gradient explosion or vanishing. Notably, the gradient norm rises somewhat during the overfitting phase (after step 200), from ≈0.13 to 0.20–0.23, consistent with the model continuing to push down an already-small training loss and requiring larger updates to fit residual noise, which independently corroborates the overfitting diagnosis. The learning-rate schedule uses linear decay, with warmup over ≈9 steps (3% × 315 total steps); the peak learning rate of 2×10⁻⁴ is reached around step 10 and then decays smoothly to the end of training, behaving as expected.
Taken together, training exhibits healthy convergence within the first 2 epochs, with the model effectively acquiring the screenshot-to-HTML mapping. However, the full 5-epoch schedule carries clear overfitting risk, and the optimal model state occurs at step 120 (≈ epoch 1.9). All subsequent ablation evaluations are therefore based on the checkpoint with the lowest eval loss in each run, rather than the final weights at the end of training.
4.2.2 Data Scaling Ablation
Experimental design:
- Fixed training configuration (QLoRA r=16, lr=2e-4, 5 epochs)
- Data variable: training-data volume (300 / 500 / 1000 samples)
- Objective: observe the effect of data volume on the final evaluation score and judge whether marginal returns diminish.
| Training data | Mean total score | Success rate | Rule-based score | LLM score |
|---|---|---|---|---|
| 300 samples | 57.51 | 23.0% | 72.45 | 42.56 |
| 500 samples | 61.74 | 32.0% | 74.53 | 49.83 |
| 1000 samples | 68.91 | 52.0% | 79.22 | 58.54 |
From the available comparison, a 3.3× increase in data volume raises the total score from 57.51 to 68.91 (+11.40), lifts the success rate from 23.0% to 52.0% (+29 percentage points), and improves the LLM-judge score from 42.56 to 58.54 (+15.98). These gains are substantial; in particular, the more-than-doubling of the success rate indicates that 300 samples is far from the saturation point and that the model remains “data-hungry” at this scale.
4.2.3 Data Quality Ablation
Under an identical training configuration, we trained separately on the in-house dataset (1000 samples) and on a public, authoritative dataset (1000 samples randomly drawn, in three groups, from 2.96M public samples), and evaluated both on the same test set. Using the base model's zero-shot inference as a lower-bound baseline, this constitutes a three-tier “baseline / public / in-house” comparison.
Compared with the in-house training-loss curve (left), the public-data curve (right) shows a cliff-like drop over steps 0–20 and is extremely flat thereafter, presumably because the public dataset is too small and too simple. Across the randomly sampled full set, the median HTML length is only ≈1.6K characters; the model essentially fits the data within the first epoch (25 steps), and the remaining 4 epochs revisit the same data with little incremental learning. Eval loss is nearly stationary from step 20 onward (≈0.068–0.072), indicating convergence by steps 20–30, with subsequent training amounting to detail-level fine-tuning.
Data-quality comparison:
| Experiment data | Mean total score | Success rate | Rule-based score | LLM score | Gain over baseline |
|---|---|---|---|---|---|
| Base model | 52.37 | 9.0% | 70.95 | 33.80 | N/A |
| Public dataset | 62.82 | 25.3% | 75.93 | 49.73 | +10.45 |
| In-house dataset | 68.91 | 52.0% | 79.22 | 58.54 | +16.54 |
Dimension-level comparison:
| Dimension | Base zero-shot | Public data | In-house data |
|---|---|---|---|
| pixel | 0.7432 | 0.7354 | 0.7588 |
| semantic | 0.8207 | 0.8876 | 0.9101 |
| spatial | 0.6359 | 0.7188 | 0.7148 |
| content | 0.6595 | 0.6986 | 0.7940 |
In terms of total score, the model trained on the in-house dataset (68.91) significantly outperforms the one trained on public data (62.82), a lead of 6.09 points, with a success rate of 52.0% versus 25.3%, more than double. Both improve substantially over the base zero-shot model (52.37), confirming that fine-tuning itself is effective; however, the in-house improvement (+16.54) far exceeds that of the public data (+10.45).
At the dimension level, the in-house data holds a clear advantage on the content (0.7940 vs 0.6986) and semantic (0.9101 vs 0.8876) dimensions, indicating higher annotation quality for text-content replication and semantic consistency. The spatial dimension is close between the two (0.7148 vs 0.7188), so layout-restoration ability is essentially on par; pixel is slightly higher for the in-house data (0.7588 vs 0.7354). Overall, the in-house dataset is of higher quality than the public dataset and yields markedly better training outcomes.
5. Experiments and Testing
5.1 Model Replication-Capability Analysis Report
5.1.1 Overall Performance
| Model | Mean | Median | Std | Min | Max |
|---|---|---|---|---|---|
| GPT-5.4 | 80.02 | 83.25 | 11.13 | 21.48 | 90.35 |
| Qwen-3.6-plus | 72.64 | 80.18 | 7.82 | 43.09 | 89.79 |
| Doubao-seed-2.0 | 72.10 | 79.16 | 6.55 | 50.26 | 89.17 |
| Kimi-k2.6 | 70.58 | 80.32 | 12.22 | 35.53 | 89.99 |
- GPT-5.4 has the highest mean (80.02), and its histogram is clearly right-shifted, with 80+ accounting for 73% (219/300) of cases.
- Doubao has the smallest variance (std = 6.55) and is the most stable, with only 7 cases (2.3%) below 60, but also the lowest ceiling (max = 89.17).
- Kimi has the largest variance (std = 12.22) and a pronounced long tail: its histogram shows ≈25 samples in the 30–55 range, indicating severe replication failures on some cases.
- Qwen has a relatively concentrated distribution centered in the 65–80 range, appearing unimodal and right-skewed.
5.1.2 Difficulty-Stratified Performance
Mean score per model across difficulty tiers:
| Difficulty | GPT-5.4 | Qwen-3.6-plus | Doubao-seed-2.0 | Kimi-k2.6 |
|---|---|---|---|---|
| Easy (n=60) | 80.7 | 75.2 | 73.9 | 72.8 |
| Medium (n=180) | 80.4 | 72.6 | 71.7 | 69.9 |
| Hard (n=60) | 78.1 | 70.2 | 71.5 | 70.4 |
| Max drop | 2.6 | 5.0 | 2.4 | 2.9 |
Key findings:
- GPT leads at every difficulty tier and shows only a small decline from Easy to Hard (80.7 → 78.1, −2.6), indicating good robustness to difficulty.
- Qwen has the largest drop from Easy to Hard (75.2 → 70.2, −5.0), with marked volatility.
- Doubao shows small differences across tiers (73.9 → 71.7 → 71.5), exhibiting good stability but lacking high-score breakthroughs.
- The trend lines show all four models recovering slightly at the tail of the Hard tier. Because the x-axis is sorted by per-case difficulty score, this suggests that original-page visual complexity correlates positively with replication difficulty as a trend, while still containing randomness.
5.1.3 Head-to-Head Comparison
Per-case win rate; a margin within ±0.5 points counts as a tie.
| Comparison | GPT win | Tie | GPT loss |
|---|---|---|---|
| GPT vs Qwen | 68.7% | 7.3% | 24.0% |
| GPT vs Kimi | 69.3% | 7.0% | 23.7% |
| GPT vs Doubao | 72.7% | 3.0% | 24.3% |
GPT's win rate against each of the other three models is in the 68–73% range, but ≈24% of cases are still won by the other model, indicating that different models hold advantages on particular page types.
5.1.4 Stability and Risk
- Low-score risk (<60) share: Kimi 10.3% > Qwen 4.0% > GPT 2.7% > Doubao 2.3%.
- Kimi's number of low-score cases is 3–4× that of the other models, with a clear second peak in the 30–55 range on its histogram, possibly tied to systematic failure on certain page types (e.g., dynamic-interaction-heavy pages).
- Although Doubao does not have the highest mean, it has the highest lower bound (min = 50.26), the lowest low-score share, and the best robustness.
5.2 Analysis-JSON Effectiveness Validation
Owing to space constraints, three test samples were selected at random for demonstration. The columns of each figure, left to right, are: the original (question), the plain replication, and the Analysis-assisted replication; all samples follow the same format. In the metric tables, replicatee.png denotes the plain replication and TTC.png the Analysis-assisted replication.
Test sample 1:
| Metric | replicatee.png | TTC.png | Change |
|---|---|---|---|
| CW-SSIM | 0.9197 | 0.9704 | +0.05 |
| LPIPS | 0.7820 | 0.8857 | +0.10 |
| CLIP Score | 0.5756 | 0.9527 | +0.38 |
| Position Sim | 0.5818 | 0.8702 | +0.29 |
| Layout Sim | 0.8210 | 0.9092 | +0.09 |
| Block-Match | 0.8803 | 0.7079 | −0.17 |
| Color Sim | 0.9379 | 0.9930 | +0.06 |
| LLEM | 0.5530 | 0.6379 | +0.08 |
| Weighted total | 0.7287 | 0.8764 | +0.15 |
Test sample 2:
| Metric | replicatee.png | TTC.png | Change |
|---|---|---|---|
| CW-SSIM | 0.7302 | 0.7197 | −0.01 |
| LPIPS | 0.6110 | 0.6842 | +0.07 |
| CLIP Score | 0.7311 | 0.7840 | +0.05 |
| Position Similarity | 0.6227 | 0.6483 | +0.03 |
| Layout Similarity | 0.7515 | 0.8845 | +0.13 |
| Block-Match | 0.5760 | 0.7500 | +0.17 |
| Color Similarity | 0.6779 | 0.6657 | −0.01 |
| LLEM | 0.5934 | 0.5934 | 0 |
| Weighted total | 0.6708 | 0.7220 | +0.05 |
Test sample 3:
| Metric | replicatee.png | TTC.png | Change |
|---|---|---|---|
| CW-SSIM | 0.7421 | 0.8524 | +0.11 |
| LPIPS | 0.7215 | 0.7773 | +0.06 |
| CLIP Score | 0.6045 | 0.7564 | +0.15 |
| Position Similarity | 0.7413 | 0.7788 | +0.04 |
| Layout Similarity | 0.8830 | 0.9075 | +0.02 |
| Block-Match | 0.7646 | 0.6875 | −0.08 |
| Color Similarity | 0.9570 | 0.9732 | +0.02 |
| LLEM | 0.8971 | 0.8971 | 0 |
| Weighted total | 0.7681 | 0.8232 | +0.06 |
5.3 Visual-Evaluation Scheme Validation
Rubrics scoring-framework stability. To assess scoring stability under the Rubrics framework, the same sample was evaluated 5 times independently. Score variance was small, consistent with expectations (results below).
Composite-score fit validation.
- Comparison baselines: human review scores, LLM review scores, and computed-metric scores.
- Weight configurations: multiple metric-to-LLM weighting ratios (9:1, 8:2, 7:3, 6:4, 5:5, 4:6, 3:7, 2:8).
The figure below shows the numeric curves for human review, LLM review, and metric computation.
Computed results:
| metric : LLM | MAE | RMSE | Bias | Pearson | Spearman |
|---|---|---|---|---|---|
| 8:2 | 4.5801 | 6.0164 | 2.0421 | 0.2639 | 0.3022 |
| 7:3 | 4.1801 | 5.6987 | 2.2186 | 0.3588 | 0.4066 |
| 6:4 | 3.9843 | 5.4730 | 2.3950 | 0.4460 | 0.5016 |
| 5:5 | 3.9394 | 5.3510 | 2.5714 | 0.5106 | 0.5161 |
| 4:6 | 4.0115 | 5.3397 | 2.7478 | 0.5467 | 0.5337 |
| 3:7 | 4.2128 | 5.4399 | 2.9242 | 0.5594 | 0.5323 |
| 2:8 | 4.4597 | 5.6456 | 3.1107 | 0.5574 | 0.5424 |
| Pure LLM fit | 5.2709 | 6.3274 | 3.4535 | 0.5356 | 0.5012 |
| Pure metric fit | 5.4943 | 6.8733 | 1.6893 | 0.0995 | 0.1483 |
Analysis:
- Pure-LLM and pure-metric evaluation each achieve only a relatively low level of agreement with human evaluation.
- The metric:LLM ratio of 5:5 yields the smallest error relative to human scores (figure below, left).
- The metric:LLM ratio of 3:7 yields the highest correlation with human scores (figure below, right).
Balancing error control against evaluation stability, the 5:5 weighting is adopted as the default computation scheme for the composite score.