--- title: WebSTAR canonical_url: "https://www.modelscope.cn/datasets/microsoft/WebSTAR" md_url: "https://www.modelscope.cn/datasets/microsoft/WebSTAR.md" repository: microsoft/WebSTAR last_updated: 2026-02-11 license: cdla-permissive-2.0 storage_size: "59 GB" downloads: 8374 stars: 0 --- # WebSTAR > WebSTAR - microsoft 在 ModelScope 开源的数据集。WebSTAR: WebVoyager Step-Level Trajectories with Augmented Reasoning microsoft/WebSTAR 是 ModelScope 魔搭社区上的数据集，存储大小 59 GB，采用 cdla-permissive-2.0 许可。 - **Repository**: microsoft/WebSTAR - **License**: cdla-permissive-2.0 - **Storage size**: 59 GB - **Downloads**: 8374 - **Stars**: 0 - **Last updated**: 2026-02-11 Source: https://www.modelscope.cn/datasets/microsoft/WebSTAR --- # WebSTAR: WebVoyager Step-Level Trajectories with Augmented Reasoning ## Dataset Description **WebSTAR (WebVoyager Step-Level Trajectories with Augmented Reasoning)** is a large-scale dataset for training and evaluating computer use agents with step-level quality scores. This dataset is part of the research presented in ["Scalable Data Synthesis for Computer Use Agents with Step-Level Filtering"](https://arxiv.org/abs/2512.10962) (He et al., 2025). Unlike traditional trajectory-level filtering approaches, WebSTAR provides fine-grained step-level scores for each action in an agent's trajectory, enabling more precise quality assessment and selective training on high-quality steps. NOTE: For the release, we combine the WebSTAR and WebSCORE datasets into a single unified dataset named WebSTAR, which includes the step-level scores and detailed justifications from the judge. ### Key Features - **Step-level scoring**: Each action in a trajectory is individually scored (1-10 scale) - **Detailed justifications**: Every score is accompanied by a comprehensive evaluation explaining the reasoning - **Multi-step reasoning**: Trajectories contain complex, multi-step web navigation tasks ## Dataset Structure ### Data Format Each example in WebSTAR is structured as follows: ```json { "system": "System prompt with action space definitions and instructions", "conversations": [ { "role": "user", "content": "Please analyze the attached screenshot and give the Thought and Action." }, { "from": "assistant", "value": "Thought: ... \nAction: ...", "score": 9, "judge": "Detailed step-by-step evaluation of the action's quality..." }, ... ], "images": [ "screenshots/task_screenshot0.png", "screenshots/task_screenshot1.png", "screenshots/task_screenshot2.png", ... ] } ``` **Note on Images**: - Each user turn includes an `` tag representing a screenshot of the current state of the web browser/application - The `images` key contains a list of file paths to all screenshots in the trajectory, ordered sequentially - The i-th screenshot in the `images` list corresponds to the i-th `` tag in the conversation - Screenshots are provided as separate PNG files referenced by these paths ### Fields - **system** (str): System prompt defining the agent's role, available actions, and output format - **conversations** (list): Alternating user requests and assistant responses - **role/from** (str): Speaker identifier ("user" or "assistant") - **content/value** (str): Message content - **User messages**: Include an `` tag representing the screenshot of the current browser/application state, followed by a request to analyze and respond - **Assistant messages**: Include structured "Thought:" and "Action:" explaining reasoning and the chosen action - **score** (int, assistant only): Quality score from 1-10 for the proposed action - **judge** (str, assistant only): Detailed multi-step evaluation providing justification for the score - **images** (list of str): Ordered list of file paths to screenshots corresponding to each step in the trajectory. The i-th image corresponds to the i-th `` tag in the conversations. ### Visual Data Each trajectory is multimodal, combining: - **Text**: Task descriptions, thoughts, actions, and evaluations - **Screenshots**: Visual state at each step (indicated by `` tags) The first user message contains the initial task and starting screenshot. Subsequent user messages contain follow-up screenshots after each action is executed, allowing the agent to observe the results of its actions. ### Action Space The dataset includes the following action types: - `click(point='x y')`: Click at coordinates - `left_double(point='x y')`: Double-click - `right_single(point='x y')`: Right-click - `drag(start_point='x1 y1', end_point='x2 y2')`: Drag operation - `hotkey(key='ctrl c')`: Keyboard shortcuts - `type(content='xxx')`: Text input - `scroll(point='x y', direction='down/up/right/left')`: Scrolling - `wait()`: Wait 5 seconds - `finished(content='xxx')`: Task completion with answer ## Step-Level Scoring System Each action is evaluated on a 1-10 scale: - **9-10**: Optimal action that directly advances the task with no better alternatives - **7-8**: Correct action that makes meaningful progress - **5-6**: Partially helpful action or correct action with strictly better alternatives - **3-4**: Off-target action that provides minimal progress - **1-2**: Incorrect action that wastes time or moves away from the goal ## Unzipping the Screenshots Due to large number of images and for easier upload/download from huggingface, we have zipped the screenshots using the script (`organize_and_zip_screenshots.py`). Before using the dataset you need to unzip the screenshots using the included `unzip_screenshots.py` script. ## Unzipping the Trajectories Due to large number of trajectory files and for easier upload/download from huggingface, we have zipped the trajectories using the script (`zip_trajectories.py`). Before using the dataset you need to unzip the trajectories using the included `unzip_trajectories.py` script. ## Example Use Case ```python import json # Load a trajectory with open('taskAllrecipes--20-2.json', 'r') as f: trajectory = json.load(f) # Extract high-quality steps (score >= 8) high_quality_steps = [ conv for conv in trajectory['conversations'] if conv.get('from') == 'assistant' and conv.get('score', 0) >= 8 ] print(f"Total steps: {len([c for c in trajectory['conversations'] if c.get('from') == 'assistant'])}") print(f"High-quality steps: {len(high_quality_steps)}") ``` ## Dataset Statistics - **Domain**: Web navigation and interaction tasks - **Task Types**: Information retrieval, multi-step searches, form interactions - **Websites**: Allrecipes, and other common web platforms - **Average Trajectory Length**: Varies by task complexity - **Score Distribution**: Detailed distribution available in paper ## Citation If you use this dataset, please cite: ```bibtex @article{he2025scalable, title={Scalable Data Synthesis for Computer Use Agents with Step-Level Filtering}, author={He, Yifei and Chawla, Pranit and Souri, Yaser and Som, Subhojit and Song, Xia}, journal={arXiv preprint arXiv:2512.10962}, year={2025} } ``` ## License This dataset is released under the [Community Data License Agreement - Permissive - Version 2.0 (CDLA-Permissive-2.0)](https://cdla.dev/permissive-2-0/). The CDLA-Permissive-2.0 is a permissive license designed specifically for data sharing, allowing you to: - Use, modify, and share the data freely - Create and share results (including ML models) without restrictions - No obligation beyond including this license text when sharing the data See the [LICENSE.txt](LICENSE.txt) file for the complete license text. ## Contact For questions or issues regarding the dataset, please reach out to the authors listed in the paper. ## Acknowledgments This dataset incorporates queries from the [OpenWebVoyager](https://github.com/minorjerry/openwebvoyager) project, described in "Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization" (He et al., arXiv:2410.19609, 2024). We thank the authors for making their work available under the Apache License 2.0.