RE: LeoThread 2025-01-04 08:14
You are viewing a single comment's thread:
OS-Genesis: A Novel GUI Data Synthesis Pipeline that Reverses the Conventional Trajectory Collection Process
Link below
0
0
0.000
You are viewing a single comment's thread:
Link below
https://www.marktechpost.com/2025/01/03/os-genesis-a-novel-gui-data-synthesis-pipeline-that-reverses-the-conventional-trajectory-collection-process/?utm_source=flipboard&utm_content=topic/technology
Collecting quality data for training GUI agents is challenging due to reliance on human supervision or rigid synthetic methods, which limit diversity and adaptability.
OS-Genesis, proposed by leading institutions, reverses traditional methods by using interaction-driven reverse task synthesis to create high-quality training data.
Agents autonomously interact with GUIs, recording actions like clicks and typing, which are then analyzed to generate structured, task-based training data.
Using the Trajectory Reward Model (TRM), synthesized data is scored on coherence, logic, and completeness, ensuring diverse, high-quality trajectories for training.
Tests on benchmarks like AndroidWorld and WebArena show OS-Genesis improves task planning, action execution, and handles dynamic environments with robustness.
By bridging abstract task instructions and dynamic GUIs, OS-Genesis enables agents to learn autonomously, advancing digital automation and GUI-focused AI.
OS-Genesis sets a new standard for GUI agent training, eliminating human dependency and enhancing adaptability, paving the way for smarter AI systems.