explaingit

bytedance/ui-tars-desktop

📈 Trending★ 34,624TypeScriptAudience · developerComplexity · 4/5ActiveLicenseSetup · moderate

TLDR

Open-source AI agents that watch and control graphical interfaces, web browsers, desktop apps, terminals, by seeing the screen and clicking or typing like a human would.

Mindmap

mindmap
  root((repo))
    What it does
      AI sees screens
      Clicks and types
      Automates GUI tasks
    Projects
      Agent TARS
      UI-TARS Desktop
    Interfaces
      Web browser
      Desktop apps
      Terminals
    How to use
      Natural language commands
      CLI or web UI
      Local or remote control
    Tech stack
      TypeScript
      MCP protocol
      Multimodal AI

Things people build with this

USE CASE 1

Automate multi-step web tasks like booking flights or filling forms by giving the AI a natural-language instruction.

USE CASE 2

Build GUI automation tools that control desktop applications or remote machines without needing API access.

USE CASE 3

Create AI agents that handle repetitive computer work, data entry, testing, screenshot analysis, by observing and interacting with screens.

Tech stack

TypeScriptMCPNode.jsReact

Getting it running

Difficulty · moderate Time to first run · 30min

Requires Node.js runtime and likely browser automation dependencies (Playwright/Puppeteer); may need system-level graphics/display setup for desktop UI control.

Open-source license allowing use, modification, and distribution for any purpose including commercial use.

In plain English

UI-TARS Desktop is an open-source stack of two related AI agent projects that let an AI model observe and interact with graphical interfaces, web browsers, desktop applications, and terminals, the same way a human user would, by looking at the screen and clicking or typing. The first project, Agent TARS, is a general-purpose multimodal AI agent (multimodal meaning it can process both text and visual information). It can be controlled through a command-line tool or a web-based interface, and it connects to external tools via MCP (a protocol for giving AI agents access to real-world capabilities). You can give it a natural-language instruction like "book the earliest flight from X to Y on this website" and it will navigate a browser to complete the task. The second project, UI-TARS Desktop, is a desktop application built on a specific AI model called UI-TARS. It provides local or remote operators for computers and browsers, meaning it can control either the machine it runs on or a remote machine. Both projects are written in TypeScript and target developers and researchers building or experimenting with GUI automation agents, software that automates tasks by operating graphical interfaces rather than APIs. Someone would use this when they want an AI to perform multi-step computer tasks on their behalf, or when they are building agent-based automation tooling.

Copy-paste prompts

Prompt 1
How do I set up Agent TARS to automate a web booking task using natural language commands?
Prompt 2
Show me how to connect UI-TARS Desktop to control a remote machine and have it perform a multi-step GUI task.
Prompt 3
What's the difference between Agent TARS and UI-TARS Desktop, and which should I use for automating desktop application workflows?
Prompt 4
How do I integrate MCP tools with Agent TARS to give my AI agent access to external capabilities?
Open on GitHub → Explain another repo

Generated 2026-05-18 · Model: sonnet-4-6 · Verify against the repo before relying on details.