explaingit

cyrildiagne/ar-cutpaste

Analysis updated 2026-06-24

★ 14,590TypeScriptAudience · researcherComplexity · 4/5Setup · hard

TLDR

Research prototype that lets you point a phone camera at a real object, cut it out, and paste it as a layer into Photoshop on your computer.

Mindmap

mindmap
  root((ar-cutpaste))
    Inputs
      Phone camera view
      Photoshop document
    Outputs
      New PS layer
      Background removed image
    Use Cases
      AR research demo
      Designer experiments
      ML pipeline study
    Tech Stack
      TypeScript
      Python
      BASNet
      Photoshop API
Click or tap to explore — scroll the page freely

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Reproduce the viral AR cut and paste demo as a learning project for AR plus ML pipelines.

USE CASE 2

Study how a phone, a local Python server and Photoshop talk to each other in a real prototype.

USE CASE 3

Swap the BASNet background-removal step for your own segmentation model to test it end-to-end.

USE CASE 4

Build a similar bridge between a mobile camera app and a desktop creative tool.

What is it built with?

TypeScriptPythonBASNetPyTorch

How does it compare?

cyrildiagne/ar-cutpastevuejs/piniajimp-dev/jimp
Stars14,59014,58314,605
LanguageTypeScriptTypeScriptTypeScript
Last pushed——2026-04-07
Maintenance——Maintained
Setup difficultyhardeasyeasy
Complexity4/52/52/5
Audienceresearcherdeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · hard Time to first run · 1day+

Needs Photoshop with Remote Connection on, exact document size match, a textured background for SIFT matching, plus a GPU or hosted BASNet endpoint.

In plain English

AR Cut and Paste is a research prototype that lets you point your phone camera at a real object in the room, cut it out of the live view, and have it appear as a new layer in Photoshop on your computer. The README is clear that this is a research demo, not a polished product. It combines augmented reality on the phone with machine learning that figures out which pixels belong to the object you want, then a small server that hands the result over to Photoshop. The README points readers who want a finished consumer tool toward clipdrop.co, which the same author started as a follow-up. For now, this repo only talks to Photoshop, though the project notes other outputs may be supported later. A Twitter thread linked in the README has a video demo of the workflow. The code is split into three modules. The mobile app, under the /app folder, runs on a phone and provides the camera view that the user aims at the object. The local server, under /server, is the glue between the phone and Photoshop. It uses a helper library called screenpoint to figure out where on the Photoshop document the phone camera is currently pointing. The third piece is an external object detection and background removal service, which receives an image and returns the same image with everything except the chosen object cut away. Setting it up takes several steps. In Photoshop you open Preferences, then Plug-ins, enable Remote Connection, and set a password that the local server will use to talk to it. The document size in Photoshop has to match settings in server/src/ps.py, or only an empty layer arrives. The README also warns that the document needs some background texture, because the SIFT feature matcher inside screenpoint will fail to align the camera view against a blank canvas. For the background removal model you have two options. You can host the BASNet model yourself using a BASNet-HTTP wrapper, which requires a CUDA GPU, and pass the resulting URL into the local server. Or you can use a community-provided endpoint hosted on CoreWeave, passed in with the --basnet_service_ip flag. After that, you follow the per-folder READMEs to start the local server and deploy the app. The README ends with thanks to the BASNet authors, to RunwayML for the Photoshop paste code, and to CoreWeave for hosting the public endpoint.

Copy-paste prompts

Prompt 1
Walk me through setting up ar-cutpaste on a Mac with Photoshop, including the Remote Connection password and matching ps.py document size.
Prompt 2
Replace the BASNet background-removal step in ar-cutpaste with a call to a modern segmentation model like SAM, and update the local server.
Prompt 3
Explain how the screenpoint library uses SIFT to align the phone camera view with the Photoshop canvas, and why a blank canvas fails.
Prompt 4
Containerize the ar-cutpaste local server with Docker so I can run it without installing Python deps system-wide.
Prompt 5
Adapt the ar-cutpaste server so the cutout target is Figma or Procreate instead of Photoshop, sketching the API calls needed.

Frequently asked questions

What is ar-cutpaste?

Research prototype that lets you point a phone camera at a real object, cut it out, and paste it as a layer into Photoshop on your computer.

What language is ar-cutpaste written in?

Mainly TypeScript. The stack also includes TypeScript, Python, BASNet.

How hard is ar-cutpaste to set up?

Setup difficulty is rated hard, with roughly 1day+ to a first successful run.

Who is ar-cutpaste for?

Mainly researcher.

Open on GitHub → Explain another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.

Alternatives & similar repos