Awesome
Magic Copy
Magic Copy is a Chrome extension that uses Meta's Segment Anything Model to extract a foreground object from an image and copy it to the clipboard.
https://user-images.githubusercontent.com/511342/234897336-f2441e00-06e0-4ca8-9b97-601210879cdb.mp4
Installation
(This might not be available yet, as the extension is still in review. If you would like to be notified when they do, subscribe to the Chrome or Firefox issues.)
Alternatively, the extension can be installed manually:
- Download the latest
magic-copy.zip
from releases. - Extract the ZIP file.
- In Chrome, go to
chrome://extensions/
, enable "Developer mode", and click "Load unpacked". - Select the folder where the extension was extracted.
Figma
Magic Copy is also available as a Figma plugin.
Implementation
This extension uses the same procedure as the Segment Anything Model demo to extract a foreground object from an image. The only difference is that the extracted object is copied to the clipboard instead of being displayed on the page.
Building
Manually
Build the extension with npm
and then run the included ./buildcrx.sh
script to generate the crx
file:
npm ci
npm run build
./buildcrx.sh -d dist
Docker
A Dockerfile
is provided to cleanly build the crx
file. To build the extension, run:
docker build --output out .
Self-hosting
The Meta Segment Anything Model requires running the vision transformer on a server to generate the image's embeddings. Magic Copy uses the same service that their demo uses, however some people may not want to send their images to a third party.
The server-example
directory contains a simple example of how to self-host the
vision transformer service. It is not meant to be used in production, but rather as
a proof of concept to document the input/output format of the service.
In particular, Magic Copy (and the SAM demo) expect a POST endpoint that accepts
an image file and returns a JSON array of length 1 with the embedding of shape
(1, 256, 64, 64)
as a base64 encoded string. See the code for specific details on how
to perform this encoding to be compatible with the demo.
If you are looking to quickly get the service running, you can use the provided
Dockerfile
to build a container and run it. The container will expose port 8000
and will serve the service at the /
endpoint.
docker build -t segment-anything .
docker run --gpus all -p 8000:8000 segment-anything
In the Magic Copy chrome extension, you can then change the endpoint to http://localhost:8000/
.