Awesome
nut.js (Native UI Toolkit)
GitHub Actions | |
---|---|
Master | |
Develop |
<a href="https://console.dev" title="Visit Console - the best tools for developers"><img src="https://console.dev/img/badges/1.0/svg/console-badge-logo-dark.svg" alt="Console - Developer Tool of the Week" /></a>
<p align="center"> Please visit </p> <h1 align="center"><a href="https://nutjs.dev">nutjs.dev</a></h1> <p align="center"> for detailed documentation and tutorials </p> <br/>About
<p align="center"> <img src="https://github.com/nut-tree/nut.js/raw/master/.gfx/nut.png" alt="logo" width="200"/> </p>nut.js
is a cross-platform native UI automation / testing tool.
It allows for native UI interactions via keyboard and / or mouse, but additionally gives you the possibility to navigate the screen based on image matching.
Sponsoring
nut.js
is developed with community in mind.
A huge "Thank you!" goes out to all sponsors who make open source a bit more sustainable!
Demo
Check out this demo video to get a first impression of what nut.js is capable of.
Tutorials
Please consult the project website at nutjs.dev for in-depth tutorials
API Docs
nut.js provides public API documentation auto-generated by TypeDoc.
Community
Feel free to join our Discord community
Modules
This list gives an overview on currently implemented and planned functionality. It's work in progress and will undergo constant modification.
Clipboard
- Copy text to clipboard
- Paste text from clipboard
Keyboard
- Support for standard US keyboard layout
- Support for multimedia keys
Mouse
- Support for mouse movement
- Support for mouse scroll
- Configurable movement speed
- Mouse drag
Window
- List all windows
- Retrieve active window
- Retrieve window title
- Retrieve window size and position
Screen
- Retrieve RGBA color information on screen
- Highlighting screen regions
- Find a single or multiple occurrences of an image on screen (requires an additional provider package like e.g. nut-tree/template-matcher)
- Wait for an image to appear on screen (requires an additional provider package like e.g. nut-tree/template-matcher)
- Find a single or multiple occurrences of text on screen (*)
- Wait for a piece of text to appear on screen (*)
- Find a single or multiple windows on screen (*)
- Wait for a window to appear on screen (*)
- Hooks to trigger actions based on detected text, images or windows (*)
(*) Requires an additional provider package, visit nutjs.dev for more info
Integration
- Jest
- Electron
- Custom log integration
Sample
The following snippet shows a valid nut.js
example (using multiple addons):
"use strict";
const {
mouse,
screen,
singleWord,
sleep,
useConsoleLogger,
ConsoleLogLevel,
straightTo,
centerOf,
Button,
getActiveWindow,
} = require("@nut-tree/nut-js");
const {
preloadLanguages,
Language,
LanguageModelType,
configure,
} = require("@nut-tree/plugin-ocr");
configure({ languageModelType: LanguageModelType.BEST });
useConsoleLogger({ logLevel: ConsoleLogLevel.DEBUG });
screen.config.autoHighlight = true;
screen.config.ocrConfidence = 0.8;
function activeWindowRegion() {
return getActiveWindow().then((activeWindow) => activeWindow.region);
}
(async () => {
await preloadLanguages([Language.English], [LanguageModelType.BEST]);
await sleep(5000);
const result = await screen.find(singleWord("@nut-tree/nut-js"));
await mouse.move(straightTo(centerOf(result)));
await mouse.click(Button.LEFT);
await screen.waitFor(singleWord("Native"), 15000, 1000, {
providerData: { partialMatch: true },
});
const content = await screen.read({ searchRegion: activeWindowRegion() });
console.log(content);
})();
Installation
Prerequisites
This section lists runtime requirements for nut.js
on the respective target platform.
Windows
In case you're running Windows 10 N and want to use ImageFinder plugins, please make sure to have the Media Feature Pack installed.
macOS
On macOS, Xcode command line tools are required. You can install them by running
xcode-select --install
Permissions:
nut.js requires the executing application, e.g. your terminal, to be given both Accessibility
and Screen Recording
permissions.
Starting with release 2.3.0
, nut.js will check for and request these permissions automatically:
It will also give you a subtle hint in case permissions are lacking:
- Accessibility:
##### WARNING! The application running this script is not a trusted process! Please visit https://github.com/nut-tree/nut.js#macos #####
- Screen Recording:
##### WARNING! The application running this script is not allowed to capture screen content! Please visit https://github.com/nut-tree/nut.js#macos #####
Attention:
Prior to release 2.3.0
you'll have to grant these permissions manually.
Settings -> Security & Privacy -> Privacy tab -> Accessibility -> Add...
For example, if you want to execute your node script in e.g. iTerm2
, you'd have to add iTerm.app
to the list.
When running your script from a built-in terminal in e.g. VSCode
or IntelliJ
, you'd have to add the respective IDE.
Linux
Depending on your distribution, Linux setups may differ.
In general, nut.js
requires
- libXtst
Installation on *buntu
distributions:
sudo apt-get install libxtst-dev
Setups on other distributions might differ.
Attention:
At the moment nut.js only supports X11.
Wayland is NOT supported!
On e.g. Ubuntu you can switch to XWayland on your login screen as a workaround.
Install nut.js
Running
npm i @nut-tree/nut-js
or
yarn add @nut-tree/nut-js
will install nut.js
and its required dependencies.
Snapshot releases
nut.js
also provides snapshot releases which allows to test upcoming features.
Running
npm i @nut-tree/nut-js@next
or
yarn add @nut-tree/nut-js@next
will install the most recent development release of nut.js
.
Attention: While snapshot releases are great to work with upcoming features before a new stable release, it is still a snapshot release. Please bear in mind that things might change and / or break on snapshot releases, so it is not recommended using them in production.