Awesome
clj-chrome-devtools
clj-chrome-devtools is a simple library for controlling a headless Chrome with the Chrome DevTools Protocol. The protocol is based on a websocket connection between clojure and chrome. All the functions are automatically generated from the protocol specification JSON files.
Quick start with Babashka
See bb.clj file for a quick script that can be used from Babashka.
Goals
The goal of the project is to provide a general purpose library for using headless chrome, both in production use and in testing scenarios.
The CDP part, which is autogenerated, should be stable and will be updated when CDP protocol changes.
The higher level automation utilities is where most of the work is done and new automation utilities are welcome. PRs welcome, just follow the common pattern of providing two arities: one for using the global automation context and one where it is provided as the first parameter.
API Docs
See codox generated API docs.
All the low-level auto-generated commands from Chrome Devtools Protocol are in clj-chrome-devtools.command.*
namespaces. Note: to use the low-level API you need to implement event handlers for listening to data Chrome sends
the client.
There is the beginnings of a rudimentary higher level API in clj-chrome-devtools.automation.
There is also a clojure.test fixture to run tests with a fresh headless chrome in clj-chrome-devtools.automation.fixture.
Example usage
For using the higher level API for screen scraping and browser testing, see chrome_test.clj file.
The following shows a simple REPL usage, navigating to a page and inspecting content.
The connection is the first parameter of all calls. You can also set the connection to use with
set-current-connection! and omit the connection parameter for convenience.
clj-chrome-devtools.core> (def c (connect "localhost" 9222))
#'clj-chrome-devtools.core/c
clj-chrome-devtools.core> (require '[clj-chrome-devtools.commands.page :as page]
'[clj-chrome-devtools.commands.dom :as dom])
nil
clj-chrome-devtools.core> (page/navigate c {:url "http://webjure.org/"})
{:frame-id "68439.1"}
clj-chrome-devtools.core> (dom/get-document c {:depth 1})
{:root
{:children
[{:node-type 1,
:node-id 2,
:backend-node-id 4,
:parent-id 1,
:node-name "HTML",
:node-value "",
:frame-id "68439.1",
:local-name "html",
:child-node-count 2,
:attributes []}],
:document-url "http://webjure.org/",
:node-type 9,
:base-url "http://webjure.org/",
:node-id 1,
:backend-node-id 3,
:node-name "#document",
:node-value "",
:xml-version "",
:local-name "",
:child-node-count 1}}
clj-chrome-devtools.core> (use 'clojure.repl)
nil
clj-chrome-devtools.core> (doc dom/get-outer-html)
-------------------------
clj-chrome-devtools.commands.dom/get-outer-html
([] [{:as params, :keys [node-id]}] [connection {:as params, :keys [node-id]}])
Returns node's HTML markup.
Parameters map keys:
:node-id Id of the node to get markup for.
Return map keys:
:outer-html Outer HTML markup.
nil
clj-chrome-devtools.core> (dom/get-outer-html c {:node-id 1})
{:outer-html
"<html><head>\n <title>Webjure</title>\n </head>\n <body>\n Coming soon-ish!\n \n\n</body></html>"}
Running ClojureScript test suite with clj-chrome-devtools
The clj-chrome-devtools.cljs.test contains a function that can be used to build and run
ClojureScript tests as part of your Clojure test without needing doo plugin or karma installed.
The build-and-test function is meant to be called inside your test and it will build the specified
ClojureScript build that is defined in project.clj (e.g. "test") with an added test runner that
requires the specified namespaces and runs tests in them.