Home

Awesome

IntelliJ Plugin Verifier

official JetBrains project X Follow Slack

IDE Diff Builder IntelliJ Feature Extractor IntelliJ Plugin Structure IntelliJ Plugin Verifier Plugins Verifier Service

IntelliJ Plugin Verifier checks the binary compatibility between IntelliJ-based IDE builds and IntelliJ Platform plugins.

This tool is useful because plugin authors often specify a wide [since; until] compatibility range but compile a plugin against only a specific IDE from the range. The IntelliJ Platform API can occasionally change between releases, so binary incompatibilities may arise, leading to NoClassDefFoundError, NoSuchMethodError, and similar exceptions at runtime.

[!TIP] In most cases, IntelliJ Plugin Verifier will be used via verifyPlugin (2.x) / runPluginVerifier (1.x) task from Gradle plugin, see Integration.

Example problems the Plugin Verifier can detect:

  1. Plugin references a class com.example.Foo, which is not available in the IDE. This can happen if the plugin was compiled against IDE v1.0, and the class com.example.Foo was removed in IDE v2.0.
  2. Plugin references a missing method of the IDE's class, which leads to NoSuchMethodError at runtime.
  3. Many other binary incompatibilities as listed in Java Specification | Binary Compatibility.
  4. Missing plugin dependencies, for example, when plugin A depends on plugin B, but plugin B doesn't have a build that’s compatible with this IDE. It means that the user cannot install plugin A as the IDE requires all dependent plugins to be installed.

Table of Contents

Installation

Download the latest available verifier-cli-<version>-all.jar from the JetBrains Package Repository or from Maven Central.

As an alternative, use curl to download the JAR archive from the command-line:

curl -L -o verifier-all.jar https://packages.jetbrains.team/maven/p/intellij-plugin-verifier/intellij-plugin-verifier/org/jetbrains/intellij/plugins/verifier-cli/<version>/verifier-cli-<version>-all.jar

The <version> is the latest version, which you can find in the package page or in the GitHub Releases.

You can use the GitHub API to retrieve the most recent release information. The resulting JSON response can be parsed using the jq tool. Following this, the artifact URL can be supplied to curl for download.

curl -s https://api.github.com/repos/JetBrains/intellij-plugin-verifier/releases/latest \
    | jq -r '.assets[].browser_download_url' \
    | xargs curl -L --output verifier-all.jar

Options

The Plugin Verifier can be run using the command line:

java -jar verifier-all.jar [Command] [Options]

Command is one of check-plugin, check-ide or check-trunk-api.

Beginning with version 1.260, the Plugin Verifier requires Java 11. Prior to this version, Java 8 is required.

Results

All the verification results are printed and saved in the following ways:

  1. The results are saved to <verification-$timestamp> directory (can be changed with -verification-reports-dir option). The layout of files beneath this directory is as follows. The format of individual files is not specified. Basically, the files contain human-readable sentences.

    <verification reports dir>/
    <IDE version #1>/
        plugins/
            <ID of plugin #1/
                <Version of plugin #1>/
                    <... report files ...>
            <ID of plugin #2>/
                ...
    <IDE version #2>/
        plugins/
            ...
    FileDescriptionExists if
    verification-verdict.txtHuman-readable verification verdict.Always
    dependencies.txtDependencies of the plugin used during verification.Plugin is valid
    compatibility-warnings.txtCompatibility warnings of this plugin with the IDE.> 0
    compatibility-problems.txtCompatibility problems of this plugin with the IDE.> 0
    deprecated-usages.txtDescriptions of "Deprecated API is used" cases.> 0
    experimental-api-usages.txtDescriptions of "Experimental API is used" cases.> 0
    internal-api-usages.txtDescriptions of "Internal API is used" cases.> 0
    override-only-usages.txtDescriptions of "Override-only API is used incorrectly" cases.> 0
    non-extendable-api-usages.txtDescriptions of "Non-extendable API is used incorrectly" cases.> 0
    plugin-structure-warnings.txtDescriptions of plugin's own problems, which are not related to IDE compatibility.> 0
    invalid-plugin.txtDescription of the invalid plugin error, in case the plugin is invalid.Plugin is invalid

    Note! If you are implementing integration with the Plugin Verifier, you may check the presence of corresponding files to distinguish "successful" and "failed" verifications.

  2. If the -teamcity (-tc) option is specified, the results are printed in TeamCity Tests Format. To choose a presentation type, specify the -tc-grouping (-g) option to either plugin, to group by each plugin, or problem_type, to group by the problem.

  3. If -teamcity isn't specified, the results are printed to console.

Commands

check-ide

This command is used to check IDE build against a set of plugins.

check-ide
    <IDE>
    [-runtime-dir | -r <file>]
    [-plugins-to-check-file | -ptcf <file>]
    [-plugins-to-check-all-builds | -p-all < ':'-separated list>]
    [-plugins-to-check-last-builds | -p-last < ':'-separated list>]
    [-excluded-plugins-file | -epf <file> ]
    [-team-city | -tc ]
    [-tc-grouping | -g ]
    [-external-prefixes <':'-separated list>]
    [-dump-broken-plugin-list | -d]
    [-ignored-problems | -ip <file>]
    [-keep-only-problems | -kop <file>]

<IDE> is either a path to local IDE installation, or an IDE pattern (see below in the common options).

If no plugins are explicitly specified, then all compatible plugins in the Plugin Repository will be verified (options).

Examples

Check IDEA Ultimate #162.1121.32 against all plugins listed in pluginsToCheck.txt:

java -jar verifier-all.jar -runtime-dir /home/user/.jdks/corretto-11.0.8 -team-city -tc-grouping problem_type -excluded-plugins-file ignorePlugins.txt -plugins-to-check-file pluginsToCheck.txt -dump-broken-plugin-list actualBroken.txt check-ide /tmp/IU-162.1121.32

Check IDEA Ultimate 162.1121.32 against all versions of Kotlin and NodeJs plugins and the last version of the PHP plugin:

java -jar verifier-all.jar -runtime-dir /home/user/.jdks/corretto-11.0.8 -plugins-to-check-all-builds org.jetbrains.kotlin:NodeJS -plugins-to-check-last-builds com.jetbrains.php check-ide /tmp/IU-162.1121.32

check-plugin

This command is used to check one or more plugins against one or more IDEs (options).

check-plugin
    <plugins>
    <IDE> [<IDE>]*
    [-runtime-dir | -r <file>]
    [-team-city | -tc ]
    [-tc-grouping | -g ]
    [-external-prefixes <':'-separated list>]
    [-suppress-internal-api-usages no|jetbrains-plugins]
    [-mute comma-separated plugin problem identifier list]

<plugins> is either <plugin path> or '@<file>' with a list of plugin paths to verify, separated by a newline.

<IDE> is either a path to local IDE installation, or an IDE pattern (see below in the common options).

Specific options

Plugin Path Specification

Plugin path may be specified in multiple formats:

Examples

Check Kotlin plugin against IDEA Ultimate 162.2032.8, 163.1024, and 163.7277:

java -jar verifier-all.jar -runtime-dir /home/user/.jdks/corretto-11.0.8 check-plugin /tmp/Kotlin /tmp/IU-162.2032.8 /tmp/IU-163.1024 /tmp/IU-163.7277

Check an individual plugin packaged as a ZIP file against IDEA Ultimate 162.2032.8 on macOS:

 java -jar verifier-all.jar check-plugin ~/counter/build/distributions/counter-12.0.0.zip /tmp/IU-162.2032.8/Contents

Check all versions of plugin with ID training against IDEA Ultimate 162.2032.8 and PyCharm PC-203.7717.81 on macOS:

java -jar verifier-all.jar check-plugin training /tmp/idea-edu-2022.2.2/Contents /tmp/PC-203.7717.81.app/Contents

Check all versions of plugin with ID training and version 231.8770.31 against IDEA Ultimate 162.2032.8 and PyCharm PC-203.7717.81 on macOS:

java -jar verifier-all.jar check-plugin version:training:231.8770.31 /tmp/idea-edu-2022.2.2/Contents /tmp/PC-203.7717.81.app/Contents

Check a specific plugin update by its ID against IDEA Ultimate 162.2032.8 and PyCharm PC-203.7717.81 on macOS. The update #323705 corresponds to the plugin training with version version 231.8770.31.

java -jar verifier-all.jar check-plugin '#323705' /tmp/idea-edu-2022.2.2/Contents tmp/PC-203.7717.81.app/Contents

Note that the update ID is quoted to prevent shell mangling.

Check a list of plugins specified in a plaintext file jetbrains-plugins.txt against IDEA Ultimate 162.2032.8 and PyCharm PC-203.7717.81 on macOS.

java -jar verifier-all.jar check-plugin @jetbrains-plugins.txt /tmp/idea-edu-2022.2.2/Contents /tmp/PC-203.7717.81.app/Contents

The file jetbrains-plugins.txt might contain two plugin paths on separate lines.

version:com.intellij.quarkus:231.8770.17
#320655

check-trunk-api

This command tracks API Changes between two IDE builds: a release and a trunk.

Note that its purpose is to detect incompatibilities between two IDE builds, not to detect all the plugins' own problems.

Given the release IDE build, all plugins' versions for release IDE will be verified with both the release and trunk and only new problems with a trunk will be reported.

For clarity, here is an example of the command:

check-trunk-api
    -r /home/user/.jdks/corretto-11.0.8
    -subsystems-to-check without-android
    -team-city
    -jetbrains-plugins-file all-jetbrains-plugins.txt
    -release-jetbrains-plugins release-plugins
    -trunk-jetbrains-plugins trunk-plugins
    -major-ide-path IU-173.4548.28
    IU-181.3741.2

The IU-173.4548.28 is IDEA Ultimate 2017.3.4 build, and IU-181.3741.2 is some IDE built from the master. This command will do the following:

  1. Take all plugins from the Plugin Repository compatible with IU-173.4548.28 and run the verification against IU-173.4548.28.
  2. Take the same versions of the plugins and verify them against IU-181.3741.2, even if those plugins' [since; until] compatibility ranges don't include the IU-181.3741.2.
  3. Report problems that are present in IU-181.3741.2 but not present in IU-173.4548.28.

There are the following points to mention:

  1. IntelliJ API is considered to consist of all the classes bundled to IDE, all its bundled plugins, and all the JetBrains-developed plugins compiled from the same sources revision. Those plugins are not bundled into IDE distribution but are available locally after the IDE build finishes. The plugins are typically uploaded to the Plugin Repository when a new release IDE gets out, but it cannot be valid for intermediate builds.
    • -jetbrains-plugins-file all-jetbrains-plugins.txt points to a file containing IDs of all JetBrains plugins built from the same sources as the IDE, like (NodeJS, Pythonid, Docker).
    • -release-jetbrains-plugins release-plugins points to a directory containing all the plugins built along with the release IDE. Plugins can be in the form of .jar, .zip, or directories.
    • -trunk-jetbrains-plugins trunk-plugins points to a directory containing all the plugins built along with the trunk IDE.
  2. -subsystems-to-check without-android specifies that the Plugin Verifier should not show problems related to Android support.

Here is the full syntax of the command:

check-trunk-api <trunk IDE>
    [-runtime-dir | -r <file>]
    [-major-ide-path | -mip <file>]
    [-external-prefixes <':'-separated list>]
    [-subsystems-to-check | -subsystems]
    [-release-jetbrains-plugins | -rjbp <path>]
    [-trunk-jetbrains-plugins | -tjbp <path>]
    [-team-city | -tc ]

Specific options

Common Options

Technical details

Plugin Verifier uses the following paths for operational purposes:

Downloading plugins

Plugins to be verified and plugins' dependencies are downloaded into <plugins-directory>. It can be reused between multiple runs of the Plugin Verifier: on the first run, all the necessary plugins will be downloaded, and on the subsequent runs, they will be taken from the cache. Note that not only the verified plugins are downloaded but also all plugins' dependencies.

Plugins are downloaded from the Plugin Repository into <plugins-directory>/<update-ID>.jar or <plugins-directory>/<update-ID>.zip, depending on the plugin's packaging type. <update-ID> is the unique ID of the plugin's version in the Plugin Repository's database. For example, Kotlin 1.2.30-release-IJ2018.1-1 has update-ID equal to 43775.

Limit size of <plugins-directory>

That's possible to limit the size of the <plugins-directory>, which is 5 GB by default. To do this, specify JVM option -Dplugin.verifier.cache.dir.max.space=<max-space-MB>. The Plugin Verifier will remove the least recently used plugins from the cache as soon as the occupied space reaches the limit.

Extracting .zip-ed plugins

Plugins packaged in .zip archives are extracted into <extracted-directory>/<temp-dir> before verifying these plugins starts. This is necessary to speed up the verification, which needs to do many searches of class-files.

Integration

The most straightforward way of integrating the Plugin Verifier with your project is using dedicated Gradle task:

If you're not using Gradle within your project, there are predefined third-party actions available in the GitHub Actions Marketplace that automate the plugin verification process.

Read more about possible integration options: Verifying Plugin Compatibility

Feedback

Please report issues to YouTrack: https://youtrack.jetbrains.com/issues/MP (MP stands for Marketplace)

Thank you in advance for reporting issues, providing feedback, and making the Plugin Verifier better!

Slack

There is also a dedicated Slack channel available: #intellij-plugin-verifier.

The JetBrains Platform Slack community is a place where you can talk with other plugin developers and JetBrains employees about plugin and extension development.