Home

Awesome

IndexedDB Polyfill

Build Status Dependencies devDependencies npm CDNJS

Tests badge Coverage badge

Known Vulnerabilities Total Alerts Code Quality: Javascript

License

Licenses badge

<!-- [![Licenses dev badge](https://raw.githubusercontent.com/indexeddbshim/indexeddbshim/master/badges/licenses-badge-dev.svg?sanitize=true)](badges/licenses-badge-dev.svg) -->

(see also licenses for dev. deps.)

<!--[![issuehunt-to-marktext](https://issuehunt.io/static/embed/issuehunt-button-v1.svg)](https://issuehunt.io/r/indexeddbshim/indexeddbshim)-->
Live Demo (stable)!Live Demo (master)!

Use a single, indexable, offline storage API across all desktop and mobile browsers and Node.js.

Even if a browser natively supports IndexedDB, you may still want to use this shim. Some native IndexedDB implementations are very buggy. Others are missing certain features. There are also many minor inconsistencies between different browser implementations of IndexedDB, such as how errors are handled, how transaction timing works, how records are sorted, how cursors behave, etc. Using this shim will ensure consistent behavior across all browsers.

Features

Installation

You can download the development or production (minified) script, or install it using NPM.

For Mac, you may need to have CMake installed for the SQLite3 install to work (See Tools->How to Install For Command Line Use) as well as build SQLite3 from source via npm install --build-from-source in the node-sqlite3 directory. Also make sure Python (2.7) is installed.

npm

npm install indexeddbshim

or

yarn add indexeddbshim

Browser set-up

Add the following scripts to your page:

<script src="./node_modules/core-js-bundle/minified.js"></script>
<script src="./node_modules/indexeddbshim/dist/indexeddbshim.min.js"></script>

If you need full Unicode compliance (handling special non-alphanumeric identifiers in store and index names), use the following instead:

<script src="./node_modules/core-js-bundle/minified.js"></script>
<script src="./node_modules/indexeddbshim/dist/indexeddbshim-UnicodeIdentifiers.min.js"></script>

Node set-up

const setGlobalVars = require('indexeddbshim');

global.window = global; // We'll allow ourselves to use `window.indexedDB` or `indexedDB` as a global
setGlobalVars(); // See signature below

ES6 Modules

Bundler for Browser

import setGlobalVars from 'indexeddbshim';

Bundler for Node

import setGlobalVars from 'indexeddbshim/src/node-UnicodeIdentifiers';

// Or without Unicode support
// import setGlobalVars from 'indexeddbshim/src/node';

Usage/API

For the browser scripts, if the browser already natively supports IndexedDB and is not known to be buggy, then the script won't do anything.

Otherwise, assuming WebSQL is available, the script will add the IndexedDB API to the browser (unless you use one of the non-invasive files, in which case setGlobalVars can be used to optionally add the API to an object of your choosing; if you also wish Unicode support, you will need to add it yourself).

Either way, you can use IndexedDB just like normal. Here's an example.

setGlobalVars(<winObj or null>, initialConfig)

In the non-invasive builds (and Node.js), globals are not automatically set. You have the choice to set globals when you wish as well as to set the API on an object of your choosing in place of setting globals.

This is done through setGlobalVars() (which is otherwise called in the browser builds automatically with no arguments).

This function defines shimIndexedDB, indexedDB, IDBFactory, etc. on one of the following objects in order of precedence:

  1. The passed in winObj object if defined
  2. window (for Node, define global.window = global;)
  3. self (for web workers)
  4. global (for Node)
  5. A new empty object

The initialConfig argument, if present, should be an object whose keys are the config properties to set and its values are the config values (see shimIndexedDB.__setConfig below).

If you are adding your own window.openDatabase implementation, supplying it within initialConfig (keyed as openDatabase) will ensure that shimIndexedDB.__useShim() is auto-invoked for you if poor IndexedDB support is detected.

shimIndexedDB.__useShim();

To force IndexedDBShim to shim the browser's native IndexedDB (if our code is not already auto-shimming your browser when detecting poor browser support), add this method call to your script.

On browsers that support WebSQL, this line will completely replace the native IndexedDB implementation with the IndexedDBShim-to-WebSQL implementation.

On browsers that don't support WebSQL, but do support IndexedDB, this line will patch many known problems and add missing features. For example, on Internet Explorer, this will add support for compound keys.

If CFG.addNonIDBGlobals has been set (e.g., on the initialConfig argument of setGlobalVars), the other non-IndexedDB shims necessitated by this library will be polyfilled as possible on the chosen "global" (i.e., ShimEvent, ShimCustomEvent, ShimEventTarget, ShimDOMException, and ShimDOMStringList). Mostly useful for testing.

If CFG.replaceNonIDBGlobals is used, it will instead attempt to add, or if already present, overwrite these globals.

If CFG.fullIDLSupport has been set, the slow-performing Object.setPrototypeOf calls required for full WebIDL compliance will be used. Probably only needed for testing or environments where full introspection on class relationships is required. See this SO topic

shimIndexedDB.__forceClose([dbName], [connIdx], [msg])

The spec anticipates the closing of a database connection with a forced flag.

The spec also mentions some circumstances where this may occur:

A connection may be closed by a user agent in exceptional circumstances, for example due to loss of access to the file system, a permission change, or clearing of the origin’s storage.

Since the latter examples are under the browser's control, this method may be more useful on the server or for unit-testing.

If the first argument, dbName is missing (or null or undefined), all connections to all databases will be force-closed.

If the second argument, connIdx is missing (or null or undefined), all connections with the given name will be force-closed. It can alternatively be an integer representing a 0-based index to indicate a specific connection to close.

The third argument msg will be appended to the AbortError that will be triggered on the transactions of the connection.

Individual IDBDatabase database instances can also be force-closed with a particular message:

db.__forceClose(msg);

shimIndexedDB.__setConnectionQueueOrigin(origin = getOrigin())

Establishes a connectionQueue for the supplied (or current) origin.

The queue is otherwise only keyed to the detected origin on the loading of the IndexedDBShim script, though this is usually the desired behavior.

shimIndexedDB.__debug(boolean)

The IndexedDB polyfill has sourcemaps enabled, so the polyfill can be debugged even if the minified file is included.

To print out detailed debug messages, add this line to your script:

shimIndexedDB.__debug(true);

shimIndexedDB.__setConfig()

Rather than using globals, a method has been provided to share state across IndexedDBShim modules.

Configuration can be set early in the non-invasive browser and Node builds via the second argument to setGlobalVars() (see its definition above).

Its signature (for setting configuration after shimIndexedDB is created) is:

shimIndexedDB.__setConfig({
    property: value, property2: value2, ...otherProperties
});

or:

shimIndexedDB.__setConfig(property, value);

createDOMException(name, message)

A utility for creating a DOMException instance. Attempts to use any available native implementation.

Configuration options

The available properties relevant to browser or Node are:

The following config are mostly relevant to Node but has bearing on the browser, particularly if one changes the defaults.

Node-only config:

Node config mostly for development debugging:

shimIndexedDB.__getConfig()

For retrieving a config value:

shimIndexedDB.__getConfig(property);

Known Issues

All code has bugs, and this project is no exception. If you find a bug, please let us know about it. Or better yet, send us a fix! Please make sure someone else hasn't already reported the same bug though.

Here is a summary of main known issues to resolve:

  1. blocked and versionchange IDBVersionChangeEvent event support (#2 and #273) across processes/browser windows
  2. Some issues related to task/micro-task timing in Node (for inherent limitations in the browser, see below).
  3. ImageData storage on Node 14 when used with node-canvas - due to this issue

There are a few bugs that are outside of our power to fix. Namely:

Browser rollback

While we do try to rollback the database version in the browser when called for, as we are not able to prolong WebSQL transactions to benefit from the auto-rollback they perform upon encountering an error (nor does WebSQL permit manual ROLLBACK commands so that we could undo the various WebSQL calls we need to make up IndexedDB transactions), we are not able to provide safe rollbacks in the browser. The synchronous WebSQL API was not apparently well supported, at least it is missing in Safari and Chrome, and it would particularly degrade performance in a Node environment.

The special build of websql that we use does allow such IndexedDB-spec-compliant (and data-integrity-friendly!) rollback behavior in Node.

See below on task/micro-task timing for more.

Task/micro-task timing

IndexedDB transactions will timeout so long as there are no detected active requests.

While a single promise delay (a "microtask") is not supposed to be long enough to cause a transaction timeout (and they do not in Node where we have control over extending the transaction), it could possibly occur in our browser implementation.

(Note that chaining multiple promises or having a long-resolving promise will likely cause a transaction to expire even in compliant implementations.)

A setTimeout timeout of 0, on the other hand (a full "task"), ought, for compliant implementations, to be long enough of a time to cause a time out of the transaction, but in Node where we prolong transactions long enough to ensure our full chain of asynchronous SQL needed for the transaction operations is run (as well as ensure complete rollback should there be an error causing a transaction abort), it may be too short.

We could fix this in Node (where we can have access to a synchronous SQLite API such as https://github.com/grumdrig/node-sqlite unlike on the browser) and ensure transactions finish before the next task (though always after a microtask), but as mentioned above, this would degrade performance particularly on a server (and in the browser, the WebSQL API on which we are relying did not apparently gain support in browsers for the synchronous API).

This test and this one demonstrate the expected timeout behavior with regard to setTimeout or promises and transaction expiration.

Structured Cloning Algorithm

Due to certain challenges in detecting cloneable objects from within JavaScript, there are certain limitations regarding cloning:

  1. We cannot properly detect Proxy to throw upon encountering such non-cloneable objects
  2. Our reliance on Object.prototype.toString to detect uncloneable objects can fail if that method is overridden or if Symbol.toStringTag is used to change the default reporting of a given "class".
  3. Although they are currently working, we were only able to resolve Blob, File, and FileList objects synchronously (as required per spec) using the now-deprecated XMLHttpRequest synchronous API.
  4. Without a means of transferring ArrayBuffer objects in Node, we cannot meet the requirement to fail upon encountering detached binary objects.
  5. They may be other subtleties we have not been able to work around.

We have, however, overcome some cloning issues still faced by browser implementations, e.g., in Chrome (issue #698564) (re: not failing on WeakMap, WeakSet, Promise, and Object.prototype).

We also have limitations in creating certain objects synchronously, namely, the one method for creating an image bitmap, createImageBitmap, returns a Promise, so we cannot clone a bona fide image bitmap synchronously so as to obtain any errors synchronously as expected by the IndexedDB methods involving cloning.

Node versions 8.9.3 to 9.0.0

Our Mocha test "query multi-entry indexes with hundreds of records" of IDBIndex/openCursor-spec.js is failing for these versions. Starting with 9.1.0, however, the test passes again.

iOS

Due to a bug in WebKit, the window.indexedDB property is read-only and cannot be overridden by IndexedDBShim. There are two possible workarounds for this:

  1. Use window.shimIndexedDB instead of window.indexedDB
  2. Create an indexedDB variable in your closure

By creating a variable named indexedDB, all the code within that closure will use the variable instead of the window.indexedDB property. For example:

(function () {
    // This works on all browsers, and only uses IndexedDBShim as a final fallback
    var indexedDB = window.indexedDB || window.mozIndexedDB || // eslint-disable-line no-var -- Older browsers
        window.webkitIndexedDB || window.msIndexedDB || window.shimIndexedDB;

    // This code will use the native IndexedDB, if it exists, or the shim otherwise
    indexedDB.open('MyDatabase', 1);
}());

Windows Phone

This information might be outdated. Reports on current support or fixes welcome.

IndexedDBShim works on Windows Phone via a Cordova/PhoneGap plug-in. There are two plugins available: cordova-plugin-indexedDB and cordova-plugin-indexeddb-async. Both plug-ins rely on a WebSQL-to-SQLite adapter, but there are differences in their implementations. Try them both and see which one works best for your app.

Building

To build the project locally on your computer:

  1. Clone this repo If you clone the repository to work against an unstable version, you only need to clone the repository recursively (via git clone https://github.com/indexeddbshim/indexeddbshim.git --recursive) if you wish to have the W3C tests available for testing (which unfortunately loads all W3C tests into the "web-platform-tests" subdirectory rather than just the IndexedDB ones). Otherwise, just use git clone https://github.com/indexeddbshim/indexeddbshim.git

  2. Install dev dependencies (and websql for Node) yarn install

  3. Run the build script npm start

  4. Done

The output files will be generated in the dist directory

Upgrading from previous versions

See Versions for migration information.

Testing

See TESTING.

Resources for IndexedDB

Contributing

Pull requests or Bug reports welcome! See CONTRIBUTING