Home

Awesome

Shared Storage API Explainer

Authors: Alex Turner, Camillia Smith Barnes, Josh Karlin, Yao Xiao

Introduction

In order to prevent cross-site user tracking, browsers are partitioning all forms of storage (cookies, localStorage, caches, etc) by top-frame site. But, there are many legitimate use cases currently relying on unpartitioned storage that will vanish without the help of new web APIs. We’ve seen a number of APIs proposed to fill in these gaps (e.g., Conversion Measurement API, Private Click Measurement, Storage Access, Private State Tokens, TURTLEDOVE, FLoC) and some remain (including cross-origin A/B experiments and user measurement). We propose a general-purpose storage API that can help to serve as common infrastructure for privacy preserving cross-site use cases.

Shared Storage is a key/value store that is partitioned by calling origin (but not top-frame site). The keys and values are strings. While it's possible to write to Shared Storage from nearly anywhere (including response headers!), it is only possible to read from Shared Storage in tightly controlled environments, such as a JavaScript worklet environment which is provided by Shared Storage. These worklets have no capability of communicating with the outside world. They have no network communication and no postMessage. The only way data can leave these worklets, is via privacy-preserving APIs.

Specification

See the draft specification.

APIs built on top of Shared Storage

This document only describes the core shared storage framework and infrastructure to store cross-site data privately and to read that data from within a secure worklet environment. APIs that use shared storage's data to produce some output are linked to below.

Private Aggregation

The Private Aggregation API allows for aggregated histograms to be sent based on data read from shared storage. The histograms are differentially private.

Select URL

The selectURL API allows for content selection based on cross-site data. It takes 8 possible URLs as input and sends them to a worklet which selects from a small list of URLs. The chosen URL is stored in a fenced frame config as an opaque form that can only be read by a fenced frame; the embedder does not learn this information.

Demonstration

You can try out Shared Storage along with some APIs built for it using Chrome 104+.

Example 1: Writing an experiment id to Shared Storage from a document

Since Shared Storage is meant for writing from anywhere, but reading is tightly constrained, it's not actually possible to know what you might have written to your storage from other sites. Is this the first time you've seen this user? Who knows! As such, Shared Storage provides some useful functions beyond just set to write to keys only if they're not already present, and to append to a value rather than overwrite it.

For example, let's say that you wanted to add a user to an experiment group, with a random assignment. But you want that group assignment to be sticky for the user across all of the sites that they visit that your third-party script is on. You may not know if you've ever written this key from this site before, but you certainly don't know if you've set it from another site. To solve this issue, utilize the ignoreIfPresent option.

try {
    sharedStorage.set('group', Math.floor(Math.random() * 1000), { ignoreIfPresent: true });
} catch (error) {
    // Error handling
}

And Shared Storage will only write the value if the key is not already present.

Example 2: Writing to Shared Storage via a worklet

In the event that ignoreIfPresent is not sufficient, and you need to read your existing Shared Storage data before adding new data, consider passing the information that you want to record to a worklet, and letting the worklet read the existing data and perform the write. Like so:

try {
    const worklet = sharedStorage.createWorklet('https://site.example/writerWorklet.js');
    worklet.run('write', {data: {group: Math.floor(Math.random() * 1000)}});
} catch (error) {
    // Error handling
}

And your writerWorklet.js script would look like this: writerWorklet.js

class Writer {
  async run(data) {
    const existingGroup = sharedStorage.get('group');
    if (!existingGroup) {
        cibst newGroup = data['group'];
        sharedStorage.set('group', newGroup);
    }
  }
}
register('write', Writer);

Example 3: Writing to Shared Storage with response headers

It may be faster and more convenient to write to Shared Storage directly from response headers than from JavaScript. This is encouraged in cases where data is coming from a server anyway as it's faster and less intensive than JavaScript methods if you're writing to an origin other than the current document's origin.

Response headers can be used on document, image, and fetch requests.

e.g.,:

<iframe src="https://site.example/iframe" sharedstoragewritable></iframe>

The document request for "https://site.example/iframe" will include a Sec-Shared-Storage-Writable: ?1 request header. Any request with this header can have a corresponding Shared-Storage-Write response header that can write, like so:

Shared-Storage-Write: set;key="group";value="4";ignore_if_present

Example 4: Counting the number of views your content has received across sites

To count the number of times the user has viewed your third-party content, consider using the append option. Like so:

e.g.,:

try {
    window.sharedStorage.append('count', '1');
} catch (error) {
    // Error handling
}

Then, sometime later in your worklet, you can get the total count:

class Counter {
  async run(data) {
    const countLog = data['count']; // e.g.,: '111111111'
    const count = countLog.length;
    // do something useful with this data (such as recording an aggregate histogram) here...
  }
}
register('count', Counter);

Goals

This API intends to support the storage and access needs for a wide array of cross-site data use cases. This prevents each API from having to create its own bespoke storage APIs.

Related work

There have been multiple privacy proposals (SPURFOWL, SWAN, Aggregated Reporting) that have a notion of write-only storage with limited output. Shared Storage allows for each of those use cases, with only one storage API which is easier for developers to learn and requires less browser code. We’d also like to acknowledge the KV Storage explainer, to which we turned for API-shape inspiration.

Proposed API surface

Outside of worklets (e.g., places where writing can happen)

The modifier methods (set, append, delete, clear, and batchUpdate) should be made generally available across most any context. That includes top-level documents, iframes, shared storage worklets, Protected Audience worklets, service workers, dedicated workers, etc.

The shared storage worklet invocation methods (addModule, createWorklet, and run) are available within document contexts.

In the worklet, during sharedStorage.worklet.addModule(url, options) or sharedStorage.createWorklet(url, options)

In the worklet, during an operation

From response headers

Locking for Modifier Methods

All modifier methods (set, append, delete, clear, batchUpdate), whether invoked from JavaScript or from response headers, accept a withLock: <resource> option. This option instructs the method to acquire a lock on the designated resource before executing.

The locks requested this way are partitioned by the shared storage data origin, and are independent of any locks obtained via navigator.locks.request in a Window or Worker context. Note that they share the same scope with the locks obtained via navigator.locks.request in the SharedStorageWorklet context.

Unlike navigator.locks.request, which offers various configuration options, the locks requested this way always use the default settings:

Example: Report on Multiple Keys

This example uses a lock to ensure that the read and delete operations inside the worklet runs atomically, preventing interference from the write operations outside the worklet.

Window context:

try {
  sharedStorage.batchUpdate([
    new SharedStorageSetMethod('key0', calculateValueFor('key0')),
    new SharedStorageSetMethod('key1', calculateValueFor('key1'))
  ], { withLock: 'report-lock' });

  await sharedStorage.worklet.addModule('report-on-multiple-keys-script.js');
  await sharedStorage.worklet.run('report-on-multiple-keys');
} catch (error) {
  // Handle error.
}

In the worklet script (report-on-multiple-keys-script.js):

class ReportOnMultipleKeysOperation {
  async run(data) {
    await navigator.locks.request("report-lock", async (lock) => {
      const value1 = await sharedStorage.get('key1');
      const value2 = await sharedStorage.get('key2');

      // Record an aggregate histogram with `value1` and `value2` here...

      await sharedStorage.delete('key1');
      await sharedStorage.delete('key2');
    });
  }
}
register('report-on-multiple-keys', ReportOnMultipleKeysOperation);

Caveat: Unexpected ordering

Modifier methods may block due to the lock, so may not execute in the order they appear in the code.

// Resolve immediately. Internally, this may block to wait for the lock to be granted.
sharedStorage.set('key0', 'value1', { withLock: 'resource0' });

// Resolve immediately. Internally, this will execute immediately.
sharedStorage.set('key0', 'value2');

Developers should be mindful of this potential ordering issue.

Recommendations for lock usage

If only a single key is involved, and the data is accessed at most once within and outside worklet, then the lock is unnecessary. This is because each access is inherently atomic. Example: A/B experiment.

If the worklet performs both read and write on the same key, then the lock is likely necessary. Example: creative selection by frequency.

If the logic involes updating data organized across multiple keys, then the lock is likely necessary. Example: Report on Multiple Keys.

Reporting embedder context

In using the Private Aggregation API to report on advertisements within fenced frames, for instance, we might report on viewability, performance, which parts of the ad the user engaged with, the fact that the ad showed up at all, and so forth. But when reporting on the ad, it might be important to tie it to some contextual information from the embedding publisher page, such as an event-level ID.

In a scenario where the input URLs for the fenced frame must be k-anonymous, e.g. if we create a FencedFrameConfig from running a Protected Audience auction, it would not be a good idea to rely on communicating the event-level ID to the fenced frame by attaching an identifier to any of the input URLs, as this would make it difficult for any input URL(s) with the attached identifier to reach the k-anonymity threshold.

Instead, before navigating the fenced frame to the auction's winning FencedFrameConfig fencedFrameConfig, we could write the event-level ID to fencedFrameConfig using fencedFrameConfig.setSharedStorageContext() as in the example below.

Subsequently, anything we've written to fencedFrameConfig through setSharedStorageContext() prior to the fenced frame's navigation to fencedFrameConfig, can be read via sharedStorage.context from inside a shared storage worklet created by the fenced frame, or created by any of its same-origin children.

In the embedder page:

// See https://github.com/WICG/turtledove/blob/main/FLEDGE.md for how to write an auction config.
const auctionConfig = { ... };

// Run a Protected Audience auction, setting the option to "resolveToConfig" to true.
auctionConfig.resolveToConfig = true;
const fencedFrameConfig = await navigator.runAdAuction(auctionConfig);

// Write to the config any desired embedder contextual information as a string.
fencedFrameConfig.setSharedStorageContext("My Event ID 123");

// Navigate the fenced frame to the config.
document.getElementById('my-fenced-frame').config = fencedFrameConfig;

In the fenced frame (my-fenced-frame):

// Save some information we want to report that's only available inside the fenced frame.
const frameInfo = { ... };

// Send a report using shared storage and private aggregation.
try {
    await window.sharedStorage.worklet.addModule('report.js');
    await window.sharedStorage.run('send-report', {
    data: { info: frameInfo },
    });
} catch (error) {
    // Error handling
}

In the worklet script (report.js):

class ReportingOperation {
  async run(data) {
    // Helper functions that map the embedder context to a predetermined bucket and the
    // frame info to an appropriately-scaled value.
    // See also https://github.com/patcg-individual-drafts/private-aggregation-api#examples
    function convertEmbedderContextToBucketId(context) { ... }
    function convertFrameInfoToValue(info) { ... }

    // The user agent sends the report to the reporting endpoint of the script's
    // origin (that is, the caller of `sharedStorage.run()`) after a delay.
    privateAggregation.contributeToHistogram({
      bucket: convertEmbedderContextToBucketId(sharedStorage.context) ,
      value: convertFrameInfoToValue(data.info)
    });
  }
}
register('send-report', ReportingOperation);

Keeping a worklet alive for multiple operations

Callers may wish to run multiple worklet operations from the same context, e.g. they might select a URL and then send one or more aggregatable reports. To do so, they would need to use the keepAlive: true option when calling each of the worklet operations (except perhaps in the last call, if there was no need to extend the worklet's lifetime beyond that call).

Writing to Shared Storage via response headers

For an origin making changes to their Shared Storage data at a point when they do not need to read the data, an alternative to using the Shared Storage JavaScript API is to trigger setter and/or deleter operations via the HTTP response header Shared-Storage-Write as in the examples below.

In order to perform operations via response header, the origin must first opt-in via one of the methods below, causing the HTTP request header Sec-Shared-Storage-Writable: ?1 to be added by the user agent if the request is eligible based on permissions checks.

An origin a.example could initiate such a request in multiple ways.

On the client side, to initiate the request:

  1. fetch() option: 
    fetch("https://a.example/path/for/updates", {sharedStorageWritable: true});
  2. Content attribute option with an iframe (also possible with an img): 
     <iframe src="https://a.example/path/for/updates" sharedstoragewritable></iframe>
  3. IDL attribute option with an iframe (also possible with an img): 
    let iframe = document.getElementById("my-iframe");
iframe.sharedStorageWritable = true;
iframe.src = "https://a.example/path/for/updates";

On the server side, here is an example response header:

Shared-Storage-Write: clear, set;key="hello";value="world";ignore_if_present;with_lock="lock2", append;key="good";value="bye", delete;key="hello", set;key="all";value="done", options;with_lock="lock1"

Sending the above response header would be equivalent to making the following call on the client side, from either the document or a worklet:


sharedStorage.batchUpdate([
  new SharedStorageClearMethod(),
  new SharedStorageSetMethod("hello", "world", {ignoreIfPresent: true, withLock: "lock2"}),
  new SharedStorageAppendMethod("good", "bye"),
  new SharedStorageDeleteMethod("hello"),
  new SharedStorageSetMethod("all", "done")
], { withLock: "lock1" })

Loading cross-origin worklet scripts

There are currently six (6) approaches to creating a worklet that loads cross-origin script. The partition origin for the worklet's shared storage data access depends on the approach.

Using the context origin as data partition origin

The first three (3) approaches use the invoking context's origin as the partition origin for shared storage data access and the invoking context's site for shared storage budget withdrawals.

  1. Call addModule() with a cross-origin script.

    In an "https://a.example" context in the embedder page:

    await sharedStorage.worklet.addModule("https://b.example/worklet.js");

    For any subsequent run() or selectURL() operation invoked on this worklet, the shared storage data for "https://a.example" (i.e. the context origin) will be used.

  2. Call createWorklet() with a cross-origin script.

    In an "https://a.example" context in the embedder page:

    const worklet = await sharedStorage.createWorklet("https://b.example/worklet.js");

    For any subsequent run() or selectURL() operation invoked on this worklet, the shared storage data for "https://a.example" (i.e. the context origin) will be used.

  3. Call createWorklet() with a cross-origin script, setting its dataOption to the invoking context's origin.

    In an "https://a.example" context in the embedder page:

    const worklet = await sharedStorage.createWorklet("https://b.example/worklet.js", {dataOrigin: "context-origin"});

    For any subsequent run() or selectURL() operation invoked on this worklet, the shared storage data for "https://a.example" (i.e. the context origin) will be used.

Using the worklet script origin as data partition origin

The fourth approach uses the worklet script's origin as the partition origin for shared storage data access and the worklet script's site for shared storage budget withdrawals.

  1. Call createWorklet() with a cross-origin script, setting its dataOption to the worklet script's origin.

    In an "https://a.example" context in the embedder page:

    const worklet = await sharedStorage.createWorklet("https://b.example/worklet.js", {dataOrigin: "script-origin"});

    For any subsequent run() or selectURL() operation invoked on this worklet, the shared storage data for "https://b.example" (i.e. the worklet script origin) will be used, assuming that the worklet script's server confirmed opt-in with the required "Shared-Storage-Cross-Origin-Worklet-Allowed: ?1" response header.

Using a custom origin as data partition origin

The fifth through eighth approaches use a custom origin as the partition origin for shared storage data access and the custom origin's site for shared storage budget withdrawals.

  1. Call createWorklet(), setting its dataOption to a string whose value is the serialization of the custom origin.

    In an "https://a.example" context in the embedder page:

    const worklet = await sharedStorage.createWorklet("https://a.example/worklet.js", {dataOrigin: "https://custom.example"});

    For any subsequent run() or selectURL() operation invoked on this worklet, the shared storage data for "https://custom.example" will be used, assuming that the /.well-known/ JSON file at "https://custom.example/.well-known/shared-storage/trusted-origins" contains an array of dictionaries, where one of its dictionaries has

  1. Call createWorklet() with a cross-origin script, setting its dataOption to a string whose value is the serialization of the custom origin.

    In an "https://a.example" context in the embedder page:

    const worklet = await sharedStorage.createWorklet("https://b.example/worklet.js", {dataOrigin: "https://custom.example"});

    For any subsequent run() or selectURL() operation invoked on this worklet, the shared storage data for "https://custom.example" will be used, assuming that the /.well-known/ JSON file at "https://custom.example/.well-known/shared-storage/trusted-origins" contains an array of dictionaries, where one of its dictionaries has

Error handling

Note that the shared storage APIs may throw for several possible reasons. The following list of situations is not exhaustive, but, for example, the APIs may throw if the site invoking the API is not enrolled and/or attested, if the user has disabled shared storage in site settings, if the "shared-storage" or "shared-storage-select-url" permissions policy denies access, or if one of its arguments is invalid.

We recommend handling exceptions. This can be done by wrapping async..await calls to shared storage JS methods in try...catch blocks, or by following calls that are not awaited with .catch:

try {
  await window.sharedStorage.worklet.addModule('worklet.js');
} catch (error) {
  // Handle error.
}

window.sharedStorage.worklet.addModule('worklet.js')
  .catch((error) => {
  // Handle error.
});

Worklets can outlive the associated document

After a document dies, the corresponding worklet (if running an operation) will continue to be kept alive for a maximum of two seconds to allow the pending operation(s) to execute. This gives more confidence that any end-of-page operations (e.g. reporting) are able to finish.

Permissions Policy

Shared storage methods can be disallowed by the "shared-storage" policy-controlled feature. Its default allowlist is * (i.e. every origin). APIs built on top of Shared Storage have their own specific permission policies, so it is possible to allow reading and writing of Shared Storage while disabling particular APIs.

Permissions Policy inside the shared storage worklet

The permissions policy inside the shared storage worklet will inherit the permissions policy of the associated document.

Data Retention Policy

Each key is cleared after thirty days of last write (set or append call). If ignoreIfPresent is true, the last write time is updated.

Data Storage Limits

Shared Storage is not subject to the quota manager, as that would leak information across sites. Therefore we limit the per-origin total key and value bytes to 5MB.

Privacy

Shared Storage takes the following protective measures to prevent its stored data from being read by means other than via approved APIs (e.g., via side channels):

Privacy-Preserving APIs

The APIs that can read data from Shared Storage have their own privacy documentation.

Enrollment and Attestation

Use of Shared Storage requires enrollment and attestation via the Privacy Sandbox enrollment attestation model.

For each method in the Shared Storage API surface, a check will be performed to determine whether the calling site is enrolled and attested. In the case where the site is not enrolled and attested, the promise returned by the method is rejected.

Possibilities for extension

Interactions between worklets

Communication between worklets is not possible in the initial design. However, adding support for this would enable multiple origins to flexibly share information without needing a dedicated origin for that sharing. Relatedly, allowing a worklet to create other worklets might be useful.

Registering event handlers

We could support event handlers in future iterations. For example, a handler could run a previously registered operation when a given key is modified (e.g. when an entry is updated via a set or append call):

sharedStorage.addEventListener(
  'key' /* event_type */,
  'operation-to-run' /* operation_name */,
  { key: 'example-key', actions: ['set', 'append'] } /* options */);

Acknowledgements

Many thanks for valuable feedback and advice from:

Victor Costan, Christian Dullweber, Charlie Harrison, Jeff Kaufman, Rowan Merewood, Marijn Kruisselbrink, Nasko Oskov, Evgeny Skvortsov, Michael Tomaine, David Turner, David Van Cleve, Zheng Wei, Mike West.