Awesome

About The Project

This is a Typescript client for Tile38 that allows for a type-safe interaction with the worlds fastest in-memory geodatabase Tile38.

Features

• fully typed
• lazy client
• optional build-in leader/follower logic
• easy to use and integrate
• no external dependencies other than redis
• build in auto-pagination

Built With

• ioredis

Getting Started

Requirements

"node": ">=20.x"
"ioredis" ">=5.0.0"

Installation

yarn add @iwpnd/tile38-ts

Import

import { Tile38 } from '@iwpnd/tile38-ts';

const tile38 = new Tile38('leader:9851');

Leader / Follower

When it comes to replication, Tile38 follows a leader-follower model. The leader receives all commands that SET data, a follower on the other hand is read-only and can only query data. For more on replication in Tile38 refer to the official documentation.

This client is not meant to setup a replication, because this should happen in your infrastructure. But it helps you to specifically execute commands on leader or follower. This way you can make sure that the leader always has enough resources to execute SETs and fire geofence events on webhooks.

For now we allow for one follower URI to bet set alongside the leader URI.

const tile38 = new Tile38('leader:9851', 'follower:9851');

Once the client is instantiated with a follower, commands can be explicitly send to the follower, but adding .follower() to your command chaining.

await tile38.follower().get('fleet', 'truck1').asObjects();

Options

We expose ioredis RedisOptions.

new Tile38(
    'leader:9851',
    'follower:9851',
    // e.g. to set a retry strategy
    {
        retryStrategy: (times) => {
            return Math.min(times * 50, 2000);
        },
    }
);

Events

We expose ioredis Events.

new Tile38()
    .on('connect', () => console.log('connected'))
    .on('error', console.error);

Pagination

Tile38 has hidden limits set for the amount of objects that can be returned in one request. For most queries/searches this limit is set to 100. This client gives you the option to either paginate the results yourself by add .cursor() and .limit() to your queries, or it abstracts pagination away from the user by adding .all().

Let's say your fleet in Berlin extends 100 vehicles, then

await tile38.within('fleet').get('cities', 'Berlin').asObjects();

will only return 100 vehicle objects. Now you can either get the rest by setting the limit to the amount of vehicles you have in the city and get them all.

await tile38.within('fleet').limit(10000).get('cities', 'Berlin').asObjects();

Or, you can paginate the results in multiple concurrent requests to fit your requirements.

await tile38
    .within('fleet')
    .cursor(0)
    .limit(100)
    .get('cities', 'Berlin')
    .asObjects();
await tile38
    .within('fleet')
    .cursor(100)
    .limit(100)
    .get('cities', 'Berlin')
    .asObjects();
await tile38
    .within('fleet')
    .cursor(200)
    .limit(100)
    .get('cities', 'Berlin')
    .asObjects();

Or, you use .all() to get all objects with a concurrency of 1 and a limit of 100:

await tile38
    .follower()
    .within('fleet')
    .get('cities', 'Berlin')
    .all()
    .asObjects();

Commands

Keys

SET

Set the value of an id. If a value is already associated to that key/id, it'll be overwritten.

await tile38
    .set('fleet', 'truck1')
    .fields({ maxSpeed: 90, milage: 90000 })
    .point(33.5123, -112.2693)
    .exec();

await tile38.set('fleet', 'truck1:driver').string('John Denton').exec();

Options

Input

FSET

Set the value for one or more fields of an object. Fields must be double precision floating points. Returns the integer count of how many fields changed their values.

await tile38.fSet('fleet', 'truck1', { maxSpeed: 90, milage: 90000 });

FEXISTS

Validate if field exists on id.

await tile38
    .set('fleet', 'truck1')
    .fields({ weight: 9001 })
    .point(33.5123, -112.2693)
    .exec();

let response = await tile38.fexists('fleet', 'truck1', 'weight');
console.log(response.exists);
// > true

response = await tile38.fexists('fleet', 'truck1', 'milage');
console.log(response.exists);
// > false

Options

GET

Get the object of an id.

await tile38.get('fleet', 'truck1').asObject();

Get the object as string if set as string

await tile38.set('fleet', 'truck1:driver').string('John').exec();
await tile38.get('fleet', 'truck1').asString();

Options

Output

EXISTS

Validate id exists.

await tile38.set('fleet', 'truck1').point(33.5123, -112.2693).exec();

let response = await tile38.exists('fleet', 'truck1');
console.log(response.exists);
// > true

response = await tile38.exists('fleet', 'truck2');
console.log(response.exists);
// > false

DEL

Remove a specific object by key and id.

await tile38.del('fleet', 'truck1');

PDEL

Remove objects that match a given pattern.

await tile38.pDel('fleet', 'truck*');

DROP

Remove all objects in a given key.

await tile38.drop('fleet');

BOUNDS

Get the minimum bounding rectangle for all objects in a given key

await tile38.bounds('fleet');

EXPIRE

Set an expiration/time to live in seconds of an object.

await tile38.expire('fleet', 'truck1', 10);

TTL

Get the expiration/time to live in seconds of an object.

await tile38.ttl('fleet', 'truck1', 10);

PERSIST

Remove an existing expiration/time to live of an object.

await tile38.persist('fleet', 'truck1');

KEYS

Get all keys matching a glob-style-pattern. Pattern defaults to '*'

await tile38.keys();

STATS

Return stats for one or more keys. The returned stats array contains one or more entries, depending on the number of keys in the request.

await tile38.stats('fleet1', 'fleet2');

Returns

JSET/JSET/JDEL

Set a value in a JSON document. JGET, JSET, and JDEL allow for working with JSON strings

await tile38.jSet('user', 901, 'name', 'Tom');
await tile38.jGet('user', 901);
> {'name': 'Tom'}

await tile38.jSet('user', 901, 'name.first', 'Tom');
await tile38.jSet('user', 901, 'name.first', 'Anderson');
await tile38.jGet('user', 901);
> {'name': { 'first': 'Tom', 'last': 'Anderson' }}

await tile38.jDel('user', 901, 'name.last');
await tile38.jGet('user', 901);
> {'name': { 'first': 'Tom' }}

RENAME

Renames a collection key to newKey.

Options

If the newKey already exists it will be deleted prior to renaming.

await tile38.rename('fleet', 'newFleet', false);

If the newKey already exists it will do nothing.

await tile38.rename('fleet', 'newFleet', true);

Search

Searches are Tile38 bread and butter. They are what make Tile38 a ultra-fast, serious and cheap alternative to PostGIS for a lot of use-cases.

WITHIN

WITHIN searches a key for objects that are fully contained within a given bounding area.

await tile38.within('fleet').bounds(33.462, -112.268,  33.491, -112.245);
>{"ok":true,"objects":[{"id":"1","object":{"type":"Feature","geometry":{"type":"Point","coordinates":[-112.2693, 33.5123]},"properties":{}}}],"count":1,"cursor":1,"elapsed":"72.527µs"}

await tile38.within('fleet').noFields().asCount()
> {"ok":true,"count":205,"cursor":0,"elapsed":"2.078168ms"}

await tile38.within('fleet').get('warehouses', 'Berlin').asCount();
> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

await tile38.within('fleet').buffer(100).get('warehouses', 'Berlin').asCount();
> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

await tile38
    .within('fleet')
    .where('weight', 1000, 1200)
    .get('warehouses', 'Berlin').asCount();
> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

await tile38
    .within('fleet')
    .whereIn('weight', [1000, 1001, 1002])
    .get('warehouses', 'Berlin').asCount();

> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

await tile38
    .within('fleet')
    .whereExpr("weight == 1000")
    .get('warehouses', 'Berlin').asCount();

> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

Options

Outputs

Query

INTERSECTS

Intersects searches a key for objects that are fully contained within a given bounding area, but also returns those that intersect the requested area. When used to search a collection of keys consisting of Point objects (e.g. vehicle movement data) it works like a .within() search as Points cannot intersect. When used to search a collection of keys consisting of Polygon or LineString it also returns objects, that only partly overlap the requested area.

await  tile38.intersects('warehouses').hash('u33d').asObjects();

await tile38.intersects('fleet').get('warehouses', 'Berlin').asIds();
> {"ok":true,"ids":["truck1"],"count":1,"cursor":0,"elapsed":"2.078168ms"}

await tile38
    .intersects('fleet')
    .where('weight', 1000, 1200)
    .get('warehouses', 'Berlin').asCount();
> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

await tile38
    .intersects('fleet')
    .whereIn('weight', [1000, 1001, 1002])
    .get('warehouses', 'Berlin').asCount();

> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

await tile38
    .intersects('fleet')
    .whereExpr("weight == 1000")
    .get('warehouses', 'Berlin').asCount();

> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

Options

Outputs

Query

Nearby

await tile38.set('fleet', 'truck1').point(33.5123, -112.2693).exec();

await  tile38.nearby('fleet').point(33.5124, -112.2694).asCount();
> {"ok":true,"count":1,"cursor":0,"elapsed":"42.8µs"}

await  tile38.nearby('fleet').point(33.5124, -112.2694, 10).asCount();
// because truck1 is further away than 10m
> {"ok":true,"count":0,"cursor":0,"elapsed":"36µs"}

// asIds request
await  tile38.nearby('fleet').point(33.5124, -112.2694).asIds();
> {"ok":true,"ids":['truck1'],"cursor":0,"elapsed":"36µs"}
// asIds with distance
await  tile38.nearby('fleet').distance().point(33.5124, -112.2694).asIds();
> {"ok":true,"ids":[{ "id":"truck1", "distance": 1 }],"cursor":0,"elapsed":"36µs"}

await tile38
    .nearby('fleet')
    .where('weight', 1000, 1200)
    .point(33.5124, -112.2694, 10).asCount();

> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

await tile38
    .nearby('fleet')
    .whereIn('weight', [1000, 1001, 1002])
    .point(33.5124, -112.2694, 10).asCount();
> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

await tile38
    .nearby('fleet')
    .whereExpr("weight == 1000")
    .point(33.5124, -112.2694, 10).asCount();

> {"ok":true,"count":50,"cursor":0,"elapsed":"2.078168ms"}

Options

Outputs

Query

Scan

Incrementally iterate through a given collection key.

await tile38.scan('fleet');

await tile38.scan('fleet').where('foo', 1, 2);
await tile38.scan('fleet').wherein('foo', [1, 2]);
await tile38.scan('fleet').whereExpr('foo == 1');

Options

Outputs

Search

Used to iterate through a keys string values.

await tile38.set('fleet', 'truck1:driver').string('John').exec();

await tile38.search('fleet').match('J*').asStringObjects();
> {"ok":true,"objects":[{"id":"truck1:driver","object":"John"}],"count":1,"cursor":0,"elapsed":"59.9µs"}

Options

Outputs

Server / Connection

CONFIG GET / REWRITE / SET

While .configGet() fetches the requested configuration, .configSet() can be used to change the configuration.

Important, changes made with .set() will only take affect after .configRewrite() is used.

Options

await tile38.configGet('keepalive');
> {"ok":true,"properties":{"keepalive":"300"},"elapsed":"54.6µs"}

await tile38.configSet('keepalive', 400);
> {"ok":true,"elapsed":"36.9µs"}

await tile38.configRewrite();
> {"ok":true,"elapsed":"363µs"}

await tile38.configGet('keepalive');
> {"ok":true,"properties":{"keepalive":"400"},"elapsed":"33.8µs"}

Advanced options Advanced configuration can not be set with commands, but has to be set in a config file in your data directory. Options above, as well as advanced options can be set and are loaded on start-up.

FLUSHDB

Delete all keys and hooks.

await tile38.flushDb();

GC

Instructs the server to perform a garbage collection.

await tile38.gc();

READONLY

Sets Tile38 into read-only mode. Commands such as.set() and .del() will fail.

await tile38.readOnly(true);

SERVER

Get Tile38 statistics. Can optionally be extended with extended=true.

await tile38.server();

// extended
await tile38.server(true);

INFO

Get Tile38 statistics. Similar to SERVER but with different properties.

await tile38.info();

Geofencing

A geofence is a virtual boundary that can detect when an object enters or exits the area. This boundary can be a radius or any search area format, such as a bounding box, GeoJSON object, etc. Tile38 can turn any standard search into a geofence monitor by adding the FENCE keyword to the search.

Geofence events can be:

• inside (object in specified area),
• outside (object outside specified area),
• enter (object enters specified area),
• exit (object exits specified area),
• crosses (object that was not in specified area, has enter/exit it).

Geofence events can be send on upon commands:

• set which sends an event when an object is .set()
• del which sends a last event when the object that resides in the geosearch is deleted via .del()
• dropwhich sends a message when the entire collection is dropped

SETHOOK

Creates a webhook that points to a geosearch (NEARBY/WITHIN/INTERSECTS). Whenever a commands creates/deletes/drops an object that fulfills the geosearch, an event is send to the specified endpoint.

// sends event to endpoint, when object in 'fleet'
// enters the area of a 500m radius around
// latitude 33.5123 and longitude -112.2693
await tile38
    .setHook('warehouse', 'http://10.0.20.78/endpoint')
    .nearby('fleet')
    .point(33.5123, -112.2693, 500)
    .exec();

await tile38.set('fleet', 'bus').point(33.5123001, -112.2693001).exec();
// event =
> {
  "command": "set",
  "group": "5c5203ccf5ec4e4f349fd038",
  "detect": "inside",
  "hook": "warehouse",
  "key": "fleet",
  "time": "2021-03-22T13:06:36.769273-07:00",
  "id": "bus",
  "meta": {},
  "object": { "type": "Point", "coordinates": [-112.2693001, 33.5123001] }
}

Geosearch

Options

Endpoints

SETCHAN / SUBSCRIBE / PSUBSCRIBE

Similar to setHook(), but opens a PUB/SUB channel.

// Start a channel that sends event, when object in 'fleet'
// enters the area of a 500m radius around
// latitude 33.5123 and longitude -112.2693
await tile38
    .setChan('warehouse')
    .nearby('fleet')
    .point(33.5123, -112.2693, 500)
    .exec();

Now add a receiving channel and add an event handler.

const channel = await tile38.channel();
channel.on('message', (message) => console.log(message));

// also as of tile38 v1.33.1 followers can open a channel
const followerChannel = await tile38.follower().channel();
followerChannel.on('message', (message) => console.log(message));

Now that channel can:

// be subscribed to
await channel.subscribe('warehouse');
// or pattern subscribed to
await channel.pSubscribe('ware*');

// also as of tile38 v1.33.1 followers can open a channel
await followerChannel.subscribe('warehouse');
// or pattern subscribed to
await followerChannel.pSubscribe('ware*');

Every set .set() results in:

await tile38.set('fleet', 'bus').point(33.5123001, -112.2693001).exec();
// event =
> {
  "command": "set",
  "group": "5c5203ccf5ec4e4f349fd038",
  "detect": "inside",
  "hook": "warehouse",
  "key": "fleet",
  "time": "2021-03-22T13:06:36.769273-07:00",
  "id": "bus",
  "meta": {},
  "object": { "type": "Point", "coordinates": [-112.2693001, 33.5123001] }
}

// to unsubscribed
await channel.unsubscribe();

// to delete
await tile38.delChan('warehouse');
// or pattern delete
await tile38.pDelChan('ware*');

Geosearch

Options

Addition Information

For more information, please refer to:

• Tile38

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feat/amazing-feature)
3. Commit your Changes (git commit -m 'fix: add some amazing feature')
4. Push to the Branch (git push origin feat/amazing-feature)
5. Open a Pull Request

License

MIT

Maintainer

Vincent Priem - @vpriem
Benjamin Ramser - @iwpnd

Project Link: https://github.com/iwpnd/tile38-ts

Acknowledgements

Josh Baker - maintainer of Tile38