Awesome
<div align="center"> <br/> <br/> <img width="360" src="images/logo.svg" alt="Redis OM" /> <br/> <br/> </div> <p align="center"> <p align="center"> Object mapping, and more, for Redis and .NET </p> </p>Redis OM .NET makes it easy to model Redis data in your .NET Applications.
Redis OM .NET | Redis OM Node.js | Redis OM Spring | Redis OM Python
<details> <summary><strong>Table of contents</strong></summary> <!-- START doctoc generated TOC please keep comment here to allow auto update --> <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->- 💡 Why Redis OM?
- 💻 Installation
- 🏁 Getting started
- 📚 Documentation
- ⛏️ Troubleshooting
- ✨ Redis Stack
- ❤️ Contributing
- ❤️ Our Contributors
💡 Why Redis OM?
Redis OM provides high-level abstractions for using Redis in .NET, making it easy to model and query your Redis domain objects.
This preview release contains the following features:
- Declarative object mapping for Redis objects
- Declarative secondary-index generation
- Fluent APIs for querying Redis
- Fluent APIs for performing Redis aggregations
💻 Installation
Using the dotnet cli, run:
dotnet add package Redis.OM
🏁 Getting started
Starting Redis
Before writing any code you'll need a Redis instance with the appropriate Redis modules! The quickest way to get this is with Docker:
docker run -p 6379:6379 -p 8001:8001 redis/redis-stack
This launches the redis-stack an extension of Redis that adds all manner of modern data structures to Redis. You'll also notice that if you open up http://localhost:8001 you'll have access to the redis-insight GUI, a GUI you can use to visualize and work with your data in Redis.
📇 Modeling your domain (and indexing it!)
With Redis OM, you can model your data and declare indexes with minimal code. For example, here's how we might model a customer object:
[Document(StorageType = StorageType.Json)]
public class Customer
{
[Indexed] public string FirstName { get; set; }
[Indexed] public string LastName { get; set; }
public string Email { get; set; }
[Indexed(Sortable = true)] public int Age { get; set; }
[Indexed] public string[] NickNames {get; set;}
}
Notice that we've applied the Document
attribute to this class. We've also specified that certain fields should be Indexed
.
Now we need to create the Redis index. So we'll connect to Redis and then call CreateIndex
on an IRedisConnection
:
var provider = new RedisConnectionProvider("redis://localhost:6379");
provider.Connection.CreateIndex(typeof(Customer));
Indexing Embedded Documents
There are two methods for indexing embedded documents with Redis.OM, an embedded document is a complex object, e.g. if our Customer
model had an Address
property with the following model:
[Document(IndexName = "address-idx", StorageType = StorageType.Json)]
public partial class Address
{
public string StreetName { get; set; }
public string ZipCode { get; set; }
[Indexed] public string City { get; set; }
[Indexed] public string State { get; set; }
[Indexed(CascadeDepth = 1)] public Address ForwardingAddress { get; set; }
[Indexed] public GeoLoc Location { get; set; }
[Indexed] public int HouseNumber { get; set; }
}
Index By JSON Path
You can index fields by JSON path, in the top level model, in this case Customer
you can decorate the Address
property with an Indexed
and/or Searchable
attribute, specifying the JSON path to the desired field:
[Document(StorageType = StorageType.Json)]
public class Customer
{
[Indexed] public string FirstName { get; set; }
[Indexed] public string LastName { get; set; }
public string Email { get; set; }
[Indexed(Sortable = true)] public int Age { get; set; }
[Indexed] public string[] NickNames {get; set;}
[Indexed(JsonPath = "$.ZipCode")]
[Searchable(JsonPath = "$.StreetAddress")]
public Address Address {get; set;}
}
Indexing Arrays of Objects
This methodology can also be used for indexing string and string-like value-types within objects within Arrays and Lists, so for example if we had an array of Addresses, and we wanted to index the cities within those addresses we could do so with the following
[Indexed(JsonPath = "$.City")]
public Address[] Addresses { get; set; }
Those Cities can then be queried with an Any
predicate within the main Where
clause.
collection.Where(c=>c.Addresses.Any(a=>a.City == "Satellite Beach"))
Limitations
The way Redis indexes fields within a collection of embedded objects does not allow multiple predictates to be specified to a given document e.g.
collection.Where(c=>c.Addresses.Any(a=>a.City == "Satellite Beach" && a.ZipCode == "32937))
In the above case the query can only check if the Addresses collection contains an entry that is Satellite Beach
, and Contains an entry that has a zip code of 32937
, rather than an entry that has both the city of Satellite Beach
and a zip code of `32937
Cascading Index
Alternatively, you can also embedded models by cascading indexes. In this instance you'd simply decorate the property with Indexed
and set the CascadeDepth
to whatever to however may levels you want the model to cascade for. The default is 0, so if CascadeDepth
is not set, indexing an object will be a no-op:
[Document(StorageType = StorageType.Json)]
public class Customer
{
[Indexed] public string FirstName { get; set; }
[Indexed] public string LastName { get; set; }
public string Email { get; set; }
[Indexed(Sortable = true)] public int Age { get; set; }
[Indexed] public string[] NickNames {get; set;}
[Indexed(CascadeDepth = 2)]
public Address Address {get; set;}
}
In the above case, all indexed/searchable fields in Address will be indexed down 2 levels, so the ForwardingAddress
field in Address
will also be indexed.
Once the index is created, we can:
- Insert Customer objects into Redis
- Get a Customer object by ID from Redis
- Query Customers from Redis
- Run aggregations on Customers in Redis
Let's see how!
Indexing DateTimes
As of version 0.4.0, all DateTime objects are indexed as numerics, and they are inserted as numerics into JSON documents. Because of this, you can query them as if they were numerics!
🔑 Keys and Ids
ULIDs and strings
Ids are unique per object, and are used as part of key generation for the primary index in Redis. The natively supported Id type in Redis OM is the ULID. You can bind ids to your model, by explicitly decorating your Id field with the RedisIdField
attribute:
[Document(StorageType = StorageType.Json)]
public class Customer
{
[RedisIdField] public Ulid Id { get; set; }
[Indexed] public string FirstName { get; set; }
[Indexed] public string LastName { get; set; }
public string Email { get; set; }
[Indexed(Sortable = true)] public int Age { get; set; }
[Indexed] public string[] NickNames { get; set; }
}
When you call Set
on the RedisConnection
or call Insert
in the RedisCollection
, to insert your object into Redis, Redis OM will automatically set the id for you and you will be able to access it in the object. If the Id
type is a string, and there is no explicitly overriding IdGenerationStrategy on the object, the ULID for the object will bind to the string.
Other types of ids
Redis OM also supports other types of ids, ids must either be strings or value types (e.g. ints, longs, GUIDs etc. . .), if you want a non-ULID id type, you must either set the id on each object prior to insertion, or you must register an IIdGenerationStrategy
with the DocumentAttribute
class.
Register IIdGenerationStrategy
To Register an IIdGenerationStrategy
with the DocumentAttribute
class, simply call DocumentAttribute.RegisterIdGenerationStrategy
passing in the strategy name, and the implementation of IIdGenerationStrategy
you want to use. Let's say for example you had the StaticIncrementStrategy
, which maintains a static counter in memory, and increments ids based off that counter:
public class StaticIncrementStrategy : IIdGenerationStrategy
{
public static int Current = 0;
public string GenerateId()
{
return (Current++).ToString();
}
}
You would then register that strategy with Redis.OM like so:
DocumentAttribute.RegisterIdGenerationStrategy(nameof(StaticIncrementStrategy), new StaticIncrementStrategy());
Then, when you want to use that strategy for generating the Ids of a document, you can simply set the IdGenerationStrategy of your document attribute to the name of the strategy.
[Document(IdGenerationStrategyName = nameof(StaticIncrementStrategy))]
public class ObjectWithCustomIdGenerationStrategy
{
[RedisIdField] public string Id { get; set; }
}
Key Names
The key names are, by default, the fully qualified class name of the object, followed by a colon, followed by the Id
. For example, there is a Person class in the Unit Test project, an example id of that person class would be Redis.OM.Unit.Tests.RediSearchTests.Person:01FTHAF0D1EKSN0XG67HYG36GZ
, because Redis.OM.Unit.Tests.RediSearchTests.Person
is the fully qualified class name, and 01FTHAF0D1EKSN0XG67HYG36GZ
is the ULID (the default id type). If you want to change the prefix (the fully qualified class name), you can change that in the DocumentAttribute
by setting the Prefixes
property, which is an array of strings e.g.
[Document(Prefixes = new []{"Person"})]
public class Person
Note: At this time, Redis.OM will only use the first prefix in the prefix list as the prefix when creating a key name. However, when an index is created, it will be created on all prefixes enumerated in the Prefixes property
🔎 Querying
We can query our domain using expressions in LINQ, like so:
var customers = provider.RedisCollection<Customer>();
// Insert customer
customers.Insert(new Customer()
{
FirstName = "James",
LastName = "Bond",
Age = 68,
Email = "bondjamesbond@email.com"
});
// Find all customers whose last name is "Bond"
customers.Where(x => x.LastName == "Bond");
// Find all customers whose last name is Bond OR whose age is greater than 65
customers.Where(x => x.LastName == "Bond" || x.Age > 65);
// Find all customers whose last name is Bond AND whose first name is James
customers.Where(x => x.LastName == "Bond" && x.FirstName == "James");
// Find all customers with the nickname of Jim
customers.Where(x=>x.NickNames.Contains("Jim"));
Vectors
Redis OM .NET also supports storing and querying Vectors stored in Redis.
A Vector<T>
is a representation of an object that can be transformed into a vector by a Vectorizer.
A VectorizerAttribute
is the abstract class you use to decorate your Vector fields, it is responsible for defining the logic to convert the object's that Vector<T>
is a container for into actual vector embeddings needed. In the package Redis.OM.Vectorizers
we provide vectorizers for HuggingFace, OpenAI, and AzureOpenAI to allow you to easily integrate them into your workflows.
Define a Vector in your Model.
To define a vector in your model, simply decorate a Vector<T>
field with an Indexed
attribute which defines the algorithm and algorithmic parameters and a Vectorizer
attribute which defines the shape of the vectors, (in this case we'll use OpenAI):
[Document(StorageType = StorageType.Json)]
public class OpenAICompletionResponse
{
[RedisIdField]
public string Id { get; set; }
[Indexed(DistanceMetric = DistanceMetric.COSINE, Algorithm = VectorAlgorithm.HNSW, M = 16)]
[OpenAIVectorizer]
public Vector<string> Prompt { get; set; }
public string Response { get; set; }
[Indexed]
public string Language { get; set; }
[Indexed]
public DateTime TimeStamp { get; set; }
}
Insert Vectors into Redis
With the vector defined in our model, all we need to do is create Vectors of the generic type, and insert them with our model. Using our RedisCollection
, you can do this by simply using Insert
:
var collection = _provider.RedisCollection<OpenAICompletionResponse>();
var completionResult = new OpenAICompletionResponse
{
Language = "en_us",
Prompt = Vector.Of("What is the Capital of France?"),
Response = "Paris",
TimeStamp = DateTime.Now - TimeSpan.FromHours(3)
};
collection.Insert(completionResult);
The Vectorizer will manage the embedding generation for you without you having to intervene.
Query Vectors in Redis
To query vector fields in Redis, all you need to do is use the VectorRange
method on a vector within our normal LINQ queries, and/or use the NearestNeighbors
with whatever other filters you want to use, here's some examples:
var prompt = "What really is the Capital of France?";
// simple vector range, find first within .15
var result = collection.First(x => x.Prompt.VectorRange(prompt, .15));
// simple nearest neighbors query, finds first nearest neighbor
result = collection.NearestNeighbors(x => x.Prompt, 1, prompt).First();
// hybrid query, pre-filters result set for english responses, then runs a nearest neighbors search.
result = collection.Where(x=>x.Language == "en_us").NearestNeighbors(x => x.Prompt, 1, prompt).First();
// hybrid query, pre-filters responses newer than 4 hours, and finds first result within .15
var ts = DateTimeOffset.Now - TimeSpan.FromHours(4);
result = collection.First(x=>x.TimeStamp > ts && x.Prompt.VectorRange(prompt, .15));
What Happens to the Embeddings?
With Redis OM, the embeddings can be completely transparent to you, they are generated and bound to the Vector<T>
when you query/insert your vectors. If however you needed your embedding after the insertion/Query, they are available at Vector<T>.Embedding
, and be queried either as the raw bytes, as an array of doubles or as an array of floats (depending on your vectorizer).
Configuration
The Vectorizers provided by the Redis.OM.Vectorizers
package have some configuration parameters that it will pull in either from your appsettings.json
file, or your environment variables (with your appsettings taking precedence).
Configuration Parameter | Description |
---|---|
REDIS_OM_HF_TOKEN | HuggingFace Authorization token. |
REDIS_OM_OAI_TOKEN | OpenAI Authorization token |
REDIS_OM_OAI_API_URL | OpenAI URL |
REDIS_OM_AZURE_OAI_TOKEN | Azure OpenAI api key |
REDIS_OM_AZURE_OAI_RESOURCE_NAME | Azure resource name |
REDIS_OM_AZURE_OAI_DEPLOYMENT_NAME | Azure deployment |
Semantic Caching
Redis OM also provides the ability to use Semantic Caching, as well as providers for OpenAI, HuggingFace, and Azure OpenAI to perform semantic caching. To use a Semantic Cache, simply pull one out of the RedisConnectionProvider and use Store
to insert items, and GetSimilar
to retrieve items. For example:
var cache = _provider.OpenAISemanticCache(token, threshold: .15);
cache.Store("What is the capital of France?", "Paris");
var res = cache.GetSimilar("What really is the capital of France?").First();
ML.NET Based Vectorizers
We also provide the packages Redis.OM.Vectorizers.ResNet18
and Redis.OM.Vectorizers.AllMiniLML6V2
which have embedded models / ML Pipelines in them to
allow you to easily Vectorize Images and Sentences respectively without the need to depend on an external API.
🖩 Aggregations
We can also run aggregations on the customer object, again using expressions in LINQ:
// Get our average customer age
customerAggregations.Average(x => x.RecordShell.Age);
// Format customer full names
customerAggregations.Apply(x => string.Format("{0} {1}", x.RecordShell.FirstName, x.RecordShell.LastName),
"FullName");
// Get each customer's distance from the Mall of America
customerAggregations.Apply(x => ApplyFunctions.GeoDistance(x.RecordShell.Home, -93.241786, 44.853816),
"DistanceToMall");
📚 Documentation
This README just scratches the surface. You can find a full tutorial on the redis.io. All the summary docs for this library can be found on the repo's github page.
⛏️ Troubleshooting
If you run into trouble or have any questions, we're here to help!
First, check the FAQ. If you don't find the answer there, hit us up on the Redis Discord Server.
✨ Redis Stack
Redis OM can be used with regular Redis for Object mapping and getting objects by their IDs. For more advanced features like indexing, querying, and aggregation, Redis OM is dependent on the Redis Stack platform, a collection of modules that extend Redis.
Why this is important
Without Redis Stack, you can still use Redis OM to create declarative models backed by Redis.
We'll store your model data in Redis as Hashes, and you can retrieve models using their primary keys.
So, what won't work without Redis Stack?
- You won't be able to nest models inside each other.
- You won't be able to use our expressive queries to find object -- you'll only be able to query by primary key.
So how do you get Redis Stack?
You can use Redis Stack with your self-hosted Redis deployment. Just follow the instructions for Installing Redis Stack.
Don't want to run Redis yourself? Redis Stack is also available on Redis Cloud. Get started here.
❤️ Contributing
We'd love your contributions! If you want to contribute please read our Contributing document.
❤️ Our Contributors
- @slorello89
- @banker
- @simonprickett
- @BenShapira
- @satish860
- @dracco1993
- @ecortese
- @DanJRWalsh
- @baldutech
- @shacharPash
- @frostshoxx
- @berviantoleo
- @AmirEsdeki
- @Zulander1
- @Jeevananthan
- @mariusmuntean
- @jcreus1
- @JuliusMikkela
- @imansafari1991
- @AndersenGans
- @mdrakib
- @jrpavoncello
- @axnetg
- @abbottdev
- @PrudiusVladislav
- @CormacLennon
- @ahmedisam99