Awesome

Stellar.FastDB - High Performance Embedded Storage for C# (.NET)

Stellar.FastDB is an exceptionally fast document store for C# with speeds approximately 100 times faster than similar products. Designed for optimal performance and high concurrency, it excels in embedded workflows that demand efficiency.

Key Features

• Serverless, embedded document storage.
• Simple, thread-safe API that supports asynchronous programming with async/await.
• Entirely written in C#, supporting .NET versions 5.0 through 9.0.
• Delivered as a compact, single DLL (60 kb).
• Supports multiple concurrent readers and writers.
• Schema-less NoSQL storage that adapts to changes.
• Optimized storage footprint with configurable storage formats (JSON, MessagePack).
• Advanced LZH compression and AES encryption to ensure data security.
• Parallel processing capabilties for serialization, compression, and encryption.
• Supports relational querying with LINQ.
• Ensures end-to-end type safety for data integrity.
• Accomodates composite keys for complex data structuring.
• Open source and free for both personal and commercial use.
• Install from Nuget. Install-Package Stellar.FastDB.

Benchmarks

A comprehensive list of benchmarks, along with a project for reproduction, is available.

Insert

Delete

Upsert

Bulk Insert

Query

Common Questions

Why was Stellar.FasbDB created?

As a game developer, I needed a high-concurrency storage solution suitable for player-managed game servers. Installing traditional local databases posed too high a demand on players. I found that the available storage solutions were either too slow or struggled with concurrency issues during high volumes of simultaneous reads and writes. Such limitations are impractical for game servers, which must support a large number of players efficiently.

Should I use Stellar.FastDB?

Use Stellar.FastDB if you need:

• Embedded, durable storage suitable for applications like game servers and desktop apps.
• Thread safety with robust support for high concurrency.
• High throughput for both reading and writing data.
• A minimal data storage footprint.

Do not use this database if you need:

• Sharing conections between processes.
• Storing data in a single file.

What additional features are planned for Stellar.FastDB?

• Integrated backup and restore functionality into a single file.
• A memory defragmentation algorithm to further optimize performance.
• An optional structured storage format with defined schema to further improve storage efficiency.
• Support for .NET Standard 2.1 and Unity.

How to use Stellar.FastDB

• Stellar.FastDB's APIs are designed to be inuitive and closely resemble .NET collections. If you are familar with using a Dictionary, you'l find the transition to FastDB seamless! By default, all writes to FastDB are immediate consistency, ensuring data integrity.
• To get started with Stellar.FastDB, install it from NuGet. Search for 'Stellar.FastDB' or use the package manager command 'Install-Package Stellar.FastDB'.

Code Samples

Getting Started

Below is a basic example demonstrating how to interact with Stellar.FastDB. This includes creating a database instance, storing customer data, updating it, retrieving it, and properly closing the database connection:

// create a class
public class Customer
{
    public int Id { get; set; }
    public string Name { get; set; }
    public DateTime DOB { get; set; }
    public string Phone { get; set; }
    public bool IsActive { get; set; }
}

// create database
FastDB fastDB = new FastDB();

// create a collection (key, value)
var customers = fastDB.GetCollection<int, Customer>();

// create your new customer instance
var customer = new Customer
{ 
   Name = "John Wick", 
   Phone = "555-555-5555"
   DOB = new DateTime(2000, 1, 1)
   IsActive = true
};

// add customer
customers.Add(customer.Id, customer);

// update customer
customer.Name = "John Wick's Dog";
customers.Update(customer);

// use LINQ to query documents
var matches = customers.Where(a => a.Name.StartsWith("John") && a.Telephone > 5555555);

// close database
fastDB.Close();

Encryption

FastDBOptions options = new FastDBOptions()
{
   IsEncrypted = true,
   EncryptionPassword = "open-sesame",
};
FastDB fastDB = new FastDB(options);

Compression

FastDBOptions options = new FastDBOptions()
{
   IsCompressed = true,
};
FastDB fastDB = new FastDB(options);

Large Record Parallelism

For operations involving serialization, compression, or encryption of large object graphs, enabling parallel data transformations can significantly enhance throughput, especially with large records. This method effectively leverages multiple processor cores to accelerate write operations.

FastDBOptions options = new FastDBOptions()
{
   BufferMode = BufferModeType.WriteParallelEnabled,
   MaxDegreesOfParallelism = 8,
   IsEncryptionEnabled = true,
   EncryptionPassword = "open-sesame",
   IsCompressed = true,
};
FastDB fastDB = new FastDB(options);

Optional Serialization Contracts

To achieve the smallest possible storage footprint, Stellar.FastDB supports serialization contracts using MessagePack. By adding MessagePack attributes to your data models (as shown in the example below), you can instruct the serializer to package the data more efficiently. Note that this feature is disabled by default and is typically not included in most benchmarks.

// create a class
public class Customer
{
    [Key(0)]
    public int Id { get; set; }
    [Key(1)]
    public string Name { get; set; }
    [Key(2)]
    public DateTime DOB { get; set; }
    [Key(3)]
    public string Phone { get; set; }
    [Key(4)]
    public bool IsActive { get; set; }
}

FastDBOptions options = new FastDBOptions()
{
   Serializer = SerializerType.MessagePack_Contract,
};
FastDB fastDB = new FastDB(options);

// create a collection (key, value)
var customers = fastDB.GetCollection<int, Customer>();

// add customer
customers.Add(customer.Id, customer);
customres.Flush()

// close database
fastDB.Close();

Contact

I may be contacted on the Stellar Conquest Discord (stonstad) and X (stonstad).