Awesome
Interprocess
Cloudtoid Interprocess is a cross-platform shared memory queue for fast communication between processes (Interprocess Communication or IPC). It uses a shared memory-mapped file for extremely fast and efficient communication between processes and it is used internally by Microsoft.
- Fast: It is extremely fast.
- Cross-platform: It supports Windows, and Unix-based operating systems such as Linux, MacOS, and FreeBSD.
- API: Provides a simple and intuitive API to enqueue/send and dequeue/receive messages.
- Multiple publishers and subscribers: It supports multiple publishers and subscribers to a shared queue.
- Efficient: Sending and receiving messages is almost heap memory allocation free reducing garbage collections.
- Developer: Developed by a guy at Microsoft.
NuGet Package
The NuGet package for this library is published here.
Note: To improve performance, this library only supports 64-bit CLR with 64-bit processor architectures. Attempting to use this library on 32-bit processors, 32-bit operating systems, or on WOW64 may throw a
NotSupportedException
.
Usage
This library supports .NET 8.0+. It is optimized for .NET dependency injection but can also be used without DI.
Usage without DI
Creating a message queue factory:
var factory = new QueueFactory();
Creating a message queue publisher:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var publisher = factory.CreatePublisher(options);
publisher.TryEnqueue(message);
Creating a message queue subscriber:
options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var subscriber = factory.CreateSubscriber(options);
subscriber.TryDequeue(messageBuffer, cancellationToken, out var message);
Usage with DI
Adding the queue factory to the DI container:
services
.AddInterprocessQueue() // adding the queue related components
.AddLogging(); // optionally, we can enable logging
Creating a message queue publisher using an instance of IQueueFactory
retrieved from the DI container:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var publisher = factory.CreatePublisher(options);
publisher.TryEnqueue(message);
Creating a message queue subscriber using an instance of IQueueFactory
retrieved from the DI container:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var subscriber = factory.CreateSubscriber(options);
subscriber.TryDequeue(messageBuffer, cancellationToken, out var message);
Sample
To see a sample implementation of a publisher and a subscriber process, try out the following two projects. You can run them side by side and see them in action:
Please note that you can start multiple publishers and subscribers sending and receiving messages to and from the same message queue.
Performance
A lot has gone into optimizing the implementation of this library. For instance, it is mostly heap-memory allocation free, reducing the need for garbage collection induced pauses.
Summary: A full enqueue followed by a dequeue takes ~250 ns
on Linux, ~650 ns
on MacOS, and ~300 ns
on Windows.
Details: To benchmark the performance and memory usage, we use BenchmarkDotNet and perform the following runs:
Method | Description |
---|---|
Message enqueue | Benchmarks the performance of enqueuing a message. |
Message enqueue and dequeue | Benchmarks the performance of sending a message to a client and receiving that message. It is inclusive of the duration to enqueue and dequeue a message. |
Message enqueue and dequeue - no message buffer | Benchmarks the performance of sending a message to a client and receiving that message. It is inclusive of the duration to enqueue and dequeue a message and memory allocation for the received message. |
You can replicate the results by running the following command:
dotnet run Interprocess.Benchmark.csproj -c Release
You can also be explicit about the .NET SDK and Runtime(s) versions:
dotnet run Interprocess.Benchmark.csproj -c Release -f net7.0 --runtimes net7.0 net6.0 netcoreapp3.1
On Windows
Host:
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
Intel Core i9-10900X CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK=6.0.201
[Host] : .NET 6.0.3 (6.0.322.12309), X64 RyuJIT
.NET 6.0 : .NET 6.0.3 (6.0.322.12309), X64 RyuJIT
Results:
Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
---|---|---|---|---|
Message enqueue | 192.7 | 3.61 | 3.21 | - |
Message enqueue and dequeue | 305.6 | 5.96 | 6.62 | - |
Message enqueue and dequeue - no message buffer | 311.5 | 5.90 | 9.85 | 32 B |
On MacOS
Host:
BenchmarkDotNet=v0.13.1, OS=macOS Big Sur 11.6 (20G165) [Darwin 20.6.0]
Intel Core i5-8279U CPU 2.40GHz (Coffee Lake), 1 CPU, 8 logical and 4 physical cores
.NET SDK=5.0.401
[Host] : .NET 5.0.10 (5.0.1021.41214), X64 RyuJIT
.NET 5.0 : .NET 5.0.10 (5.0.1021.41214), X64 RyuJIT
Results:
Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
---|---|---|---|---|
Message enqueue | 487.50 | 4.75 | 3.96 | - |
Message enqueue and dequeue | 666.10 | 10.91 | 10.20 | - |
Message enqueue and dequeue - no message buffer | 689.33 | 13.38 | 15.41 | 32 B |
On Ubuntu (through WSL)
Host:
BenchmarkDotNet=v0.13.2, OS=ubuntu 20.04
Intel Core i9-10900X CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK=6.0.403
[Host] : .NET 6.0.11 (6.0.1122.52304), X64 RyuJIT AVX2
.NET 6.0 : .NET 6.0.11 (6.0.1122.52304), X64 RyuJIT AVX2
Results:
Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
---|---|---|---|---|
Message enqueue | 5.3 | - | - | - |
Message enqueue and dequeue | 169.9 | 3.08 | 4.01 | - |
Message enqueue and dequeue - no message buffer | 179.4 | 1.91 | 1.60 | 32 B |
Implementation Notes
This library relies on Named Semaphores To signal the existence of a new message to all message subscribers and to do it across process boundaries. Named semaphores are synchronization constructs accessible across processes.
.NET Core 3.1 and .NET 6/7 do not support named semaphores on Unix-based OSs (Linux, macOS, etc.). Instead we are using P/Invoke and relying on operating system's POSIX semaphore implementation. (Linux and MacOS implementations).
This implementation will be replaced with System.Threading.Semaphore
once .NET adds support for named semaphores on all platforms.
How to Contribute
- Create a branch from
main
. - Ensure that all tests pass on Windows, Linux, and MacOS.
- Keep the code coverage number above 80% by adding new tests or modifying the existing tests.
- Send a pull request.
Author
Pedram Rezaei is a software architect at Microsoft with years of experience building highly scalable and reliable cloud-native applications for Microsoft.
What is next
Here are a couple of items that we are working on.
- Create a marketing/documentation website
- Once .NET supports named semaphores on Linux, then start using them.