Home

Awesome

.NET Stream Processing Library for Apache Kafka <sup>TM</sup> · GitHub license · Join the chat at https://discord.gg/J7Jtxum build

PackageNuget versionDownloads
Streamiz.Kafka.NetNuget (with prereleases)Nuget
Streamiz.Kafka.Net.SchemaRegistry.SerDesNuget (with prereleases)Nuget
Streamiz.Kafka.Net.SchemaRegistry.SerDes.AvroNuget (with prereleases)Nuget
Streamiz.Kafka.Net.SchemaRegistry.SerDes.ProtobufNuget (with prereleases)Nuget
Streamiz.Kafka.Net.SchemaRegistry.SerDes.JsonNuget (with prereleases)Nuget
Streamiz.Kafka.Net.Metrics.PrometheusNuget (with prereleases)Nuget
Streamiz.Kafka.Net.Metrics.OpenTelemetryNuget (with prereleases)Nuget

Quality Statistics

Sonar Cloud Quality Gate Sonar Cloud Quality Gate Sonar Cloud Reliability Rate Sonar Cloud Security Rate Sonar Cloud Maintainability Rate Sonar Cloud Duplicated Code

Project Statistics

<div> <img alt="GitHub issues" src="https://img.shields.io/github/issues/LGouellec/kafka-streams-dotnet"> <img alt="GitHub pull requests" src="https://img.shields.io/github/issues-pr/LGouellec/kafka-streams-dotnet"> </div> <br/> <img src="./resources/logo-kafka-stream-net.png" width="150">

Streamiz Kafka .NET is .NET stream processing library for Apache Kafka.

KAFKA is a registered trademark of The Apache Software Foundation and
has been licensed for use by Streamiz. Streamiz has no
affiliation with and is not endorsed by The Apache Software Foundation.

Try it with Gitpod

Open in Gitpod

Step 1

Waiting run task is complete. The task is consider complete some seconds after viewing this message "🚀 Enjoy Streamiz the .NET Stream processing library for Apache Kafka (TM)"

Step 2

Switch to producer terminal and send sentences or word. The sample case is "Count the number words" similar to here

Step 3

Switch to consumerterminal and check aggregation result

Documentation

Read the full documentation on https://lgouellec.github.io/kafka-streams-dotnet/

Installation

Nuget packages are listed to nuget.org

Install the last version with :

dotnet add package Streamiz.Kafka.Net

Usage

There, a sample streamiz application :

static async System.Threading.Tasks.Task Main(string[] args)
{ 
    var config = new StreamConfig<StringSerDes, StringSerDes>();
    config.ApplicationId = "test-app";
    config.BootstrapServers = "localhost:9092";
    
    StreamBuilder builder = new StreamBuilder();

    var kstream = builder.Stream<string, string>("stream");
    var ktable = builder.Table("table", InMemory.As<string, string>("table-store"));

    kstream.Join(ktable, (v, v1) => $"{v}-{v1}")
           .To("join-topic");

    Topology t = builder.Build();
    KafkaStream stream = new KafkaStream(t, config);

    Console.CancelKeyPress += (o, e) => {
        stream.Dispose();
    };

    await stream.StartAsync();
}

Compare Kafka Streams vs Streamiz

FeaturesKafka Streams (JAVA) supportedStreamiz supportedComment
Stateless processorsXX
RocksDb storeXX
Standby replicasXNo plan for now
InMemory storeXX
Transformer, Processor APIXX
PunctuateXX
KStream-KStream JoinXX
KTable-KTable JoinXX
KTable-KTable FK JoinXPlan for future
KStream-KTable JoinXX
KStream-GlobalKTable JoinXX
KStream Async Processing (external call inside the topology)XNot supported in Kafka Streams JAVA
Hopping windowXX
Tumbling windowXX
Sliding windowXNo plan for now
Session windowXNo plan for now
CacheXXEA 1.6.0
Suppress(..)XNo plan for now
Interactive QueriesXNo plan for now
State store batch restoringXNo plan for now
Exactly Once (v1 and v2)XXEOS V1 supported, EOS V2 not supported yet

Contributing

Maintainers:

Streamiz Kafka .Net is a community project. We invite your participation through issues and pull requests! You can peruse the contributing guidelines.

When adding or changing a service please add tests and documentations.

Support

You can found support here

Star History

<a href="https://star-history.com/#LGouellec/kafka-streams-dotnet&Date"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=LGouellec/kafka-streams-dotnet&type=Date&theme=dark" /> <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=LGouellec/kafka-streams-dotnet&type=Date" /> <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=LGouellec/kafka-streams-dotnet&type=Date" /> </picture> </a>