Home

Awesome

Ballerina Confluent Avro Serialization/Deserialization Connector

Build Trivy GraalVM Check GitHub Last Commit GitHub Issues License

Avro Serializer/Deserializer for Confluent Schema Registry is an Avro serializer/deserializer designed to work with the Confluent Schema Registry. It is designed to not include the message schema in the message payload but instead includes the schema ID.

The Ballerina Avro Serializer/Deserializer for Confluent Schema Registry connector integrates with the Confluent Schema Registry for Avro serialization and deserialization.

Quickstart

To use the Confluent schema registry connector in your Ballerina project, modify the .bal file as follows.

Step 1: Import the module

Import the ballerinax/confluent.cavroserdes module into your Ballerina project.

import ballerinax/confluent.cavroserdes;

Step 2: Invoke the connector operation

You can now utilize the operations available within the connector. To instantiate a cregistry:Client instance refer to the guidelines here.

public function main() returns error? {
   cregistry:Client registry = ; // instantiate a schema registry client

   string schema = string `
      {
         "type": "int",
         "name" : "value", 
         "namespace": "data"
      }`;

   int value = 1;
   byte[] bytes = check cavroserdes:serialize(registry, schema, value, "subject");
   int number = check cavroserdes:deserialize(registry, bytes);
}

Step 3: Run the Ballerina application

Use the following command to compile and run the Ballerina program.

bal run

Examples

The Ballerina Avro Serializer/Deserializer connector for Confluent Schema Registry provides practical examples illustrating usage in various scenarios. Explore these examples.

  1. Kafka Avro producer - This example demonstrates how to publish Avro serialized data to a Kafka topic.

  2. Kafka Avro consumer - This guide demonstrates how to consume data in the correct format according to the Avro schema from a Kafka topic.

Issues and projects

The Issues and Projects tabs are disabled for this repository as this is part of the Ballerina library. To report bugs, request new features, start new discussions, view project boards, etc., visit the Ballerina library parent repository.

This repository only contains the source code for the package.

Building from the source

Prerequisites

  1. Download and install Java SE Development Kit (JDK) version 17. You can download it from either of the following sources:

    Note: After installation, remember to set the JAVA_HOME environment variable to the directory where JDK was installed.

  2. Download and install Ballerina Swan Lake.

  3. Download and install Docker.

    Note: Ensure that the Docker daemon is running before executing any tests.

  4. Generate a Github access token with read package permissions, then set the following env variables:

    export packageUser=<Your GitHub Username>
export packagePAT=<GitHub Personal Access Token>

Build options

Execute the commands below to build from the source.

  1. To build the package:

    ./gradlew clean build

  2. To run the tests:

    ./gradlew clean test

  3. To build the without the tests:

    ./gradlew clean build -x test

  4. To debug package with a remote debugger:

    ./gradlew clean build -Pdebug=<port>

  5. To debug with Ballerina language:

    ./gradlew clean build -PbalJavaDebug=<port>

  6. Publish the generated artifacts to the local Ballerina central repository:

    ./gradlew clean build -PpublishToLocalCentral=true

  7. Publish the generated artifacts to the Ballerina central repository:

    ./gradlew clean build -PpublishToCentral=true

Contributing to Ballerina

As an open source project, Ballerina welcomes contributions from the community.

For more information, go to the contribution guidelines.

Code of conduct

All contributors are encouraged to read the Ballerina Code of Conduct.

Useful links