eMagiz Kafka Connector

Content Type: Service
Categories: Communication

Overview

This Connector module allows to make a fast and easy connection from your Mendix application to the eMagiz Event Streaming Cluster. All required eMagiz configurations are provisioned by the Connector module. In versions 1.x.x we support Mendix version 8.18.2 and higher. In versions 2.x.x we support Mendix version 9.0.5 and higher.

Documentation

Demo urlhttps://docs.emagiz.com/bin/download/Main/Videos/Microlearning/WebHome/intermediate-event-streaming-connectors-using-kafka-module-mendix.mp4?rev=1.1

Description

This module allows applications to be able to produce and consume data to and from topics that are managed within an external Kafka Cluster, such as the eMagiz Kafka Cluster.

Producing data on a topic means that the external system, in this case, Mendix, writes data to a pre-defined topic where the data is stored temporarily to make sure that the data can be consumed by one or more other systems. Consuming data from a topic means that the external system, in this case, Mendix, reads data from a pre-defined topic where the data is stored temporarily.

Typical usage scenario

By knowing how you can easily set up Mendix to consume and produce data from and to topics you have the option to better transport large volumes of data between several systems that includes a Mendix application.

Features and limitations

  1. Set up a connection to the external Kafka Cluster (i.e the eMagiz Kafka Cluster) from Mendix
  2. Configure a Producer that can write (publish) data to a topic
  3. Configure a Consumer that can read (listen) data from a topic
  4. Support Mendix mult-instance setup to consume data

Dependencies

  • MxModelReflection
  • LibraryLogging
  • Encryption 2.1.0

Installation

Regular import of the package into your project. Within the Kafka Module there is a microflow called OpenAdministration, make sure an admin can reach this microflow as this is the starting point for the rest of the configuration.

Configuration

Please refer to the relevant Microlearning in the eMagiz Academy section:

https://docs.emagiz.com/bin/view/Main/eMagiz%20Academy/Microlearnings/Intermediate%20Level/Event%20Streaming%20Connectors/intermediate-event-streaming-connectors-using-kafka-module-mendix

 

Releases

Version: 2.3.0
Framework Version: 9.0.5
Release Notes: The 2.3.0 version of the eMagiz Kafka connector (EMKC) improves the default client connection settings and better caching of connections. Especially high throughput environments will greatly benefit from these settings. Major changes The default client connection settings and better caching of connections. Specific changes: the "batch size" setting to 200000 and "linger ms" to 300.
Version: 1.3.0
Framework Version: 8.18.2
Release Notes: The 1.3.0 version of the eMagiz Kafka connector (EMKC) improves the default client connection settings and better caching of connections. Especially high throughput environments will greatly benefit from these settings. Major changes The default client connection settings and better caching of connections. Specific changes: the "batch size" setting to 200000 and "linger ms" to 300.