Datacenter User Guide

Version 2.2 | Published May 16, 2024 ©

Kafka

This section provides the steps to configure a Kafka dataset, which can be used to connect your outputs to data incoming from one of more Apache Kafka Brokers.

  1. Create a new dataset of Type Kafka, specifying a name and (optionally) a prefix to use to identify the dataset and its values:

    images/download/thumbnails/149924392/image-2024-1-24_16-13-53.png
  2. Click Add. The dataset is added to the list, and it has an initial WARNING status as it contains no Kafka broker source yet.

    images/download/thumbnails/149924392/image-2023-8-15_17-28-27.png
  3. You can now select a host to replicate from the table in the Dataset configuration panel by double clicking a row. The table lists all the recently used hosts.

    images/download/attachments/149924392/image-2024-1-24_16-15-53.png


    Alternatively, you can add a new host by clicking on Add new host on the right-side of the panel, and then clicking on the empty host that is added at the bottom of the table.

    images/download/attachments/149924392/image-2023-8-15_17-0-31.png
  4. Doing so reveals the new host details, where its properties can be entered.

    images/download/attachments/149924392/image-2024-1-24_12-7-33.png
  5. If the specified host runs a Kafka Broker, the topics advertised by the broker are listed in the dropdown Topics menu available in the Dataset parameters section.
    images/download/attachments/149924392/image-2024-1-24_12-8-52.png images/download/thumbnails/149924392/image-2024-1-24_12-11-32.png
    Select one or more topics from this list. The data published on these topics are added to the Dataset Entries table.

Info: Datacenter expects from the Kafka broker messages to contain valid JSON as payload. All other formats are discarded and the dataset state is set to Warning.

Note: Data from a Kafka data provider is typically event based. Therefore, if your broker publishes updating data (values of the same key that change over time), the data that is pushed to Datacenter updates the graphics instantly.

Please refer to the Filtering and Manipulating the Dataset Entries page to learn how to enable, disable and edit the entries added.