Kafka Tools

Overview

KafkaTools provides the following functionality that allows publishing to and consuming from topics on Kafka Servers.

Smart Services

  • Publish To Kafka
  • Consume From Kafka

In order to process the messages consumed from Kafka, it is recommended to use the Transaction Manager application. The models designed to process messages will be configured an assigned through the transaction manager job types. See documentation for Transaction Manager

Key Features & Functionality

Please refer to the README for additional details.

Anonymous
  • Hi,

    We are considering using this plug-in with a client and we have seen that the smart service Consume from Kafka does not return information about the recovered messages and it's necessary to use the Transaction manager application.

    Is there a way to retrieve and process the events from Kafka directly without the Transaction manager application? Has anyone tried something similar?

    Thanks in advance

  • Please someone help me to know which Kafka client version is being used by below mentioned versions of Kafka tools

    * Kafka-tools 1.0.2

    * Kafka-tools 1.4.1

  • Does anybody know what version of Kafka this plugin is intended to be used with? I don't see any kafka version in the associated documentation. I'm attempting to connect to a kafka server running kafka version 2.7 but I'm continually getting an error.

  • Yep that was the issue. Thank you very much.

  • Try it out the label with underscores: "SASL_SHA_512" could be work for you

  • Hi,

    I am facing an issue when publishing to a Kafka queue. When running the node it is returning the error [Invalid sasl mechanism]. I have double checked the mechanism value in the config file of the server and it is SCRAM-SHA-512 which is what I have configured in the smart service node. 

  • Hi, as those JKS are in general not portable between environments, we use to keep them inside "XXX_Infra" applications that are exists on every environments, then use a constant of type Integer with values relative to the environment and use it when configuring the Smart Service. As this constant is Environment Specific treat it in your CI/CD the same way you already do with other customization files

  • Hi Team,

    Use case: Given a process model with a Smart service "Consume from Kafka" configured with the "Runtime in minutes" parameter to 58 minutes.

    We have detected that if we manually cancel the instance of the live process model (which is alive for the entire subscription, 58 minutes) when it has spent, for example, 30 minutes of instance execution, it seems that the events continue arriving to Appian, that is, the subscription is still alive even after canceling the process model instance that has the Smart Service "Consume From Kafka".

    Once 58 minutes have passed, topic events stop arriving at Appian, even though the process model was canceled about 30 minutes before.

    We don't see any reference to that behavior in the documentation, but it is a behavior that is happening to us, and we understand that it happens to everyone who uses this plugin.

    Can someone confirm this behavior?

  • Hi,

    Does this Appian plugin community.appian.com/.../kafka-tools use the Kafka Rest API protocol?


    Thanks

  • Hi Team,

    We are facing issue while producing the message using Kafka Tools plugin (Publish to Kafka). We are getting an error with log4j.  The error is:

    This repository is not sync with any central repository. Please help us to solve the issue.