Kafka Tools

Overview

KafkaTools provides the following functionality that allows publishing to and consuming from topics on Kafka Servers.

Smart Services

  • Publish To Kafka
  • Consume From Kafka

In order to process the messages consumed from Kafka, it is recommended to use the Transaction Manager application. The models designed to process messages will be configured an assigned through the transaction manager job types. See documentation for Transaction Manager

Key Features & Functionality

Please refer to the README for additional details.

Anonymous
  • Are there plans to support non-Text values like Avro messages? 

    One options is to define a custom (De)serializer class, but this seem not to be used if you define this.. To code seems to assume the value is always a String.  is the support for a customer Deserializer class with non-text values really supported?  

  •  , In the plugin Third-Party Credentials, Scope field missing / not possible to configure, due to that token generation is failing , is that possible to upgrade the plugin with Scope field.

  • We have the same issue. Are there any updates on this?

  • We are able to successfully produce messages to Kafka, but have need for the message to use a Json Serializer. Is it possible to add the ability to use a different serializer than just String serializer when producing messages?

  • We have an issue consuming messages from one topic.

    We use 2 consumers with same configuration (test and acc) to consume from same topic. TEST consumed 2 messages and ACC consumed 3 messages. Both consumers are alive on the topic and have lag 0. Any ideas what could have gone wrong ? Also, any logs where I could check something about what is consumed ?

    We've been using this plugin successfully for the past 2 years to consume from other topics but this has happened twice in past month with a new topic.

    v1.4.2

    Thanks

  • We successfully integrated the Kafka tool in our dev environment. We're now looking to integrate it in our other environments. The Kafka tool requires the setup of third-party credentials. We use Salt Stack to set up all our environments. Is it possible to set up the Kafka Tool third party credentials using Salt? Thanks

  • As a weird workaround, you may be able to upload the kafka certificate into the Admin Console. Login as admin -> go to the admin console -> Integration/Certificates -> Trusted Server Certificates -> New Trusted Server Certificate and upload the PEM here.

    If for some reason that didn't work, you can possibly disable feature toggle `ae.databases-and-search.trust-manager-factory-provider.enabled`.

  • v1.4.3 Release Notes
    • Upgraded kafka-clients library
  • Hello, 


    We are using 1.2.1 version plugin. Recently appian cloud version got updated with 24.4 version. From there onwards it is throwing below error.

    ERROR com.appiancorp.cs.plugin.kafka.consumer.AppianKafkaConsumerThread - Received an exception from consumer: 1177940 - SSL handshake failed

    Could any one please help on this on urgent basis.?

    Thank you in advance.!