Kafka Tools

Overview

KafkaTools provides the following functionality that allows publishing to and consuming from topics on Kafka Servers.

Smart Services

  • Publish To Kafka
  • Consume From Kafka

In order to process the messages consumed from Kafka, it is recommended to use the Transaction Manager application. The models designed to process messages will be configured an assigned through the transaction manager job types. See documentation for Transaction Manager

Key Features & Functionality

Please refer to the README for additional details.

Anonymous
Parents
  • Hi Sylvain,

    We are using this plugin to post messages to Kafka Topic. We would like to also send headers with the message. How can we do that using this plugin. We are using SASL Mechanism as 'PLAIN'.

    Thanks in Advance. 

  • Hi

    Request you to please let us know - if this enhancement will be taken in upcoming version

  • Bharat - it seems that the code is pretty straightforward to update.

    My only observation would be to change the header parameter from required to optional.

    I would recommend for you to make the updates and submit a new version of the plug-in to community. As you may know, plug-ins are not owned by any individual. They are community supported so feel free to update as needed.

  • Hi Sylvain,

    In continue Ketan's thread, I have check plug-in version 'kafka-tools-1.0.4. and we still not have option to pass the 'Header' as parameter with 'Topic' details.

    Below changes requires in plug-in code in order to fulfill our requirement

    Request to add in plug-in and share the latest version

    1) PublishKafkaSmartService.java 

    private String _header;

    @Order({
    // inputs
    "SecureCredentialsStoreKey", "Servers", "Topic", "Header", "Payload", "SecurityProtocol", "SASLMechanism",
    "Truststore", "Keystore",

    // outputs
    "Success", "ErrorMessage" })

    kafkaProducer.publish(_topic, _payload, _header);

    @Input(required = Required.ALWAYS)
    public void setHeader(String val) {
    this._header = val;
    }

    2) AppianKafkaProducer.java

    // _producer.send(new ProducerRecord<>(payload, header),
    ProducerRecord<String, String> record = new ProducerRecord<>(topic, payload, header);
     

    Thanks,

    Bharat

Comment
  • Hi Sylvain,

    In continue Ketan's thread, I have check plug-in version 'kafka-tools-1.0.4. and we still not have option to pass the 'Header' as parameter with 'Topic' details.

    Below changes requires in plug-in code in order to fulfill our requirement

    Request to add in plug-in and share the latest version

    1) PublishKafkaSmartService.java 

    private String _header;

    @Order({
    // inputs
    "SecureCredentialsStoreKey", "Servers", "Topic", "Header", "Payload", "SecurityProtocol", "SASLMechanism",
    "Truststore", "Keystore",

    // outputs
    "Success", "ErrorMessage" })

    kafkaProducer.publish(_topic, _payload, _header);

    @Input(required = Required.ALWAYS)
    public void setHeader(String val) {
    this._header = val;
    }

    2) AppianKafkaProducer.java

    // _producer.send(new ProducerRecord<>(payload, header),
    ProducerRecord<String, String> record = new ProducerRecord<>(topic, payload, header);
     

    Thanks,

    Bharat

Children
  • Bharat - it seems that the code is pretty straightforward to update.

    My only observation would be to change the header parameter from required to optional.

    I would recommend for you to make the updates and submit a new version of the plug-in to community. As you may know, plug-ins are not owned by any individual. They are community supported so feel free to update as needed.