Skip to main content
dineshkumark513645
July 12, 2022
Question

How to Consume Data from Kafka to appian application?

  • July 12, 2022
  • 7 replies
  • 1 view

Hi Guys,

I had a requirement that my Appian Application needs to consume data from Kafka.

For this i have added a Plug-in called (Kafka tool). It contains 2 smart services. 

1. Consume from Kafka.

2. Publish To Kafka.

So, i used Consume from Kafka Node in my process model. In data tab there are following Node inputs and node outputs to configure.

Node Inputs:

Servers * (Text)
Data Source Name * (Text)
Transaction Table Name * (Text)
Security Protocol * (Text)
Secure Credentials Store Key (Text)
S A S L Mechanism (Text)
Group Id * (Text)
Num Consumers * (Number (Integer))
Topic * (Text)
Job Type Id * (Number (Integer))
Runtime In Minutes * (Number (Integer))
Polling Interval In Ms * (Number (Integer))
Sleeping Interval In Ms * (Number (Integer))
Session Timeout (Number (Integer))
Deserializer Class Name (Text)
Message Filter (Text)
Key Store (Document)
Trust Store (Document)

Node Outputs:

Success (Boolean)
Error Message (Text)

Above Highlighted fields are mandatory fields.

My Doubts:

  1. Do we need to use integration to connect with Kafka? or Not needed as we already have smart services to connect with external System.
  2. If Integration is not required then in node outputs the data coming from node is a BOOLEAN type. So, How can we get the data?
  3. Can anyone guide me with the procedure to consume data from Kafka with sample node inputs and node outputs.
  4. If the procedure i have followed till now was wrong. Please feel free to give your suggestions to achieve my task.
  5. Do i need to do any further Configurations?

    7 replies

    stefanhelzle0001
    Brainy
    July 12, 2022

    Did you try to download the plugin from the AppMarket and check for included documentation?

    A smart service does not need another integration.

    The boolean output is called "success" and will only tell whether a message was retrieved.

    The data from the message will be written to a database table configured in the smart service.

    dineshkumark513645
    July 12, 2022

    Ohh!. Okay got it now. I haven't tried to download the plugin in AppMarket. Now I downloaded the Plugin. It has complete Documentation.

    Thanks [mention:126a676c85024855948336691b0a7b92:e9ed411860ed4f2ba0265705b8793d05]