Appian integrated with Kafka

I am trying to use this plugin as well. But I can't configure it properly to work. I can't find any documentations on how to use it in appian or how it works

I am trying to use this plugin as well. But I can't configure it properly to work. I can't find any documentations on how to use it in appian or how it works

Hi Mr. Agda, if you download the .zip file for the plugin, you'll find a guide inside.

I was able to connect to Confluent Cloud to produce messages, but I wasn't able to consume (you need to install the Transaction Manager in order to properly configure it).

Where did you get the private key? 
I only recieve a username and password for credentials

Has anyone ever figured out the solution to capture Kafka message inside Appian ? Any help is greatly appreciated as there is no documentation around this technical feature. Thanks!

We are able to use this kafka plugin in appian. We are able to consume topics directly to the database though this also relies on the transaction manager. 

Hello Agda, 

We are trying to consume data from Kafka topic but getting below error, can you please help step by step procedure to configure the keystore and truststore. We struck with the below issue and your guidance would be highly appreciated.

2023-05-25 11:44:18,210 [Thread-6289] ERROR com.appiancorp.cs.plugin.kafka.consumer.AppianKafkaConsumerThread - Received an exception from consumer: 13640642 - Failed to construct kafka consumer
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:830)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:665)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:646)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:626)
at com.appiancorp.cs.plugin.kafka.consumer.AppianKafkaConsumer.run(AppianKafkaConsumer.java:94)
at com.appiancorp.cs.plugin.kafka.consumer.AppianKafkaConsumerThread.run(AppianKafkaConsumerThread.java:30)
Caused by: org.apache.kafka.common.KafkaException: Failed to load SSL keystore /usr/local/appian/ae/_admin/accdocs2/8355/10010502.jks of type JKS
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.load(DefaultSslEngineFactory.java:375)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.<init>(DefaultSslEngineFactory.java:347)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createTruststore(DefaultSslEngineFactory.java:320)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:168)
at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:140)
at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:97)
at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:73)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:737)
... 5 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:792)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:57)
at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:224)
at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:71)
at java.security.KeyStore.load(KeyStore.java:1445)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.load(DefaultSslEngineFactory.java:372)
... 15 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:790)
... 20 more

in our case we left the truststore and keystore as blank on the consume smart service. From what I see on your logs, you are getting a password verification issue. 
Did you create a third party credentials on the admin console? and use it as a 'secure credentials store key' value on the smart service? 

Thanks for your reply!!

Yes, we have created a third party credentials on the admin console and used it as a 'secure credentials store key' value on the smart service.

Have you used self signed certificate (keystore) or signed by any CA authority ? 

Mr. Agda said:

Did you create a third party credentials on the admin console? and use it as a 'secure credentials store key' value on the smart service?

We have confusions in creating keystore and adding them to the third party credentials.

Can you please explain how did you generate the keystore, what are the parameter that used from kafka and appian side?

Hi Sirb,

Just quick question where did you find these error listed? Cannot find any logs related to Kafka Consumer errors. And there is no error in PM as well.

Thank you so much!

I am trying to produce a message on my topic. I am getting an exception which is a timeout exception. Topic not present in metadata after 60000ms.

But the same topic I am using through the spring boot application and it is working as expected. how to solve this issue.