Appian integrated with Kafka

RE: Appian integrated with Kafka

kaushik.vejju — Tue, 10 Sep 2024 17:31:40 GMT

Did you ever resolve this issue? Currently encountering the same problem on my end.

RE: Appian integrated with Kafka

Sumit Fadale — Wed, 13 Mar 2024 04:30:56 GMT

I am trying to produce a message on my topic. I am getting an exception which is a timeout exception. Topic not present in metadata after 60000ms.

But the same topic I am using through the spring boot application and it is working as expected. how to solve this issue.

RE: Appian integrated with Kafka

qihans0001 — Wed, 24 Jan 2024 15:49:34 GMT

Hi Sirb,

Just quick question where did you find these error listed? Cannot find any logs related to Kafka Consumer errors. And there is no error in PM as well.

Thank you so much!

RE: Appian integrated with Kafka

srib0003 — Wed, 31 May 2023 11:56:23 GMT

[quote userid="35227" url="~/discussions/f/integrations/22156/appian-integrated-with-kafka/113571#113571"]Did you create a third party credentials on the admin console? and use it as a 'secure credentials store key' value on the smart service?[/quote]

We have confusions in creating keystore and adding them to the third party credentials.

Can you please explain how did you generate the keystore, what are the parameter that used from kafka and appian side?

RE: Appian integrated with Kafka

srib0003 — Wed, 31 May 2023 10:11:39 GMT

Thanks for your reply!!

Yes, we have created a third party credentials on the admin console and used it as a 'secure credentials store key' value on the smart service.

Have you used self signed certificate (keystore) or signed by any CA authority ?

RE: Appian integrated with Kafka

Mr. Agda — Wed, 31 May 2023 09:23:02 GMT

in our case we left the truststore and keystore as blank on the consume smart service. From what I see on your logs, you are getting a password verification issue.
Did you create a third party credentials on the admin console? and use it as a 'secure credentials store key' value on the smart service?

RE: Appian integrated with Kafka

srib0003 — Wed, 31 May 2023 08:48:02 GMT

Hello Agda,

We are trying to consume data from Kafka topic but getting below error, can you please help step by step procedure to configure the keystore and truststore. We struck with the below issue and your guidance would be highly appreciated.

2023-05-25 11:44:18,210 [Thread-6289] ERROR com.appiancorp.cs.plugin.kafka.consumer.AppianKafkaConsumerThread - Received an exception from consumer: 13640642 - Failed to construct kafka consumer
org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:830)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:665)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:646)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:626)
at com.appiancorp.cs.plugin.kafka.consumer.AppianKafkaConsumer.run(AppianKafkaConsumer.java:94)
at com.appiancorp.cs.plugin.kafka.consumer.AppianKafkaConsumerThread.run(AppianKafkaConsumerThread.java:30)
Caused by: org.apache.kafka.common.KafkaException: Failed to load SSL keystore /usr/local/appian/ae/_admin/accdocs2/8355/10010502.jks of type JKS
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.load(DefaultSslEngineFactory.java:375)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.<init>(DefaultSslEngineFactory.java:347)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createTruststore(DefaultSslEngineFactory.java:320)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:168)
at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:140)
at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:97)
at org.apache.kafka.common.network.SslChannelBuilder.configure(SslChannelBuilder.java:73)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:737)
... 5 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:792)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:57)
at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:224)
at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:71)
at java.security.KeyStore.load(KeyStore.java:1445)
at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$FileBasedStore.load(DefaultSslEngineFactory.java:372)
... 15 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:790)
... 20 more

RE: Appian integrated with Kafka

Mr. Agda — Wed, 24 May 2023 02:36:32 GMT

We are able to use this kafka plugin in appian. We are able to consume topics directly to the database though this also relies on the transaction manager.

RE: Appian integrated with Kafka

Harsha Pasunuri — Mon, 01 May 2023 20:52:46 GMT

Has anyone ever figured out the solution to capture Kafka message inside Appian ? Any help is greatly appreciated as there is no documentation around this technical feature. Thanks!

RE: Appian integrated with Kafka

Mr. Agda — Wed, 01 Dec 2021 05:32:27 GMT

Where did you get the private key?
I only recieve a username and password for credentials

RE: Appian integrated with Kafka

Marta — Thu, 25 Nov 2021 12:09:50 GMT

Hi Mr. Agda, if you download the .zip file for the plugin, you'll find a guide inside.

I was able to connect to Confluent Cloud to produce messages, but I wasn't able to consume (you need to install the Transaction Manager in order to properly configure it).

RE: Appian integrated with Kafka

Mr. Agda — Mon, 22 Nov 2021 07:58:35 GMT

I am trying to use this plugin as well. But I can't configure it properly to work. I can't find any documentations on how to use it in appian or how it works