Is task manager application necessary to consume data from Kafka?

RE: Is task manager application necessary to consume data from Kafka?

dineshkumark513645 — Fri, 09 Jun 2023 11:14:08 GMT

Hi I just worked on the use case on initial level. Later our requirements changed and we didn't even implemented it.

RE: Is task manager application necessary to consume data from Kafka?

srib0003 — Thu, 08 Jun 2023 18:34:42 GMT

Hello Dinesh,

We are in process of setting up the kafka tool plugin to read data via kafka. We were able to establish the connection.

as you are already using the setup and reading data, can you please explain the steps of reading and processing data using transaction manager.

RE: Is task manager application necessary to consume data from Kafka?

dineshkumark513645 — Wed, 13 Jul 2022 11:49:43 GMT

In Kafka tool node inputs there was a field called "Transaction table Name" with default value "tm_job_transaction" which is table name in Task manager Application. So if we Consume data from Kafka the data will be stored in "tm_job_transaction" table of Task manager Application.

So Even if i consume data in my current application it is going to store the data in table of another application. So if we specify our tablename in node inputs then we can consume data from kafka without using Task manager Application.

Got It by discussing with you. Thanks . :-)

RE: Is task manager application necessary to consume data from Kafka?

Stefan Helzle — Wed, 13 Jul 2022 11:16:32 GMT

Depending on your requirements, use the transaction manager, or implement your own way of pulling the messages from DB and start your own logic.

RE: Is task manager application necessary to consume data from Kafka?

dineshkumark513645 — Wed, 13 Jul 2022 11:15:00 GMT

I have gone through the Documentation provided in Kafka tool.

There they have mentioned the below line.

"Install Transaction Manager and
Follow instructions to create new job_type(s) and process model(s) to handle kafka topics, could be one or multiple depending on requirements."

RE: Is task manager application necessary to consume data from Kafka?

Stefan Helzle — Wed, 13 Jul 2022 11:09:17 GMT

Impossible to say without more details !?!

AFAIK, the plugin itself will only store the messages to DB.