Skip to main content
abdulrahimb9853
abdulrahimb9853
February 28, 2025
Solved

Token Use

  • February 28, 2025
  • 4 replies
  • 0 views

Hi Appian Community,

Quick question: Is there any way for Appian Gen-AI to tell us how many tokens are used for a particular task, from input to output?

I believe there isn’t, but just in case, I wanted to ask. The purpose of this is to analyze token usage so developers can be better informed.

Thank you!

Best answer by karumurua531442

Hi [mention:c156a43eea0d473194ad62f647367f54:e9ed411860ed4f2ba0265705b8793d05] AFAIK there is no way  to count how many tokens used. A model breaks down data into meaningful parts  tokens before it begins processing that data. Language processing models break down paragraphs of text into their smallest units, but these units aren't always equivalent to words. Instead, a token could be part of a word. Therefore, the number of tokens won't always equal the number of words in the text analyzed.

4 replies

karumurua531442
karumurua531442Answer
February 28, 2025

Hi [mention:c156a43eea0d473194ad62f647367f54:e9ed411860ed4f2ba0265705b8793d05] AFAIK there is no way  to count how many tokens used. A model breaks down data into meaningful parts  tokens before it begins processing that data. Language processing models break down paragraphs of text into their smallest units, but these units aren't always equivalent to words. Instead, a token could be part of a word. Therefore, the number of tokens won't always equal the number of words in the text analyzed.

    abdulrahimb9853
    abdulrahimb9853Author
    February 28, 2025

    I see, thank you for the reply.

    konduruc733182
    konduruc733182
    February 28, 2025

    Simple answer to your question is NO, there is no proven method to monitor or analyze the token consumption data.

      abdulrahimb9853
      abdulrahimb9853Author
      February 28, 2025

      Alright, thank you for the reply. 

      Terms & ConditionsAccessibility statement