AI Analytics and Monitoring

Overview

Monitoring model performance over time: By versioning models and tracking execution statistics like accuracy, the application provides visibility into how well models are performing and where improvements may be needed. This helps ensure models remain effective as data and use cases evolve.

Gathering user feedback: Features like prompt playback and feedback collection allow users to test models, while also surfacing insights on how the output is received and where errors occur most. This feedback loop helps drive better model development.

Standardizing model deployment: The application aims to standardize how models are configured, versioned, tested and deployed across environments. This helps improve reproducibility, as well as governance over how AI/ML solutions are managed within the business.

Key Features & Functionality

Prompt Playground - Allows users to quickly test AI responses using a prompt playground without fully developing a model. Prompts can be exported and imported.

Classification Monitoring - Supports generative and ML-based classification models. Models are versioned and classification accuracy is monitored. Reconciliation is supported.

Extraction Development and Monitoring - Supports structured and unstructured extraction models. Fields can be configured manually or using an extraction tool. Models are versioned and extraction accuracy at the field level is monitored. Reconciliation is supported.

Reconciliation - Framework and custom reconcile forms are supported.

Anonymous
  • v2.1.0 Release Notes
    • New Features:
    • Auto convert input document types for extraction and classifcation (docx, html, jpg, gif, tiff, png)
    • Rollup of classification accuracy and confidence on home page
    • Status for extraction and classification instances (Received, Processed, Reconciled)
    • EML file support
    • Run classification tests on front-end
    • Update from using classifation id to key (requires updating process models and passing in modelKey instead of modelId)
    • Added classification instances to model
    • Bug Fixes:
    • Fixing missing process locales

  • Thats great, thank you Michael

  • Hmm, usually the app market adds the comment automatically, but they are:

    New Features:
    • Auto convert input document types for extraction and classifcation (docx, html, jpg, gif, tiff, png)
    • Rollup of classification accuracy and confidence on home page
    • Status for extraction and classification instances (Received, Processed, Reconciled)
    • EML file support
    • Run classification tests on front-end
    • Update from using classifation id to key (requires updating process models and passing in modelKey instead of modelId)
    • Added classification instances to model
    Bug Fixes:
    • Fixing missing process locales

    Let me know if you are having any trouble as this app is still in active development.

  • Hi, what is the update to this app?