Performance Stats

How can I get the statistics/metrics of my model performance over time?
For example:
Can I view the margin of error?
Can I see the model’s current confidence levels on certain fields and what exactly it might need?
Do I have any feedback that I can view?

To get the statistics and metrics of your model performance over time, you can utilize these features and tools available in the Instabase AI Hub:

  1. Confidence Scores: Confidence scores are percentage values that indicate the level of certainty in results, including classification, digitization (OCR), or field results. These scores are calculated by the model or OCR processor and are displayed in the field editor. Higher percentages suggest greater confidence. You can use these scores to fine-tune prompts or establish validation rules. (Creating apps | Instabase AI Hub Documentation)
  2. Accuracy Testing: You can perform accuracy tests by comparing app run results against ground truth values to measure performance and identify areas for improvement. The accuracy tests tab for each app displays a list of accuracy tests run against all app versions, summarizing key accuracy metrics for each test. Full accuracy reports include testing metadata and key metrics, providing visibility into the exact data used for the test. (Testing apps | Instabase AI Hub Documentation)
  3. Validation Rules and Feedback: Validation rules based on model confidence scores can flag fields that fail validation in production runs, allowing them to be identified and corrected. This provides immediate feedback about the effectiveness of prompts and supports field-level validation. (Reviewing results | Instabase AI Hub Documentation)
  4. Handling Time and Automation Metrics: Handling time metrics measure the average time to process a document, and automation metrics measure how accurately fields are processed. These metrics can help you understand processing efficiency and identify areas that might require app improvements or additional human review bandwidth. (Reviewing results | Instabase AI Hub Documentation)
  5. Accuracy Metrics: Quantitative measures indicate how valid and accurate app results are when compared to ground truth values. Validated accuracy is the percentage of automated classes or fields that match the ground truth dataset, indicating both validity and accuracy. (Testing apps | Instabase AI Hub Documentation)

By utilizing these tools and metrics, you can effectively monitor and improve your model’s performance over time.