Loading eval run detail…
Loading eval run detail…
Run inspection
RAG QA Seed · gemini-3.1-flash-lite · v1
75c1cd78-5cae-49fa-be97-d0113957904b
Pass rate
0.0%
Share of cases meeting the grader threshold
Average score
0.000
Mean final score after configured grader weighting
Cost
$0
Estimated total model cost
Avg latency
149.6 ms
P95 214 ms
Failed cases
20
Cases below the composite threshold
Status
Progress
Errors
Total cost
Average latency
P95 latency
Created
Started
Completed
Dataset ID
f67a8007-d9ce-4816-9bed-676547a5f78b
Prompt version ID
76f01278-2380-4481-82d0-51394f25f108
Model
gemini-3.1-flash-lite
Pass rate and score by workflow type.
| Key | Pass rate | Avg score | Cases | Avg latency | Cost |
|---|---|---|---|---|---|
| rag_qa | 0.0% | 0.000 | 0/2020 failed | 149.6 ms | $0.000000 |
Pass rate and score by difficulty.
| Key | Pass rate | Avg score | Cases | Avg latency | Cost |
|---|---|---|---|---|---|
| easy | 0.0% | 0.000 | 0/88 failed | 179.9 ms | $0.000000 |
| hard | 0.0% | 0.000 | 0/55 failed | 137.4 ms |
Inspect model inputs, deterministic scores, judge feedback, and rubric details.
Showing 20 of 20
| Test case | Workflow | Difficulty | Final score | Judge score | Failure modes | Details |
|---|---|---|---|---|---|---|
1fcd0247-1220-4a06-b462-bbc543308fe6 | rag qa | easy | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
1ff5795f-b8d0-4ccf-ba8d-dd70e457f8cb | rag qa | medium | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
2c106ddc-d43e-4b74-8c18-b9025cf8f488 | rag qa | easy | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
2e3e13c9-2cdc-48b9-bd28-bdc80ddfdc1b | rag qa | easy | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
334fc972-81eb-409d-ae77-c66fcadb0e7c | rag qa | hard | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
3cbbf6ee-04c5-41d8-9552-09f9546957c3 | rag qa | medium | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
48f10ff4-1f2e-4485-8475-d660ace8291e | rag qa | medium | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
4ac9f9cc-24f0-4bfc-b1e8-e45bf7d03436 | rag qa | easy | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
4d878405-abbc-41d5-9f56-97f69eac808e | rag qa | hard | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
5e9825dc-f444-496c-b4c7-6ba40904115b | rag qa | medium | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
7dc549a8-6a4a-4813-be3a-2dc743d1cd6e | rag qa | medium | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
80842481-39a0-4791-b2cf-f75b86c94913 | rag qa | easy | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
84e9370b-511f-4ab3-8f11-42ff4a0aca2d | rag qa | medium | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
9be3a757-073c-43c4-8498-fceb4c66ac1c | rag qa | hard | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
a3afb5e4-0820-4102-a879-7e7f7ed828e1 | rag qa | easy | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
ab2c94b9-5432-4cf6-a685-b1c6c799575d | rag qa | hard | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
ad45517e-4142-4336-bcae-4807f1d84ab7 | rag qa | hard | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
cc3f0bd7-b8a3-4910-8140-8f7a984fa6e0 | rag qa | medium | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
e8b0f977-794b-4a5b-b8eb-953917641561 | rag qa | easy | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | |
f54f13fd-1821-4ef6-9b22-a12edfcd4652 | rag qa | easy | 0.000 | — | provider errorinvalid jsonmissing actual textmissing citationunsupported claim |
20 results with expandable grader details
| Test case | Final score | Judge score | Outcome | Failure modes | Latency | Input tokens | Output tokens | Cost | Error | Grader details | Model output |
|---|---|---|---|---|---|---|---|---|---|---|---|
1fcd0247-1220-4a06-b462-bbc543308fe6 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 408 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} |
| $0.000000 |
| medium | 0.0% | 0.000 | 0/77 failed | 123.7 ms | $0.000000 |
Pass rate and score by dataset tags.
| Key | Pass rate | Avg score | Cases | Avg latency | Cost |
|---|---|---|---|---|---|
| api | 0.0% | 0.000 | 0/22 failed | 263.5 ms | $0.000000 |
| auth | 0.0% | 0.000 | 0/22 failed | 138 ms | $0.000000 |
| billing | 0.0% | 0.000 | 0/33 failed | 115.7 ms | $0.000000 |
| communication | 0.0% | 0.000 | 0/11 failed | 145 ms | $0.000000 |
| compliance | 0.0% | 0.000 | 0/22 failed | 153 ms | $0.000000 |
| datasets | 0.0% | 0.000 | 0/11 failed | 119 ms | $0.000000 |
| distractors | 0.0% | 0.000 | 0/11 failed | 112 ms | $0.000000 |
| errors | 0.0% | 0.000 | 0/11 failed | 119 ms | $0.000000 |
| exports | 0.0% | 0.000 | 0/22 failed | 123.5 ms | $0.000000 |
| feature-flags | 0.0% | 0.000 | 0/22 failed | 128 ms | $0.000000 |
| incidents | 0.0% | 0.000 | 0/33 failed | 160 ms | $0.000000 |
| invoices | 0.0% | 0.000 | 0/11 failed | 106 ms | $0.000000 |
| limits | 0.0% | 0.000 | 0/11 failed | 408 ms | $0.000000 |
| logs | 0.0% | 0.000 | 0/11 failed | 158 ms | $0.000000 |
| multi-doc | 0.0% | 0.000 | 0/22 failed | 138 ms | $0.000000 |
| negation | 0.0% | 0.000 | 0/11 failed | 136 ms | $0.000000 |
| oncall | 0.0% | 0.000 | 0/22 failed | 149.5 ms | $0.000000 |
| paraphrase | 0.0% | 0.000 | 0/11 failed | 163 ms | $0.000000 |
| privacy | 0.0% | 0.000 | 0/33 failed | 131 ms | $0.000000 |
| refunds | 0.0% | 0.000 | 0/11 failed | 113 ms | $0.000000 |
| rollout | 0.0% | 0.000 | 0/11 failed | 108 ms | $0.000000 |
| safety | 0.0% | 0.000 | 0/11 failed | 148 ms | $0.000000 |
| security | 0.0% | 0.000 | 0/22 failed | 136.5 ms | $0.000000 |
| sla | 0.0% | 0.000 | 0/33 failed | 163 ms | $0.000000 |
| tricky | 0.0% | 0.000 | 0/33 failed | 142.3 ms | $0.000000 |
View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: rate-limits. Unsupported claims: 1,000 requests.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} |
— |
1ff5795f-b8d0-4ccf-ba8d-dd70e457f8cb | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 133 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: privacy-policy. Unsupported claims: never used to train.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
2c106ddc-d43e-4b74-8c18-b9025cf8f488 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 128 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 2. Missing required citations: export-guide. Unsupported claims: JSONL, CSV.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
2e3e13c9-2cdc-48b9-bd28-bdc80ddfdc1b | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 140 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: password-reset. Unsupported claims: 60 minutes.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
334fc972-81eb-409d-ae77-c66fcadb0e7c | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 163 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 2. Missing required citations: oncall-runbook. Unsupported claims: platform on-call, 20%.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
3cbbf6ee-04c5-41d8-9552-09f9546957c3 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 119 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: rate-limits. Unsupported claims: HTTP 429.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
48f10ff4-1f2e-4485-8475-d660ace8291e | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 145 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: sla-guide. Unsupported claims: 30 minutes.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
4ac9f9cc-24f0-4bfc-b1e8-e45bf7d03436 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 130 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: sla-guide. Unsupported claims: 15 minutes.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
4d878405-abbc-41d5-9f56-97f69eac808e | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 148 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 2, cited 0, missing 2, invalid 0, unsupported claims 2. Missing required citations: privacy-policy, data-retention. Unsupported claims: AES-256, 365 days.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
5e9825dc-f444-496c-b4c7-6ba40904115b | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 136 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: oncall-runbook. Unsupported claims: 20%.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
7dc549a8-6a4a-4813-be3a-2dc743d1cd6e | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 106 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: billing-cycles. Unsupported claims: first calendar day.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
80842481-39a0-4791-b2cf-f75b86c94913 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 158 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: data-retention. Unsupported claims: 365 days.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
84e9370b-511f-4ab3-8f11-42ff4a0aca2d | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 119 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 2. Missing required citations: export-guide. Unsupported claims: JSONL, 24 hours.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
9be3a757-073c-43c4-8498-fceb4c66ac1c | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 112 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: privacy-policy. Unsupported claims: never used to train.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
a3afb5e4-0820-4102-a879-7e7f7ed828e1 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 214 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: sla-guide. Unsupported claims: 15 minutes.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
ab2c94b9-5432-4cf6-a685-b1c6c799575d | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 136 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: password-reset. Unsupported claims: 60 minutes.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
ad45517e-4142-4336-bcae-4807f1d84ab7 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 128 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 2, cited 0, missing 2, invalid 0, unsupported claims 2. Missing required citations: refund-policy, billing-cycles. Unsupported claims: 30 days, first calendar day.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
cc3f0bd7-b8a3-4910-8140-8f7a984fa6e0 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 108 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 2. Missing required citations: feature-flags. Unsupported claims: 1%, workspace ID.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
e8b0f977-794b-4a5b-b8eb-953917641561 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 148 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: feature-flags. Unsupported claims: workspace ID.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |
f54f13fd-1821-4ef6-9b22-a12edfcd4652 | 0.000 | — | Failed | provider errorinvalid jsonmissing actual textmissing citationunsupported claim | 113 ms | — | — | — | 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | View grader detailsProvider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Composite score 0.000 with pass threshold 0.850. json_schema=0.000 (Model output is not a valid JSON object.) text_similarity=0.000 (Actual text is missing.) citation=0.000 (Citation score 0.000. Required 1, cited 0, missing 1, invalid 0, unsupported claims 1. Missing required citations: refund-policy. Unsupported claims: 30 days.) LLM judge error: Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}}. Deterministic graders
LLM judge: — No judge reason available. Gemini judge provider error: 429 RESOURCE_EXHAUSTED. {'error': {'code': 429, 'message': 'Your prepayment credits are depleted. Please go to AI Studio at https://ai.studio/projects to manage your project and billing. Learn more at https://ai.google.dev/gemini-api/docs/billing#prepay. ', 'status': 'RESOURCE_EXHAUSTED'}} | — |