Why Evaluate Agents

https://google.github.io/adk-docs/assets/evaluate_agent.png

Preparing for Agent Evaluations

Define Success

Identify Critical Tasks

Choose Relevant Metrics

What to Evaluate?

1. Evaluating Trajectory and Tool Use

2. Evaluating the Final Response

How Evaluation works with the ADK

First approach: Using a test file

*.test.json ファイルを使う

This approach involves creating individual test files, each representing a single, simple agent-model interaction (a session).

It's most effective during active agent development, serving as a form of unit testing.

IMO：ユニットテストの粒度でもエージェント（LLMの推論）を実行することになる

Optionally, a folder can also include a test_config.json file that specifies the evaluation criteria.

Second approach: Using An Evalset File

This evalset evaluation method requires the use of a paid service, Vertex Gen AI Evaluation Service API.

code:デフォルト.json

{

"criteria": {

"tool_trajectory_avg_score": 1.0,

"response_match_score": 0.8

}

Recommendations on Criteria（積ん読）

How to run Evaluation with the ADK

1. Web-based UI (adk web)

Step 1: Create and Save a Test Case

Step 2: View and Edit Your Test Case

Step 3: Run the Evaluation with Custom Metrics

Step 4: Analyze Results

2. Programmatically (pytest)

3. Command Line Interface (adk eval)

This runs the same evaluation that runs on the UI, but it helps with automation,