Introduce support for three separate language models in the retrieval-agent benchmark framework:

• Agent model (retrieval_agent.llm_model): Used for retrieval/planning agent
• Answer model (retrieval_agent.answer_llm_model): Used for answer generation
• Judge model (retrieval_agent.judge_llm_model): Used for LLM judge evaluation

Each model falls back to llm_model if not explicitly set.

Purpose of the change

The retrieval-agent benchmark framework previously used a single language model (retrieval_agent.llm_model) for all operations: agent planning, answer generation, and LLM judge evaluation. This limited flexibility when users wanted to use different models for different tasks (e.g., a smaller/faster model for agent planning, a more capable model for answer generation, and a separate model for evaluation).

Description

Introduce support for three separate language models in the retrieval-agent benchmark framework:

• Agent model (retrieval_agent.llm_model): Used for retrieval/planning agent
• Answer model (retrieval_agent.answer_llm_model): Used for answer generation
• Judge model (retrieval_agent.judge_llm_model): Used for LLM judge evaluation

Each optional model (answer_llm_model, judge_llm_model) falls back to llm_model if not explicitly set, ensuring backward compatibility.

Changes include:

• Update init_memmachine_params() in agent_utils.py to return two models and their IDs
• Update all benchmark callers (wikimultihop, locomo, hotpotqa, longmemeval) to handle the new return signature
• Update llm_judge.py to resolve judge_llm_model with fallback to llm_model
• Update evaluate.py to record judge_model_id in evaluation results
• Update README.md with three-model configuration documentation

Scope is limited to The changes are limited to the evaluation/retrieval_agent/ and evaluation/utils/ directories. No changes to core MemMachine server functionality. The configuration.yml schema extends existing retrieval_agent section with two optional fields.

Fixes/Closes

Fixes #1264

Type of change

[Please delete options that are not relevant.]

• New feature (non-breaking change which adds functionality)
• Documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.

[Please delete options that are not relevant.]

• Unit Test
• Test Script (please provide)

Test Results: [Attach logs, screenshots, or relevant output]

./run_test.sh wikimultihop multi_model search retrieval_agent 100 --search-concurrency 1 --judge-concurrency 2

(.venv) dkjeong@Supermicro-EMR:~/mm/evaluation/retrieval_agent$ cd result/
(.venv) dkjeong@Supermicro-EMR:~/mm/evaluation/retrieval_agent/result$ ll
total 1744
drwxrwxr-x 3 dkjeong dkjeong   4096  4월 28 14:58 ./
drwxrwxr-x 4 dkjeong dkjeong   4096  4월 28 14:34 ../
drwxrwxr-x 2 dkjeong dkjeong   4096  4월 28 14:58 final_score/
-rw-rw-r-- 1 dkjeong dkjeong 887178  4월 28 14:58 wikimultihop_retrieval_agent_evaluation_metrics_multi_model.json
-rw-rw-r-- 1 dkjeong dkjeong 883476  4월 28 14:58 wikimultihop_retrieval_agent_output_multi_model.json

• wikimultihop_retrieval_agent_output_multi_model.json — Contains the generated answers along with answer_model_id indicating which answer model was used.
• wikimultihop_retrieval_agent_evaluation_metrics_multi_model.json — Contains the evaluation metrics along with judge_model_id indicating which judge model was used.

Checklist

[Please delete options that are not relevant.]

• I have signed the commit(s) within this pull request
• My code follows the style guidelines of this project (See STYLE_GUIDE.md)
• I have performed a self-review of my own code
• I have commented my code
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added unit tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• I have checked my code and corrected any misspellings

Maintainer Checklist

• Confirmed all checks passed
• Contributor has signed the commit(s)
• Reviewed the code
• Run, Tested, and Verified the change(s) work as expected

Screenshots/Gifs

Further comments

[Add any other relevant information here, such as potential side effects, future considerations, or any specific questions for the reviewer. Otherwise, type "None".]