Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark

Su ; Wenhao ; Chen ; Siyu ; Zhao ; Chloe
Abstract
Multi-hop question answering requires a retrieval system to assemble several complementary evidence documents before an answer module can reason reliably. Single-shot retrieval is efficient, but it often misses later-hop evidence when a question combines source, time, comparison, and entity constraints. This paper evaluates a budgeted multi-hop retrieval agent for compositional question answering on the official MultiHop-RAG benchmark. The benchmark contains 2,556 queries and 609 news-article corpus documents, with answerable evidence distributed across two to four documents. Four retrieval policies are compared under the same sparse lexical scorer: fixed top-k retrieval, iterative retrieval, query decomposition, and the proposed budgeted retrieval agent. The revised evaluation frames the task as retrieval-policy evaluation rather than as a full free-form generative QA system: retrieval-conditioned EM/F1 are reported together with evidence recall, MRR, retrieval rounds, selected documents, and context-token cost. On the official data, the budgeted agent achieves the strongest overall retrieval-conditioned EM/F1 at 62.75% and the highest final evidence recall at 74.67%, using 3.011 average retrieval calls and 509.7 average context tokens. Query decomposition improves over fixed top-k and iterative retrieval but is less stable across question types. Fixed top-k is cheapest but incomplete on longer chains. The four-hop results remain difficult for every policy, showing that a fixed 620-token controller should be extended with hop-aware or dynamic budget allocation. The findings support a moderated contribution claim: explicit budget control is useful for auditable multi-hop retrieval, but it should be evaluated as a cost-accuracy trade-off rather than as a universally dominant RAG architecture.
Keywords
How to Cite

Su, et al. (2025). Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark. Journal of Technology Informatics and Engineering, 4(3). https://doi.org/10.51903/jtie.v4i3.543

Su, Wenhao; Chen, Siyu ; Zhao, Chloe , "Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark," Journal of Technology Informatics and Engineering, vol. 4, no. 3, 2025.

Su, Wenhao; Chen, Siyu ; Zhao, Chloe . "Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark." Journal of Technology Informatics and Engineering, vol. 4, no. 3, 2025.

Su, Wenhao; Chen, Siyu ; Zhao, Chloe . "Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark." Journal of Technology Informatics and Engineering 4, no. 3 (2025).

Su, et al. (2025) 'Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark', Journal of Technology Informatics and Engineering, 4(3). doi: 10.51903/jtie.v4i3.543.

Su, Wenhao; Chen, Siyu ; Zhao, Chloe . Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark. Journal of Technology Informatics and Engineering. 2025;4(3).

Artikel Terkait
Tren Sitasi Jurnal