Add new paper:

[wyzh0912]

Title

Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models

Published Date

2025-1-14

Source

arxiv

Head Name

Retrieval Head

Summary

• Innovation: Proposed retrieval hit rate, a more direct metric to evaluate attention heads.
• Tasks: In long-context retrieval tasks, identify the retrieval head and utilize it to improve the model's retrieval performance.
• Significant Result: Through Joint Retrieval Head Training, a new retrieval head can be obtained to predict the Top-K context for generating final responses, enhancing the model's in-context retrieval and reasoning capabilities.