The MUNCH (Metaphor Understanding Challenge) Dataset

For testing large language models' capabilities for understanding metaphors as cross-domain mappings.

Overview:

Input data for testing LLMs are in tasks/
- Prompts: prompts.md
- Paraphrase generation: generation.json
- Paraphrase judgement:
o For word judgement, input sentences and options are formatted in the same way for all 3 conditions: word_judge.json
o Sentence judgement: sent_judge_{implicit,msent,mword}.json
o Please always shuffle the 2 given options when formatting your prompts.

Gold labels (human annotations) are in correct_answers/
- For the paraphrase generation task: for_generation.csv
- For the paraphrase judgement task: for_judgement.csv