--- title: metalingo-indirect-metaphor canonical_url: "https://www.modelscope.cn/models/TommyLeo/metalingo-indirect-metaphor" md_url: "https://www.modelscope.cn/models/TommyLeo/metalingo-indirect-metaphor.md" repository: TommyLeo/metalingo-indirect-metaphor last_updated: 2026-09-23 license: apache-2.0 pipeline_tag: token-classification tasks: - token-classification model_type: - deberta-v2 architectures: - DebertaV2ForTokenClassification base_model: - microsoft/deberta-v3-large base_model_relation: finetune parameters: 434.0M tensor_type: - F32 library_name: - pytorch - transformer - safetensors frameworks: - Pytorch language: - en downloads: 16 stars: 0 tags: - metaphor-detection - token-classification - knowledge-distillation - deberta-v3 --- # metalingo-indirect-metaphor > metalingo-indirect-metaphor - TommyLeo 在 ModelScope 开源的模型。MetaLingo-Indirect-Metaphor TommyLeo/metalingo-indirect-metaphor 是 ModelScope 魔搭社区上的 434.0M 参数token-classification模型，采用 apache-2.0 许可，基于 microsoft/deberta-v3-large 构建。 - **Repository**: TommyLeo/metalingo-indirect-metaphor - **License**: apache-2.0 - **Tasks**: token-classification - **Parameters**: 434.0M - **Base model**: microsoft/deberta-v3-large - **Tags**: metaphor-detection, token-classification, knowledge-distillation, deberta-v3 - **Downloads**: 16 - **Stars**: 0 - **Last updated**: 2026-09-23 Source: https://www.modelscope.cn/models/TommyLeo/metalingo-indirect-metaphor --- # MetaLingo-Indirect-Metaphor This model is a fine-tuned version of **Microsoft DeBERTa-v3-large** for **binary indirect metaphor detection** in English at token level, trained on the VU Amsterdam Metaphor Corpus (VUAMC). It is produced by a two-stage knowledge distillation pipeline: Stage 1 transfers knowledge from an auxiliary teacher model via soft-label distillation on an out-of-domain reference corpus (BE06, ~954K tokens), and Stage 2 fine-tunes on the VUAMC gold standard with hard labels. Annotation follows MIPVU (Steen et al. 2010); only **indirect metaphors** (`metaphor_type == "met"`) are labeled positive. ## Model description - **Base model:** [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) - **Task:** Token classification (binary metaphor detection) - **Training unit:** Sentences, word-level labels - **Teacher model (Stage 1):** An auxiliary DeBERTa-v3-large indirect-metaphor classifier used to produce soft labels for Stage 1 distillation; not released as a standalone model. ## Two-stage distillation pipeline **Stage 1 — Distillation on BE06 (~954K tokens).** The student is pre-trained with KL-divergence distillation (temperature = 2.0) against soft labels produced by an auxiliary DeBERTa-v3-large teacher specialized for indirect-metaphor detection. All Stage 1 training happens on **BE06**, a British-English reference corpus that is completely independent of VUAMC (different source corpus, different documents, **zero overlap**). This stage exposes the student to a much larger and more varied token distribution than VUAMC alone provides, giving it a strong initialization for metaphor detection before it ever sees a VUAMC sentence. **Stage 2 — Gold fine-tuning on VUAMC (90 train docs).** Starting from the Stage 1 checkpoint, the model is fine-tuned with hard-label cross-entropy on the 90-document official VUAMC training split (10% randomly held out as dev, seed=42), then evaluated once on the 27-document official test split. ## Training & evaluation data - **Dataset:** VUAMC (VU Amsterdam Metaphor Corpus), covering four registers: News, Academic, Fiction, Conversation. - **Split:** Official NAACL FLP partition (Leong et al. 2018): **90 train documents / 27 test documents**. A random 10% of training sentences is held out as dev set (seed=42). - **Test set:** 27 documents, 4,080 sentences, 57,811 tokens, 6,697 gold metaphor tokens. ## Training hyperparameters | | Stage 1 (Distillation on BE06) | Stage 2 (Fine-tune on VUAMC) | |---|---|---| | Loss | KL Divergence (T=2) | Cross-Entropy | | Epochs | 3 | 3 | | Learning rate | 2e-5 | 6e-6 | | Effective batch size | 32 (8×4) | 16 (8×2) | | Max length | 192 | 192 | | Warmup ratio | 0.1 | 0.1 | ## Results Evaluated on the official 27-document test split (NAACL FLP 2018): | Metric | Value | |---|---| | **F1** | **82.29** | | Precision | 85.26 | | Recall | 79.53 | | Accuracy | 96.04 | ### By genre Genre is determined by BNC document ID per the NAACL FLP 2018 shared-task split (News: `a*`; Academic: `b17`, `clw`, `cty`, `ecv`; Fiction: `bmw`, `ccw`, `faj`; Conversation: `k*`). | Genre | Docs | Tokens | Metaphors | Precision | Recall | **F1** | |---|---:|---:|---:|---:|---:|---:| | Academic | 4 | 14,208 | 2,425 | 90.90 | 82.80 | **86.66** | | News | 14 | 13,405 | 1,953 | 86.94 | 77.73 | **82.08** | | Fiction | 3 | 12,764 | 1,079 | 80.72 | 83.04 | **81.86** | | Conversation | 6 | 17,434 | 1,240 | 76.48 | 72.90 | **74.65** | ### By part of speech (Universal POS) Tags with no gold metaphors in the test set (CCONJ, INTJ, NUM, SPACE, SYM, X) are omitted. | POS | Tokens | Metaphors | Precision | Recall | **F1** | |---|---:|---:|---:|---:|---:| | ADP | 4,959 | 1,922 | 93.62 | 90.89 | **92.24** | | DET | 4,163 | 213 | 95.77 | 95.77 | **95.77** | | PRON | 6,545 | 275 | 89.66 | 85.09 | **87.31** | | SCONJ | 1,299 | 99 | 79.63 | 86.87 | **83.09** | | VERB | 6,237 | 1,908 | 84.06 | 77.94 | **80.88** | | PUNCT | 7,627 | 2 | 66.67 | 100.00 | **80.00** | | ADV | 2,527 | 245 | 85.57 | 70.20 | **77.13** | | NOUN | 8,630 | 1,383 | 83.19 | 69.78 | **75.89** | | ADJ | 3,511 | 595 | 81.80 | 68.74 | **74.70** | | PART | 1,881 | 8 | 57.14 | 50.00 | **53.33** | | AUX | 4,219 | 10 | 45.45 | 50.00 | **47.62** | | PROPN | 2,197 | 37 | 57.89 | 29.73 | **39.29** | ## Label dictionary ```json { "0": "non_metaphor", "1": "metaphor" } ``` Subwords are aligned to words via the tokenizer's `word_ids`; the **first subword** of each word is used for prediction. ## Usage example ```python from transformers import AutoTokenizer, AutoModelForTokenClassification import torch model_path = "tommyleo2077/metalingo-indirect-metaphor" # or local path tokenizer = AutoTokenizer.from_pretrained(model_path) model = AutoModelForTokenClassification.from_pretrained(model_path) model.eval() words = ["The", "conference", "threw", "a", "spanner", "in", "the", "works", "."] inputs = tokenizer( words, is_split_into_words=True, return_tensors="pt", truncation=True, max_length=192, ) word_ids = inputs.word_ids(batch_index=0) with torch.no_grad(): logits = model(**inputs).logits preds = logits.argmax(dim=-1)[0].tolist() word_preds = {} for i, wid in enumerate(word_ids): if wid is not None and wid not in word_preds: word_preds[wid] = preds[i] for i, w in enumerate(words): label = "metaphor" if word_preds.get(i, 0) == 1 else "non_metaphor" print(f"{w}\t{label}") ``` ## Citation **Model author:** [Tommy Leo](https://huggingface.co/tommyleo2077) — 1683619168tl@gmail.com **Dataset (MIPVU / VUAMC):** ```bibtex @book{steen2010method, title = {A Method for Linguistic Metaphor Identification: From {MIP} to {MIPVU}}, author = {Steen, Gerard and Dorst, Aletta G. and Herrmann, J. Berenike and Kaal, Anna and Krennmayr, Tina and Pasma, Thea}, year = {2010}, publisher = {John Benjamins} } ``` **Train/test split:** ```bibtex @inproceedings{leong2018vua, title = {A Report on the 2018 {VUA} Metaphor Detection Shared Task}, author = {Leong, Chee Wee and Beigman Klebanov, Beata and Shutova, Ekaterina}, booktitle = {Proceedings of the Workshop on Figurative Language Processing at NAACL-HLT 2018}, year = {2018} } ``` **Base model:** [Microsoft DeBERTa](https://github.com/microsoft/DeBERTa) **This model:** ```bibtex @misc{leo2025metalingoindirectmetaphor, title = {metalingo-indirect-metaphor: Two-Stage Knowledge Distillation for Metaphor Detection}, author = {Leo, Tommy}, year = {2025}, howpublished = {\url{https://huggingface.co/tommyleo2077/metalingo-indirect-metaphor}}, note = {Contact: 1683619168tl@gmail.com} } ``` ## License Apache License 2.0 — see [LICENSE](https://www.apache.org/licenses/LICENSE-2.0) for details.