LLM4Subjects

The task

SemEval-2025's LLMs4Subjects task: assign subject tags from the GND authority file to bilingual (English and German) library records. The label space is over 200,000 entries and the training set is roughly 80,000 records — which puts it squarely in extreme multi-label territory, where most labels are seen a handful of times or never.

A flat classifier over 200k outputs is the obvious approach and the wrong one. The head alone dominates the parameter count, and the long tail gets no gradient signal worth the name.

Approach

• Retrieve-then-rank: narrow 200k candidate labels to a shortlist per record, then score the shortlist — turns an intractable classification into a tractable ranking problem
• Encoder baselines (BERT / RoBERTa) to establish a floor and to sanity-check the retrieval stage in isolation
• Llama3 fine-tuning for the ranking stage, where label semantics — not just label ids — carry the signal
• DeepSpeed for memory-efficient training at the scale the ranking model needed
• Bilingual handling: label semantics are shared across languages, so the label side is embedded once and reused for both EN and DE records

What the experiments showed

Recall at the retrieval stage caps everything downstream — no ranker recovers a label the shortlist never contained, so shortlist recall was the metric worth optimising first.

Using the label's text rather than its id is what makes the tail tractable: a label seen twice in training is still describable, and a model that reads the description can place it.

Stack