APARSIN: A Multi-Variety Sentiment and Translation Benchmark for Iranic Languages

Sadegh Jafari , Tara Azin , Farhad Roodi , Zahra Dehghani Tafti , Mehrdad Ghadrdan , Elham Vatankhahan Esfahani , Aylin Naebzadeh , Mohammadhadi Shahhosseini , Ghafoor Khan , Kazem Forghani , Danial Namazi , Seyed Mohammad Hossein Hashemi , Farhan Farsi , Mohammad Osoolian , Maede Mohammadi , Mohammad Erfan Zare , Muhammad Hasnain Khan , Muhammad Hussain , Nooreen Zaki , Joma Mohammadi , Shayan Bali , Mohammad Javad Ranjbar , Els Lefever , Veronique Hoste

The Proceedings of the 1st Silkroad Workshop, EACL, Rabat, Morocco

March 1, 2026

Abstract

The Iranic language family includes many underrepresented languages and dialects that remain largely unexplored in modern NLP research. We introduce APARSIN, a multi-variety benchmark covering 14 Iranic languages, dialects, and accents, designed for sentiment analysis and machine translation. The dataset includes both high and low-resource varieties, several of which are endangered, capturing linguistic variation across them. We evaluate a set of instruction-tuned Large Language Models (LLMs) on these tasks and analyze their performance across the varieties. Our results highlight substantial performance gaps between standard Persian and other Iranic languages and dialects, demonstrating the need for more inclusive multilingual and dialectally diverse NLP benchmarks.

DOI PDF