Aashish Dhawan, Christopher Driggers-Ellis, Dzmitry Kasinets, Daisy Zhe Wang, Christan Grant
AmericasNLP 2026 · ACL · 2026
1st place overall — AmericasNLP 2026 Shared Task on Cultural Image Captioning
We present the University of Florida (Gators) submission to the AmericasNLP 2026 shared task on cultural image captioning for Indigenous languages. Our system uses a two-stage pipeline: it first generates a Spanish intermediate caption for each image with Qwen2.5-VL, then produces the target-language caption via retrieval-augmented, many-shot prompting with Gemini 2.5 Flash. Retrieval surfaces culturally and visually relevant in-context examples, letting a general-purpose LLM translate into low-resource languages it was never explicitly trained on. On the development set the approach improves over the shared-task baseline by 164.1%, 131.7%, and 122.6% for the three language variants, with sustained improvements exceeding 150% on the evaluation set. We find that retrieval effectiveness varies substantially by language and depends on having a sufficiently large in-domain example pool, and that language-specific synthetic data contributes roughly 28 chrF++ points to Guaraní. Our submission was the overall winner of the shared task and ranked second in human evaluation among finalist entries.
Rather than fine-tune a captioning model on scarce Indigenous-language data, we decompose the task into two stages that each play to a strong general-purpose model. In the first stage, Qwen2.5-VL looks at the image and produces a faithful Spanish caption — a high-resource pivot the vision-language model handles reliably.
In the second stage, we translate that caption into the target Indigenous language with Gemini 2.5 Flash using retrieval-augmented, many-shot prompting. For each input we retrieve the most relevant caption pairs from an in-domain example pool and pack them into a long-context prompt, letting the model translate into languages it was never explicitly trained on by analogy to nearby examples.
The system improves over the shared-task baseline by 164.1%, 131.7%, and 122.6% across the three language variants on the development set, with improvements above 150% carried over to the held-out evaluation set. It was the overall winner of the shared task and placed second in the human evaluation among finalist entries.
Two findings stood out. Retrieval effectiveness is highly language-dependent and hinges on a sufficiently large in-domain pool of examples to retrieve from. And targeted synthetic data matters: language-specific augmentation contributed roughly 28 chrF++ points for Guaraní.
Back to Aashish Dhawan's site