What would LLMs do on climate change policy making? Comparative analysis between generic and African-centric LLMs in South African Parliament

Pfesesani Makongoza , Mukondeleli Negukhula

Partner: ems

Year: 2026

Abstract: This study presents the modelling phase of an analysis examining the use of Large Language Models (LLMs) to measure climate change discourse in South African parliamentary proceedings. Building on exploratory data analysis showing a steady increase in climate-related discussions between 2014 and 2026, the modelling focuses on the 2022-2023 period, with emphasis on the April 2022 KwaZulu-Natal flood event. An event-based Natural Language Processing (NLP) framework is developed to capture shifts in parliamentary attention before, during, and after the disaster. Three approaches are evaluated: a TF-IDF Logistic Regression baseline, a BART zero-shot classifier, and AfriBERTa, an African-centric transformer model. AfriBERTa achieved the strongest overall performance, with an accuracy of 0.9626, precision of 0.9348, and recall of 0.9773, highlighting the effectiveness of African-centric transformer models for parliamentary text classification. Results reveal a clear event-driven spike in climate discourse during April 2022, indicating that parliamentary attention is largely reactive to extreme weather events.

Presentation Video