Tropentag, September 16 - 18, 2026, Göttingen "Towards multi-functional agro-ecosystems promoting climate-resilient futures"

Functional assessment of vegetation using knowledge-grounded scene graphs

¹University of Kassel, Agricultural and Biosystems Engineering, Germany
²Swedish University of Agricultural Sciences (SLU), Dep. of Energy and Technology, Sweden

Abstract

Spatial arrangement carries ecological meaning in semi-arid rangelands, where invasive shrubs like Prosopis juliflora suppress useful neighbouring species while canopies such as Acacia tortilis facilitate understory forage. Conventional computer vision tasks, specifically detection and segmentation, often struggle to represent the functional affordances defined as the action possibilities for grazing livestock encoded within this vegetation structure. We address this by proposing a knowledge grounded pipeline that extracts relational scene graphs directly from ground-based rangeland imagery. Our framework utilises a schema of nodes (taxa and cover) and edges (predicates such as shades, facilitates, and dominates) derived from local community knowledge and rangeland ecology literature. Through constrained inference using a state-of-the-art vision language model API, the pipeline enforces strict validation against a community informed schema to mitigate hallucination. To align visual perception with ecological logic, the framework incorporates contextual heuristics that refine assessments based on species specific risks and seasonal dynamics. This allows for dry season overrides that recognise leaf litter from palatable species as a resource and prioritise toxicity signals that are often overlooked by purely visual classifiers. Evaluation on 1,120 images from Marsabit County provided schema valid graphs for every image. Agreement with site-curated quality labels reached a weighted Cohen's kappa of 0.70, with minimal systematic confusion between the Good and Poor rangeland assessment classes. Disagreements predominantly occurred at the Medium to Poor boundary, where the invasive species logic can override aggregate visual features and shift scenes toward Poor. This method maintains a verifiable decision trace by integrating pastoralist logic directly into the analytical process. The results demonstrate that the pipeline captures the relational topology of vegetation patches and offers high model explainability, establishing a foundation for formal reasoning and functional affordance modelling in future ecological research.

Keywords: Functional rangeland assessment, local community knowledge, relational scene understanding

Contact Address: Zakieh Alizadehsani, University of Kassel, Agricultural and Biosystems Engineering, Nordbahnhofstrasse 1a, 37213 Witzenhausen, Germany, e-mail: zakieh.alizadehsaniuni-kassel.de