Helmets Labeling Crops Makes Crop ID Simple and Efficient

As climate change causes more frequent droughts, variability, and detrimental levels of food insecurity, it is increasingly important to gather accurate crop and vegetation data to predict and plan for food shortages. Traditional data collection methods require extensive time, effort, and resources. Led by Dr. Catherine Nakalembe, Harvest Africa’s “Helmets Labeling Crops” project aims to partner with agricultural communities and citizens on the ground to increase the data collected while simultaneously decreasing acquisition costs.

The project seeks to create more citizen-driven science and build capacity among local institutions and partners to have a more prominent role in data collection and research. The Helmets project is funded through the Lacuna Fund, a collaborative funding initiative to reduce data gaps and biases in machine learning and artificial intelligence models. 

The gap in data due to a lack of on-site collection is a problem for accurately assessing current and future agricultural production, health, and stability. These datasets cover vast geographic areas and are nearly impossible to label completely with limited on-site field agents, which creates further inconsistencies and gaps. 

The Helmets project began with a focus on five African countries: Kenya, Mali, Rwanda, Tanzania, and Uganda. With the generous help of volunteers and Harvest partners, the Helmets Labeling Crops Project has expanded its data collection efforts to Nigeria, the United States, and France. 

A GoPro attachment on a motorcycle helmet allows for crop data to be gathered by trained citizens who drive around the local area.

How Helmets Labeling Crops Works

The Helmets project’s innovative approach involves capturing street-level images and using computer vision algorithms to convert these images into valuable labeled data. Data collectors drive through target areas and capture images from cameras attached to their helmets or vehicles. These images are then processed according to the NASA Harvest framework called Street2Sat. 

Street2Sat pre-processes the collected data, predicts crop type in individual photos, and estimates the crop field’s distance from the camera. This process provides large amounts of georeferenced and labeled crop-type data in a much faster and less labor-intensive method than traditional labeling techniques. 

The generated labeled data can then be fed into crop-type mapping machine-learning models to produce crop-type maps for the targeted region. This results in a highly efficient data collection process, leading to a substantial reduction in current gaps in the labeled crop-type data required to make crop-type maps.

Data is collected via GoPro camera’s installed on ‘'Helmets" or on cars as teams drive-by cropland. The photos are then processed to classify photo. The distance between the GoPro and the captured crop is then estimated, which is used to correct the GPS coordinates of the given labeled crop location.

Training

The NASA Harvest team produced a simple setup and direction guide for trainees, which outlines setup and optimal capture parameters, and provides participants with an equipment kit and instructions for participating in the data campaign. The guide (accessible on the Helmets Webpage) is in development and will be included in an upcoming website, along with information on how to participate in future data collection efforts. 

The kit includes a GoPro camera and mounting system that enables rapid image gathering of cropland as the trained ‘Helmets’ drive around a designated area. Captured data covers a large area with the help of local teams, which is incredibly useful for maximizing the amount of data gathered within a single field campaign. Local teams are more familiar with the area. They can also work towards building rapport among farming communities as we expand existing crop labeling datasets through this cost-efficient method. 

During the summer of 2022, the RCMRD team and various government agricultural officers hosted training sessions in five different regions in Kenya, inviting trainees from diverse backgrounds and different levels of experience that could also improve the data campaign approach. These training sessions aimed to impart the objectives of the Helmets project and focus on camera setup and technicalities. Similar trainings took place in Tanzania in April, led by partners Sixbert Mourice and Winfried Mbungu from the Sokoine University of Agriculture (SUA). Lutheran World Relief hosted three successful training sessions utilizing OpenDataKit in Segou and Sikasso in Mali in January 2022 and January 2023. 

The training sessions were excellent opportunities to work closely with those collecting the majority of the data and to extend the “scientific hand” to those who would not usually be involved. Fostering relationships with local people is an exciting way to nurture new connections between the scientific and citizen worlds and bring diverse backgrounds and viewpoints to improve data collection methods.

The NASA Harvest team works with 'Helmets' across various African countries and agricultural ministries.

Partners

NASA Harvest and Helmets hope to continue collaborating with our main partners – RCMRD, Lutheran World Relief, Makerere University Kampala, Sokoine University of Agriculture, and Radiant Earth Foundation – to continue to improve tools and data collection methods. These partnerships are integral to continuing the data-gathering process after the main project is completed, leading to lasting, longitudinal relationships that continue to expand datasets. Project sustainability is highly prioritized so that the benefits and partnerships gained along the way are long lasting and fruitful. 

University partnerships, in particular, offer a meaningful opportunity for collaboration with scientists and students in these partner countries. The University of Maryland and the Sokoine University of Agriculture have collaborated on other projects such as AgriSens-STARS, ECAAS, AGRA, and now Helmets, strengthening integral relationships simultaneously. Faculty partners like Dr. Joyce Nakatumba-Nabende, Director of the AI Lab at Makerere University, support the connection and more recently, the National Space Research and Development Agency (NASRDA) of Nigeria requested their own set of Helmets kits. They have already led a series of trainings in-country in preparation for a data campaign this season.

Looking Forward

As the first phase of the Helmets project comes to a close, the team is exploring new opportunities to sustain the initiative in the long-term. The Helmets approach is designed to be scalable and adaptable to a wide variety of contexts and EO research and to enable multi-season and multi-country data campaigns.

Currently, the Helmets team is looking for new partners who can benefit from and adapt these workstreams to support and scale our approach in new contexts. Through additional funding, we can expand the Helmets Kits program and provide more teams with the resources to launch data campaigns and continue to to build NASA Harvest’s CropHarvest dataset. 

From the beginning, the mission of Helmets has been to develop an innovative pipeline for generating crop-type maps in partnership with local organizations to build regional capacity for EO and ML research in support of food security. This first iteration of the Helmets project has demonstrated interest and demand from such partners and the capacity to scale the approach successfully.