Phase II Testing of UAV/Sensor Systems for Humanitarian Mine Action in Ukraine

This article is brought to you by the Center for International Stabilization and Recovery (CISR) from issue 28.3 of The Journal of Conventional Weapons Destruction available on the JMU Scholarly Commons and Issuu.com.

By Edward Crowther [ UNDP Ukraine ] and Natalia Shepel [ Ministry of Economy of Ukraine ]

The United Nations Development Programme (UNDP) conducted the Phase II trials of unmanned aerial vehicle (UAV)/sensor systems for humanitarian mine action (HMA) in Ukraine in May 2024. Seven organizations participated, four of which were Ukrainian. Each organization brought systems with differing combinations of unmanned aircraft system (UAS) platforms, sensors, and data processing, and were tested against a trial area set up in central Ukraine by UNDP.

Run at the request of the Ministry of Economy, Phase II trials involving Ukrainian and international organizations yielded promising results. They demonstrated that combining technologies to detect both metal and plastic explosive remnants of war (ERW) items—whether surface-laid or buried—could significantly enhance the efficiency of traditional humanitarian demining methods by enabling more targeted clearance.

Introduction

In May 2024, UNDP conducted Phase II trials of UAV/sensor systems for HMA in Ukraine. Officially titled the “Uncrewed Technologies for Effective Mine Action In Agriculture In Ukraine,” the project was more commonly referred to as “Phase II” as it built upon a previous trial held in July 2023.

The main aim of this second round of trials was to analyze the accuracy, efficiency, scalability, advantages, and limitations of such UAV/sensor systems in supporting and potentially accelerating future HMA land release efforts in Ukraine.

Phase II was generously funded by the governments of Croatia, France, Malta, the Netherlands, and Sweden.

Scope of Phase II Trials

The Ukrainian mine action landscape is characterized by a great deal of innovation across all sectors of mine action, and one key aim of Phase II was to test various UAS/sensor systems in an objective way, including a number of systems developed by Ukrainian organizations. Of the seven organizations that participated in Phase II, four were Ukrainian.

Setup of Phase II Trials

Phase II trials took place on a polygon in central Ukraine and was supported by the Ukrainian National Mine Action Authority, National Police of Ukraine, and the Ukrainian State Scientific Institute for the Testing and Certification of Military Equipment; the latter being the Ukrainian state body, which ultimately certifies all HMA equipment (such as demining machines). Both the Organization for Security and Co-Operation in Europe and the Geneva International Centre for Humanitarian Demining provided an HMA technical expert to support the Phase II trials.

Three areas were laid out for the trials:

• A control and calibration area, containing four ERW items—two surface-laid items and two sub-surface items—against which participating teams could calibrate their equipment.
• An area replicating a confirmed hazardous area (CHA) in Ukraine. The Phase II CHA was 1.44 hectares in size, within which were twenty-seven items of mines, cluster munitions, and unexploded ordnance (UXO), placed both surface and sub-surface, mimicking real-life mined areas in Ukraine as much as possible. Though small in geographical size, the CHA test area was varied with different vegetation, trench lines, and relatively flat areas.
• An area simulating a suspected hazardous area (SHA) with low contamination by ERW—only six ERW items were laid here. The SHA measured 5.75 hectares. The aim of this area was to determine how potentially accurate the systems were for technical survey in SHAs by measuring factors such as the false detection rate.

Items that were laid by the Phase II team were all real, free from explosive (FFE). Surrogates or mock-ups such as 3D printed models were not used. After emplacement, the location of each item was recorded with Differential Global Positioning System (D-GPS), as well as details about its location (buried, semi-buried, obscured by vegetation, in open air, etc.). In terms of items laid during Phase II, there was a higher proportion of metal objects compared to plastic objects, and more surface objects than buried ones.

Participating Teams

The Phase II trials aimed to test these various combinations of platform/sensor/data processing to see which combinations were the most successful against different categories of ERW targets: metal/plastic, surface/sub-surface, etc.

In terms of technologies used, all participants utilized electro-optical (EO) cameras. This technology can, if combined with either human and/or computer vision imagery analysis, provide a route to the detection of surface-laid ERW items.

For three of the participating organizations, a magnetometer was the second sensor of choice. For landmine detection, magnetometers are used to detect the magnetic anomalies caused by the presence of metal objects, such as UXO and landmines.

One organization used electromagnetic induction (EMI) sensors. EMI sensors detect the presence of metallic objects by measuring the response of a target to an induced electromagnetic field. When applied to landmine detection, EMI can identify both the metallic components of landmines and other sub-surface metallic objects.

Multispectral cameras including infrared (IR) were used by four participating organizations. This method leverages the unique thermal signatures of landmines to distinguish them from other objects and materials. Usage of IR sensors appeared to increase the detection rate for those companies which used them.

Though none of the sensor technologies deployed by any of the participating organizations during Phase II were novel, the organizations were each utilizing different forms of sensor fusion, as well as aspects of artificial intelligence (AI)/machine learning, in order to process data from the platforms and sensors.

In short, it seems that advancements in sensor fusion and data processing, rather than any specific hardware breakthroughs, are likely to enhance ERW detection rates in the Ukrainian context.

Phase II Conduct

Each team was allocated two days to scan both areas (the CHA and SHA) using their UAV/sensor systems. All participating organizations took part in the examination of the CHA, but owing to a variety of technical issues (faulty equipment), security considerations (UAS flight bans during the testing windows), and weather conditions (wind speeds exceeding normal operating conditions), four participating organizations were unable to assess the SHA test area.

The UAS flight bans were a result of the wartime situation in Ukraine, with the airspace being subject to sudden closure at any moment. One particular ban lasted an entire day during the trial period, and as such the teams affected by this ban during their testing window were rescheduled to complete their flying.

Elements of potential electromagnetic noise and GPS degradation were observed during the testing, most likely due to the wartime conditions in Ukraine at the time of testing. However, this did not appear to have a significant effect on the overall completion of testing.

Phase II Evaluation

Organization four and organization seven each identified more than 55 percent of the targets across both test sites (i.e., the CHA and SHA), with varying performance in different target categories. In the CHA, organization seven (utilizing EO cameras and a magnetometer) achieved better results for metal and buried objects (71 percent and 67 percent), while organization four (using solely EO cameras) excelled in detecting plastic (62 percent) and surface objects (78 percent) with distinctive rate for metal (57 percent).

Organization five, using EO cameras, magnetometers, and pulsating electromagnetic scanning (PEMS), demonstrated distinctive results for metal and buried objects with detection rates of 50 percent and 44 percent respectively.

Organization three performed well in detecting plastic objects, achieving a level comparable to organization seven (54 percent), despite using a different type of UAS (organization seven was the only one to use a fixed wing UAS). However, organization three showed lower results for buried and metal objects, presumably due to its reliance solely on visual sensors.

These high detection rates suggest that combinations of remote sensing technologies can reliably identify ERW if the item is comprised of metal and/or is surface-laid, and could therefore be useful in humanitarian demining operations.

The success in detecting 1) metal ERW items (both surface-laid and buried), and 2) surface-laid (plastic or metal) ERW items suggests that these technologies, used in conjunction, could significantly improve the efficiency of traditional demining methods by allowing more targeted clearance.

Detecting sub-surface plastic ERW items (e.g., the buried PMN-2 anti-personnel mines) remains a challenge. None of the demonstrated technologies were able to reliably detect such objects, especially in the highly contaminated environments likely found in real SHAs and CHAs in Ukraine.

Future Trial Design

Phase II also indicated important considerations for future potential rounds of objective testing of such systems in the Ukrainian context.

For example, the requirements of the testing polygon chosen meant that the ERW items could only be emplaced the day before testing started, meaning that even some buried items were probably readily observable owing to freshly disturbed earth and/or dead vegetation in the vicinity. For future testing, it would be preferable to place items in the ground at least a week, a month, or perhaps even longer, before testing, to give items time to settle.

The heavy contamination of the test range introduced additional noise into the data, which affected the calculations of the false alarm rate. Before commencing trials, it is recommended to conduct magnetometry or use other methods to identify metal elements and provide a detailed description of soil types and compositions. Additionally, a visual inspection of the range should be conducted to identify debris or scrap items lying on the surface, which may be picked up by EO sensors.

In general, the Ukrainian HMA sector would benefit from the establishment of a (semi-) permanent testing facility for such technologies. This would allow for a larger set of emplaced ERW items, as well as an established place where technologies could be tested on a regular basis.

Phase II Summary

The following general conclusions can be drawn after analysis of the performance of participants at Phase II:

Core remote sensing technologies. The most reliable remote sensing technologies identified during Phase II include high-resolution optical cameras, thermal cameras, and magnetometry. Future testing should focus on further improvements to sensor/data fusion, as well as adjustments for different weather conditions and types of ERW common to the Ukrainian context.

Geophysical remote sensing methods. Classic geophysical methods, such as PEMS, require further calibration and experimental diversification to detect resonance in various small-scale metal and plastic objects.

Detection efficiency by depth. All surface objects were detected by one or more of the teams, and 60 percent of buried objects were found. Future testing rounds should focus on data collection and calibration for magnetic anomalies and thermal radiation to improve detection of sub-surface ERW items. This should be with particular focus on locating buried, plastic/minimum-metal ERW items, the detection of which remains challenging for existing technologies. 

Visual inspection and AI. Visual inspections using AI for detection can yield excellent results with precisely trained neural networks. AI software can potentially significantly improve data processing speed and noise reduction, though human experts are still necessary for final data review. Computer vision models trained on larger datasets with real-life images show higher accuracy and lower false alarm rates.

For further information on Phase II trials, please email the authors at edward.crowther@undp.org

  Edward Crowther is a Humanitarian Mine Action Field Operations Advisor with the United Nations Development Programme (UNDP) Ukraine. He holds a Master of Science in Development Studies from the University of London’s School of Oriental and African Studies. 

Natalia Shepel worked on Phase II as an Information Management Analyst with UNDP Ukraine. Currently she works as a consultant within the Ministry of Economy of Ukraine’s humanitarian demining team. She holds an MSc in oceanography from the University of Southampton with a focus on applied artificial intelligence integration in robotics.