Sumitomo Heavy Industries (SHI) and NEC Corporation (NEC) have outlined a joint initiative to build a near-miss detection system designed for construction environments, with development scheduled to commence in April 2026. The SHI and NEC effort is centred on improving site safety through the analysis of camera footage and sensor-derived operational data gathered from hydraulic excavators.
At the core of the near-miss detection system is an extraction AI model capable of processing video inputs alongside machine operation data sourced from SHI’s hydraulic excavators. This model has been trained using datasets accumulated through the SHI Group’s ICT/IoT platform, SHICuTe, enabling it to isolate scenes that indicate potential risks within recorded site activity. NEC’s proprietary technologies are then applied to interpret these scenes, combining video recognition with generative AI to convert them into multimodal datasets that incorporate both temporal and spatial elements.
The system further integrates SHI’s operational expertise, using insights into machinery behaviour and human workflows to validate identified scenarios against known hazardous and prohibited actions. These benchmarks are derived from historical accident data, equipment-related failures, and defined operational constraints. Company-specific parameters are also incorporated, allowing the system to adapt to different site requirements. Based on this combined input, high-risk situations are automatically flagged, and near-miss reports are generated with concise summaries of each identified incident.
A technical proof of concept carried out in September 2025 confirmed that the system could successfully extract and report near-miss cases using footage captured directly from cameras mounted on hydraulic excavators. The results demonstrated the ability to identify potential accident scenarios along with their contextual conditions. Moving forward, SHI and NEC plan to continue development and validation efforts throughout fiscal year 2026, leveraging on-site data and safety management expertise alongside NEC’s AI capabilities, with practical implementation targeted for fiscal year 2027.
Future plans include extending the system’s scope beyond worker–machinery interactions to detect less visible unsafe conditions and incorporate site-specific operational rules. “SHI and NEC will continue to combine their respective strengths to develop and validate new technologies, contributing to the realisation of safer construction sites,” SHI stated.
