Multi Modal Deep Learning for Real-Time Human Activity Analysis in Smart Urban Environments

Abstract

Human Activity Recognition (HAR) plays a vital role in public safety and intelligent surveillance within smart urban environments. Rapid urbanization has increased the demand for accurate and real-time activity recognition systems. However, challenges such as occlusion, complex crowd dynamics, illumination variations, and cluttered backgrounds significantly degrade recognition performance. Manual surveillance systems are also inefficient due to delayed responses and susceptibility to human error. Traditional machine learning approaches that rely on handcrafted features lack robustness and scalability when applied to complex urban scenarios. To overcome these limitations, a Multimodal Deep Learning–based HAR (MDL-HAR) framework is proposed. The system utilizes RGB images of urban human activities, where deep feature representations are extracted using DenseNet121. These extracted features are then used to train and evaluate multiple classification algorithms, including K-Nearest Neighbours (KNN), Perceptron, and Nearest Centroid classifiers (NCC), to establish baseline performance. Furthermore, a proposed Neuro-Fuzzy Gradient Ensemble Classifier (NF-GEC) is employed to enhance recognition accuracy by effectively modelling uncertainty, non-linearity, and complex decision boundaries. The integration of deep feature extraction with ensemble and fuzzy learning techniques significantly improves the robustness and reliability of human activity recognition in smart urban surveillance systems.