Research Overview

Our research lies at the intersection of AI, connected/autonomous systems, and transportation engineering, centering on situation-aware agentic AI to build safer and highly resilient mobility. We develop advanced, customized machine intelligence by integrating real-time data from vehicles and infrastructure with generative AI (LLMs, LVMs, and diffusion models) and distributed learning architectures.

This integration enables scalable and reliable AI deployment across diverse transportation applications, allowing us to simulate rare or complex traffic conditions and proactively manage dynamic networks. It enhances safety through robust training and decision-making, and fosters resilience through fault-tolerant designs capable of adapting to disruptions like adverse weather or cyberattacks.

Cyber-Physical Cooperation

3 projects

View projects →

Integrated Sensing Technologies

2 projects

View projects →

Distributed Machine Intelligence for Smart and Connected Systems

3 projects

View projects →

Cyber-Physical Cooperation

As deep learning technologies increasingly integrate into transportation systems, concerns are escalating regarding system resilience and adaptability in the face of a range of uncertainties, including variable environmental factors, fluctuating traffic conditions, and irregular distributions of road users. The development of an adaptive and autonomous traffic system through cyber-physical cooperation promises a transformative approach to mitigating such uncertainties, ensuring more resilient, safer, and equitable transportation networks.

A grid of images and plots showing the Edge-MuSE sensing system's analysis of four road conditions: dry, wet, snow-covered, and icy. The grid compares original images with brightness, dark channel, and gray value representations, each paired with a line graph showing the average value of the column across the pixel x-index.

Comprehensive Traffic Environment Perception with Edge Intelligence

Motivation: The physical traffic environment plays a crucial role in influencing the traffic system. Unfortunately, neglecting these essential environmental aspects can lead to severe repercussions, especially under adverse weather conditions, including a decline in accuracy and the emergence of anomalies. 

Research Summary: To mitigate the uncertainties arising from the traffic environment, we proposed an Edge-based Multi-task Safety-oriented Environmental (Edge-MuSE) sensing system. Edge-MuSE is designed to provide an extensive perception of the traffic environment, encompassing temperature, humidity, image dehazing, visibility measurement, and road surface condition classification. This innovative system is tailored for comprehensive, efficient, and cost-effective real-time monitoring, enhancing the reliability and safety of traffic management systems.

Surveillance camera view of a large city intersection with pedestrians crossing multiple wide zebra-striped crosswalks. Pedestrians are highlighted using SARLES

Multi-Scale Representation Learning Empowered Crowd Sensing

Motivation: As urban populations rise, the surge in road users within confined urban spaces leads to densely crowded scenarios, posing significant safety and mobility challenges for transportation systems. Sensing methods that rely on individual feature processing and analysis can be heavily compromised by occlusions, intricate backgrounds, and diverse crowd density, often resulting in a marked decline in perception accuracy.

Research Summary: To address these challenges, we proposed an ensemble sensing scheme, the Scale-Aware Representation Learning Empowered Sensing (SARLES) System, which leverages an innovative encoder-decoder module for the extraction and processing of both global and local texture features. Uniquely, SARLES is designed to adjust to diverse transportation settings by using a multi-scale representation learning approach to generate crowd density maps for accurate crowd counting and individual pinpointing. Pioneer tests on surveillance systems in Tokyo and Shanghai highlighted its exceptional crowd perception capabilities, achieving an accuracy rate of 93.21%. This research paved the way for transportation safety and mobility applications when the system cannot capture sufficient features for accurate sensing and perception.

A technical diagram illustrating the DEbias Traffic Object Recognition (DETOR) scheme. The top section shows current methods where an imbalanced traffic object distribution leads to a 'Bias Amplifier'. The bottom section shows the DETOR scheme, which uses both 'ALL Classes Input' and 'Minority Tail Classes Input' through a 'Debias Learning & Optimization' process to achieve 'Bias Reduction'.

Bias Mitigation in Uneven Distributions by Integrating Physical Information

Motivation: Given the inherently uneven and long-tailed distribution of road users in transportation scenarios, deep learning technologies can unintentionally amplify biases, resulting in diminished accuracy for less-represented classes like children and wheelchair users. Accurately quantifying and mitigating the bias of deep-learning-empowered perception systems to address this equity issue is therefore a significant research focus for daily traffic services and applications.

Research Summary: In response, we developed a trustworthy perception system based on few-shot and representation learning algorithms, DEbias Traffic Object Recognition (DETOR), for road user and traffic facility detection and classification. By employing DETOR, the residual neural network achieved a 290% boost in accuracy for minority transportation object detection. The research developed a pioneering trustworthy, transportation-adaptive, de-biased traffic perception system, paving the way for more promising future transportation services and applications.


Integrated Sensing Technologies

Many current traffic sensing systems face significant challenges due to the limitations of isolated sensing mechanisms, leading not only to sensor saturation and data overload, but also to fragmented awareness of traffic conditions. Cooperative sensing technologies offer a transformative solution by integrating data from multiple sources, both within a sensor unit (intra-sensor) and among various entities (inter-sensor), to enable a comprehensive understanding of traffic scenarios that enhances decision-making and improves hazard awareness.

System architecture diagram of the COCO-SENSOR system, showing a flow from the Mobile Unit for Sensing Center (MUSC) sensors up through key inputs and sensing technologies to transportation applications for agencies and road users.

Cooperative Multi-Task Surveillance and Connected IoT System

Research Summary: Addressing data redundancy and boosting efficiency for comprehensive traffic scene perception, the research proposes the concept of “Sensing as a Service (SaaS),” a demand-driven sensing scheme, and implements it in the Cooperative and Comprehensive Smart Edge Node for Sensing and OpeRation (COCO-SENSOR) system. COCO-SENSOR is a cutting-edge multi-task sensing system designed to fulfill critical real-world needs in a single unit: real-time traffic and environmental monitoring, precise recognition and classification of road users, and seamless live communication among infrastructure, transportation agencies, and road users. By incorporating advanced sensing technologies and a cooperative mechanism alongside parallel computing, COCO-SENSOR optimally allocates edge device computational resources. Implemented across three testbeds, complete with a well-developed sensor unit, mobile app, and informative dashboard, COCO-SENSOR has demonstrated a remarkable ability to bridge the divide between sensory data and traffic service needs, achieving over 95% accuracy and a consistent 30 FPS efficiency.

System architecture diagram of the Edge-empowered Cooperative Multi-camera Sensing (ECoMS) system, showing the workflow from real-time video input at multiple cameras through edge processing to an edge server for vehicle Re-ID and traffic status summary.

Privacy-Preserving Cooperation Systems for Cross-Camera Network Monitoring

Research Summary: To track objects across multiple cameras and help public agencies collect link travel time and speed information without any sensitive privacy information, an Edge-empowered Cooperative Multi-camera Sensing (ECoMS) System is proposed. ECoMS presents a novel algorithmic and edge-server cooperative framework that pushes edge computing and multi-camera re-identification workflows to serve traffic sensing on a cooperative IoT architecture. On the algorithm side, ECoMS proposes a lightweight edge-based computer vision framework for real-time vehicle detection, tracking, and representation selection. By sending only the objects’ representations to the edge server, high-bandwidth data transmission and heavy post-processing can be eliminated. Further, a hierarchical clip-based deep vehicle re-identification framework is integrated into ECoMS and significantly outperforms other state-of-the-art methods with 92.43% Rank-1 accuracy. Finally, a cooperative cross-camera traffic information estimation framework based on kernel density estimation with a kernel smoother is implemented, producing precise and reliable cross-region traffic parameters (link travel time, speed, and area OD) and distributions (less than 1.01 KL distance).


Distributed Machine Intelligence for Smart and Connected Systems

Distributed machine intelligence stands as a critical innovation for transportation systems, offering decentralized, privacy-preserving, and robust processing capabilities that enhance safety and smart management across infrastructure.

A system architecture diagram for the PEARL project by Chenxi Liu, showing the data flow from traffic metadata through feature extraction and LSTM mapping to predict traffic flow, which then informs a control plan to optimize throughput and travel cost.

Collaborative Traffic Perception and Control System

Research Summary: The fusion of traffic perception and control systems facilitates a dynamic and proactive approach to traffic management, markedly diminishing congestion and enhancing traffic flow. Our research introduces the Predictive Empowered Assignment scheme for Reversible Lane (PEARL), which leverages real-time traffic sensing to enable dynamic reversible lane control informed by short-term traffic flow forecasts. This strategy aims to optimize throughput and minimize delays. The real-time predictive control scheme employs an iterative approach, allowing the system to anticipate and plan for future traffic scenarios and thereby optimize current lane assignments. Field experiments demonstrate that PEARL boosts total throughput by 9.93%, showcasing its potential for practical implementation in contemporary traffic systems.

Animated daytime roadway surveillance approaching an intersection, with a traffic signal indicator and a blue box highlighting the rightmost lane.

Synchronized Infrastructure System with Cooperative Engagement for Dilemma Zone Mitigation

Research Summary: The dilemma zone is a significant safety concern in transportation, where existing infrastructure systems often lack the real-time processing and communication capabilities to mitigate the heightened risk of crashes caused by driver hesitation at signal changes. This research introduces an edge-server-based platform that unifies surveillance cameras, traffic signal controllers, and road users through a cooperative framework of perception, communication, and control to mitigate dilemma zone risks. The platform is structured into three core modules: a sensing module for instantaneous vehicle detection and tracking; a data processing module for trajectory data analysis and forecasting; and a communication module that facilitates immediate interaction with infrastructure and motorists. Deployed at two pioneer intersections in downtown Bellevue for one week, the system detected 1,076 dilemma events with 92.84% accuracy. It is the first connected infrastructure system to integrate sensing, data processing, and information dissemination for self-operating signal services.

Architecture diagram of the 3D Structural Information Sensing System (3D-SISS) showing the pipeline from image inputs, through a Mask-RCNN preprocessing module and an Encoder-Decoder feature extraction module, to a Car Keypoints Detection (CKPD) model utilizing a Graph Neural Network for 3D traffic scene sensing.

Advanced 3D Spatial Mapping for In-Depth Traffic Scene Analysis

Research Summary: Advanced 3D spatial mapping offers a sophisticated, high-dimensional view that is essential for detailed traffic scene analysis, significantly improving accuracy in sensing spatial relationships among transportation entities. Our research introduces the 3D Structural Information Sensing System (3D-SISS), anchored by an innovative, fully automatic camera calibration technique that enables precise real-time vehicle localization and scene reconstruction from a single monocular surveillance camera. 3D-SISS consists of three integral components: a pre-processing module for detecting, classifying, and segmenting target vehicles; a cutting-edge Car Keypoints Detection (CKPD) model; and an automatic camera calibration method that accurately maps 2D keypoints into 3D space. Field tests in the City of Bellevue demonstrate that 3D-SISS surpasses contemporary methods in vehicle localization, speed measurement, and scene comprehension, notably in dense urban settings with occlusive environments.