|
Mathematical optimization and configuration analysis of multimodal end-to-end deep learning for joint steering and speed control
|
M. Aghamohammadi   , K. Khoshhal Roudposhti |
| Department of Computer Engineering, La.C., Islamic Azad University, Lahijan, Iran |
|
|
Abstract: (8 Views) |
| End-to-end autonomous driving can be interpreted as a high-dimensional nonlinear optimization problem in which heterogeneous sensory observations are mapped directly to continuous vehicle-control commands. This paper provides an optimization-oriented mathematical interpretation and configuration-sensitivity analysis of joint steering-angle and vehicle-speed prediction using real-world camera and LiDAR observations across two successive phases of model development. In Phase I, Xception and ResNeXt are evaluated under six RGB/LiDAR and temporal configurations to quantify the effects of depth representation and Long Short-Term Memory (LSTM). In Phase II, complementary backbones are combined through stacking, followed by LSTM and attention, allowing the incremental contributions of ensemble learning, temporal modeling, and attention to be analyzed. The reported experiments use a fixed-weight scalarized steering-speed loss optimized with Adam; adaptive weighting, gradient-conflict mitigation, and bilevel optimization are discussed as mathematical extensions and future research directions rather than experimentally implemented methods. A post-hoc Balanced Accuracy Index is introduced to summarize the two control outputs without altering the original training objective, and horizon-wise RMSE is used for contextual comparison with a related camera-LiDAR E2E baseline. Phase I shows that adding LSTM to RGB-only models increases steering accuracy by 9.2 percentage points for Xception and 10.5 percentage points for ResNeXt. Phase II improves performance substantially: the ResNeXt-Xception-Attention-LSTM configuration reaches 94.91% steering accuracy and 94.09% speed accuracy. Its RMSE values across 1-5 s horizons are 0.6116, 0.8626, 1.0866, 1.3005, and 1.5055, respectively, with reductions of up to 19.8% for M6 relative to the selected published baseline under non-identical experimental protocols. The results show that the attained solution is sensitive to architecture, input representation, temporal context, and fusion strategy. |
|
| Keywords: Multi-Objective Optimization, End-To-End Autonomous Driving, Multimodal Fusion, LSTM, Attention, Steering And Speed Control. |
|
|
Full-Text [PDF 883 kb]
(6 Downloads)
|
Type of Study: Research |
Subject:
General
Received: 2026/07/5 | Accepted: 2026/08/25 | Published: 2026/09/3
|
|
|
|
|
|
|
| Send email to the article author |
|
|
|
| Add your comments about this article |
|
|
|
