结合注意力机制和LSTM的参数自适应无模型状态估计

作者

  • 陈 伟 北京工商大学,计算机与人工智能学院,北京 100048 作者
  • 金 学波 北京工商大学,计算机与人工智能学院,北京 100048 作者
  • 马 慧鋆 北京工商大学,计算机与人工智能学院,北京 100048 作者
  • 曹 欣然 厦门大学,航空航天学院,厦门 361102 作者

DOI:

https://doi.org/10.52810/JIR.2024.005

关键词:

轨迹跟踪, 状态估计, Kalman滤波, Transformer, 长短期记忆网络

摘要

机动目标跟踪广泛地应用于无人车的自动驾驶跟踪领域。在实际应用中,系统噪声协方差很难获得准确值。传统的Kalman滤波器在系统噪声的协方差未知情况下,跟踪性能会下降。为了解决由于实际目标运动复杂、测量传感器噪声特性很难准确建模的困难,本文提出了一种基于注意力参数学习模块的自适应KF算法的状态估计方法:将Transformer的编码器和长短时记忆网络(LSTM)相结合,本文设计了注意力学习模块。通过离线对测量数据进行学习,获得了系统的运动特性,无需进行系统动力学和测量特性建模。进而,基于注意力学习模块的输出,利用期望最大化(EM)算法在线估计系统模型参数,并使用Kalman滤波器获得状态估计。本文使用GPS轨迹路径数据集进行验证,实验结果证明了本文提出的无模型状态估计方法的估计精度优于其他模型,为利用深度学习网络进行轨迹跟踪提供了一种有效方法。

下载统计

##plugins.themes.default.displayStats.noStats##

作者简历

  • 作者
    陈伟 2024年毕业于北京工商大学控制工程专业,获硕士学位。研究方向为模式识别与信息融合、无人车、机器学习等。
  • 作者
    金学波,教授,博士生导师.1994年毕业于吉林大学(原吉林工业大学)获学士学位,1997年毕业于吉林大学(原吉林工业大学)获硕士学位,2004年获得浙江大学控制科学与工程博士学位,导师为孙优贤院士.研究方向为信息融合、模式识别与预测、大数据分析、深度学习等.近年来在相关领域主持了1项国家科技支撑计划课题、4项国家自然基金面上项目等多项研究课题.获2021年度中国粮油学会科学技术奖一等奖。在时序信号模式识别、图像目标检测与识别等研究领域,已发表SCI、EI收录等高水平学术论文159篇,其中7篇为ESI高被引论文(前1%)、3篇ESI热点论文(前0.1%),已授权国家发明专利20余项,出版关于传感器信号识别与状态估计、多传感器信息融合的学术专著3部.担任SCI收录期刊Sensors编委,为IEEE/CAA Journal of Automatica Sinica、Knowledge-Based Systems等中科院一区SCI期刊审稿人.
  • 作者
    马慧鋆,2010年毕业于长春光机学院原子与分子物理专业,获硕士学位,目前为北京工商大学系统科学专业在职博士生。研究方向为复杂系统建模、模式识别与信息融合、机器学习等。
  • 作者
    曹欣然 就读于厦门大学航空航天学院,研究方向为人工智能、电子信息等。

参考文献

Baatar, G., Eichhorn, M., & Ament, C. (2014). Precise indoor localization of multiple mobile robots with adaptive sensor fusion using odometry and vision data. IFAC Proceedings Volumes, 47(3), 7182-7189. https://doi.org/10.3182/20140824-6-ZA-1003.02345

Wang, J., Wang, S., & Leng, W. (2021). Vision Positioning‐Based Estimation Method and Its Simulation Studies on State of Underwater Manipulator. Mathematical Problems in Engineering, 2021(1), 6656928. https://doi.org/10.1155/2021/6656928

Zhang, B., Lyu, M., Zhang, L., & Wu, Y. (2021). Artificial Intelligence‐Based Joint Movement Estimation Method for Football Players in Sports Training. Mobile Information Systems, 2021(1), 9956482. https://doi.org/10.1155/2021/9956482

Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., & Schmid, C. (2020). Vectornet: Encoding hd maps and agent dynamics from vectorized representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11525-11533).

Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. https://doi.org/10.1115/1.3662552

Kalman, R. E., & Bucy, R. S. (1961). New results in linear filtering and prediction theory. https://doi.org/10.1115/1.3658902

Julier, S. J., Uhlmann, J. K., & Durrant-Whyte, H. F. (1995, June). A new approach for filtering nonlinear systems. In Proceedings of 1995 American Control Conference-ACC'95 (Vol. 3, pp. 1628-1632). IEEE. https://doi.org/10.1109/ACC.1995.529783

Arasaratnam, I., & Haykin, S. (2011). Cubature kalman smoothers. Automatica, 47(10), 2245-2250. https://doi.org/10.1016/j.automatica.2011.08.005

Gordon, N. J., Salmond, D. J., & Smith, A. F. (1993, April). Novel approach to nonlinear/non-Gaussian Bayesian state estimation. In IEE proceedings F (radar and signal processing) (Vol. 140, No. 2, pp. 107-113). IET Digital Library. https://doi.org/10.1049/ip-f-2.1993.0015

Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the royal statistical society: series B (methodological), 39(1), 1-22. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x

Shumway, R. H., & Stoffer, D. S. (1982). An approach to time series smoothing and forecasting using the EM algorithm. Journal of time series analysis, 3(4), 253-264. https://doi.org/10.1111/j.1467-9892.1982.tb00349.x

Ghahramani, Z., & Hinton, G. E. (1996). Parameter estimation for linear dynamical systems.

Connor, J. T., Martin, R. D., & Atlas, L. E. (1994). Recurrent neural networks and robust time series prediction. IEEE transactions on neural networks, 5(2), 240-254. https://doi.org/10.1109/72.279188

Choi, H. M., Kim, M. K., & Yang, H. (2021). Abnormally high water temperature prediction using LSTM deep learning model. Journal of Intelligent & Fuzzy Systems, 40(4), 8013-8020.

Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. https://doi.org/10.48550/arXiv.1409.0473

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

Tong, Y., Zheng, Z., Fan, W., & Liu, Z. (2022). Improved unscented Kalman filter for state of charge estimation of lithium-ion battery with one-step randomly measurement loss and inaccurate noise covariance matrices. Digital Signal Processing, 131, 103780. https://doi.org/10.1016/j.dsp.2022.103780

Greenberg, I., Yannay, N., & Mannor, S. (2021). Noise estimation is not optimal: How to use kalman filter the right way. arXiv preprint arXiv2104, 2372.

Sage, A. P., & Husa, G. W. (1969, November). Algorithms for sequential adaptive estimation of prior statistics. In 1969 IEEE Symposium on Adaptive Processes (8th) Decision and Control (pp. 61-61). IEEE. https://doi.org/10.1109/SAP.1969.269927

Tamura, K., Peterson, D., Peterson, N., Stecher, G., Nei, M., & Kumar, S. (2011). MEGA5: molecular evolutionary genetics analysis using maximum likelihood, evolutionary distance, and maximum parsimony methods. Molecular biology and evolution, 28(10), 2731-2739. https://doi.org/10.1093/molbev/msr121

Zia, A., Kirubarajan, T., Reilly, J. P., Yee, D., Punithakumar, K., & Shirani, S. (2008). An EM algorithm for nonlinear state estimation with model uncertainties. IEEE Transactions on Signal Processing, 56(3), 921-936. https://doi.org/10.1109/TSP.2007.907883

Menon, V., Yarahmadian, S., & Rezania, V. (2018). Novel EM based ML Kalman estimation framework for superresolution of stochastic three-states microtubule signal. BMC Systems Biology, 12, 117-128.

Wang, X., Song, B., Liang, Y., & Pan, Q. (2017). EM‐based adaptive divided difference filter for nonlinear system with multiplicative parameter. International Journal of Robust and Nonlinear Control, 27(13), 2167-2197. https://doi.org/10.1002/rnc.3674

Chen, C., Lu, X., Markham, A., & Trigoni, N. (2018, April). Ionet: Learning to cure the curse of drift in inertial odometry. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 32, No. 1). https://doi.org/10.1609/aaai.v32i1.12102

Kim, W. Y., Seo, H. I., & Seo, D. H. (2021). Nine-Axis IMU-based Extended inertial odometry neural network. Expert Systems with Applications, 178, 115075. https://doi.org/10.1016/j.eswa.2021.115075

Gao, R., Xiao, X., Zhu, S., Xing, W., Li, C., Liu, L., ... & Chai, H. (2021). Glow in the dark: Smartphone inertial odometry for vehicle tracking in GPS blocked environments. IEEE Internet of Things Journal, 8(16), 12955-12967. https://doi.org/10.1109/JIOT.2021.3064342

Akbari, H., Yuan, L., Qian, R., Chuang, W. H., Chang, S. F., Cui, Y., & Gong, B. (2021). Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text. Advances in Neural Information Processing Systems, 34, 24206-24221.

Liu, Y., He, G., Wang, Z., Li, W., & Huang, H. (2022). NRT-YOLO: Improved YOLOv5 based on nested residual transformer for tiny remote sensing object detection. Sensors, 22(13), 4953. https://doi.org/10.3390/s22134953

Yunus, A. P., Shirai, N. C., Morita, K., & Wakabayashi, T. (2022, August). Comparison of RNN-LSTM and kalman filter based time series human motion prediction. In Journal of Physics: Conference Series (Vol. 2319, No. 1, p. 012034). IOP Publishing.

Song, F., Li, Y., Cheng, W., Dong, L., Li, M., & Li, J. (2022). An Improved Kalman Filter Based on Long Short‐Memory Recurrent Neural Network for Nonlinear Radar Target Tracking. Wireless Communications and Mobile Computing, 2022(1), 8280428. https://doi.org/10.1155/2022/8280428

Liu, Y., Wang, X., Chen, L., Liu, S., Han, J., Shi, H., & Zhong, F. (2022). Driver’s visual attention characteristics and their emotional influencing mechanism under different cognitive tasks. International journal of environmental research and public health, 19(9), 5059. https://doi.org/10.3390/ijerph19095059

Zha, Y., Liu, X., Ma, F., & Liu, C. C. (2022). Vehicle state estimation based on extended Kalman filter and radial basis function neural networks. International Journal of Distributed Sensor Networks, 18(6), 15501329221102730. https://doi.org/10.1177/15501329221102730

Arjas, A., Alles, E. J., Maneas, E., Arridge, S., Desjardins, A., Sillanpää, M. J., & Hauptmann, A. (2022). Neural network kalman filtering for 3-d object tracking from linear array ultrasound data. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 69(5), 1691-1702. https://doi.org/10.1109/TUFFC.2022.3162097

Havangi, R. (2020). Intelligent adaptive unscented particle filter with application in target tracking. Signal, Image and Video Processing, 14(7), 1487-1495.

Lichota, P. (2023). Wavelet transform-based aircraft system identification. Journal of Guidance, Control, and Dynamics, 46(2), 350-361. https://doi.org/10.2514/1.G006654

Goswami, D., & Paley, D. A. (2021). Non-gaussian estimation and dynamic output feedback using the gaussian mixture kalman filter. Journal of Guidance, Control, and Dynamics, 44(1), 15-24. https://doi.org/10.2514/1.G005005

Goff, G. M., Showalter, D., Black, J. T., & Beck, J. A. (2015). Parameter requirements for noncooperative satellite maneuver reconstruction using adaptive filters. Journal of Guidance, Control, and Dynamics, 38(3), 361-374. https://doi.org/10.2514/1.G000941

Wang, L., & Zhou, G. (2021). Pseudo-spectrum based track-before-detect for weak maneuvering targets in range-Doppler plane. IEEE Transactions on Vehicular Technology, 70(4), 3043-3058. https://doi.org/10.1109/TVT.2021.3065665

Eltoukhy, M., Ahmad, M. O., & Swamy, M. N. S. (2020). An adaptive turn rate estimation for tracking a maneuvering target. IEEE Access, 8, 94176-94189. https://doi.org/10.1109/ACCESS.2020.2995672

Ye, X., Zhang, O., Li, X., & Qiu, Y. (2019, February). Tracking algorithm for cruise missile based on IMM-singer model. In AIP Conference Proceedings (Vol. 2073, No. 1). AIP Publishing. https://doi.org/10.1063/1.5090750

Zhenkai, X., Fanying, L., & Lei, Z. (2018). Study on Maneuvering Target On-axis Tracking Algorithm of Modified Current Statistical Model. In MATEC Web of Conferences (Vol. 160, p. 02008). EDP Sciences. https://doi.org/10.1051/matecconf/201816002008

Ansari, M. S., Bartoš, V., & Lee, B. (2022). GRU-based deep learning approach for network intrusion alert prediction. Future Generation Computer Systems, 128, 235-247. https://doi.org/10.1016/j.future.2021.09.040

Zha, W., Liu, Y., Wan, Y., Luo, R., Li, D., Yang, S., & Xu, Y. (2022). Forecasting monthly gas field production based on the CNN-LSTM model. Energy, 260, 124889. https://doi.org/10.1016/j.energy.2022.124889

Huang, H., Zeng, Z., Yao, D., Pei, X., & Zhang, Y. (2021). Spatial-temporal ConvLSTM for vehicle driving intention prediction. Tsinghua Science and Technology, 27(3), 599-609. https://doi.org/10.26599/TST.2020.9010061

Jin, X. B., Gong, W. T., Kong, J. L., Bai, Y. T., & Su, T. L. (2022). PFVAE: a planar flow-based variational auto-encoder prediction model for time series data. Mathematics, 10(4), 610. https://doi.org/10.3390/math10040610

Wang, N., Fang, F., & Feng, M. (2014, May). Multi-objective optimal analysis of comfort and energy management for intelligent buildings. In The 26th Chinese control and decision conference (2014 CCDC) (pp. 2783-2788). IEEE. https://doi.org/10.1109/CCDC.2014.6852646

Liu, J., Wang, Q., Song, Z., & Fang, F. (2021). Bottlenecks and countermeasures of high-penetration renewable energy development in China. Engineering, 7(11), 1611-1622. https://doi.org/10.1016/j.eng.2020.08.016

Zhang, J., Feng, J., Zhou, Y., Fang, F., & Yue, H. (2012). Linear active disturbance rejection control of waste heat recovery systems with organic Rankine cycles. Energies, 5(12), 5111-5125. https://doi.org/10.3390/en5125111

Wei, L., & Fang, F. (2016). ${H} _ {infty} $-LQR-Based Coordinated Control for Large Coal-Fired Boiler–Turbine Generation Units. IEEE Transactions on Industrial Electronics, 64(6), 5212-5221. https://doi.org/10.1109/TIE.2016.2622233

Wang, W., Liu, J., Zeng, D., Fang, F., & Niu, Y. (2020). Modeling and flexible load control of combined heat and power units. Applied Thermal Engineering, 166, 114624. https://doi.org/10.1016/j.applthermaleng.2019.114624

Liu, J., Song, D., Li, Q., Yang, J., Hu, Y., Fang, F., & Joo, Y. H. (2023). Life cycle cost modelling and economic analysis of wind power: A state of art review. Energy Conversion and Management, 277, 116628. https://doi.org/10.1016/j.enconman.2022.116628

Liu, J., Zeng, D., Tian, L., Gao, M., Wang, W., Niu, Y., & Fang, F. (2015). Control strategy for operating flexibility of coal-fired power plants in alternate electrical power systems. Proceedings of the CSEE, 35(21), 5385-5394.

Fang, F., & Xiong, Y. (2014). Event-driven-based water level control for nuclear steam generators. IEEE Transactions on Industrial electronics, 61(10), 5480-5489. https://doi.org/10.1109/TIE.2014.2301735

Fang, F. A. N. G., Tan, W., & Liu, J. Z. (2005). Tuning of coordinated controllers for boiler-turbine units. Acta Automatica Sinica, 31(2), 291-296.

Liu, Y., Fang, F., & Park, J. H. (2018). Decentralized dissipative filtering for delayed nonlinear interconnected systems based on T–S fuzzy model. IEEE Transactions on Fuzzy Systems, 27(4), 790-801. https://doi.org/10.1109/TFUZZ.2018.2870079

Lv, Y., Fang, F. A. N. G., Yang, T., & Romero, C. E. (2020). An early fault detection method for induced draft fans based on MSET with informative memory matrix selection. ISA transactions, 102, 325-334. https://doi.org/10.1016/j.isatra.2020.02.018

Fang, F., Zhu, Z., Jin, S., & Hu, S. (2020). Two-layer game theoretic microgrid capacity optimization considering uncertainty of renewable energy. IEEE Systems Journal, 15(3), 4260-4271. https://doi.org/10.1109/JSYST.2020.3008316

Fang, F., Jizhen, L., & Wen, T. (2004). Nonlinear internal model control for the boiler-turbine coordinate systems of power unit. PROCEEDINGS-CHINESE SOCIETY OF ELECTRICAL ENGINEERING, 24(4), 195-199.

Zhang, X., Fang, F., & Liu, J. (2019). Weather-classification-MARS-based photovoltaic power forecasting for energy imbalance market. IEEE Transactions on Industrial Electronics, 66(11), 8692-8702. https://doi.org/10.1109/TIE.2018.2889611

Lv, Y., Lv, X., Fang, F., Yang, T., & Romero, C. E. (2020). Adaptive selective catalytic reduction model development using typical operating data in coal-fired power plants. Energy, 192, 116589. https://doi.org/10.1016/j.energy.2019.116589

Fang, F., & Wu, X. (2020). A win–win mode: The complementary and coexistence of 5G networks and edge computing. IEEE Internet of Things Journal, 8(6), 3983-4003. https://doi.org/10.1109/JIOT.2020.3009821

结合注意力机制和LSTM的参数自适应无模型状态估计

##submission.downloads##

已出版

2024-06-15

期次

栏目

文章

引用本文

伟陈., 学波金., 慧鋆马., & 欣然曹. (2024). 结合注意力机制和LSTM的参数自适应无模型状态估计. 智能机器人, 1(1), 55-72. https://doi.org/10.52810/JIR.2024.005