KMS of Xinjiang Astronomical Observatory, CAS
| 基于异构平台的高速射电天文信号实时传输算法研究 | |
| Alternative Title | Research on Real-time Transmission Algorithm of High Speed Radio Astronomical Signal Based on Heterogeneous Platform |
张萌
| |
| Subtype | 硕士 |
| Thesis Advisor | 张海龙 |
| 2020 | |
| Degree Grantor | 中国科学院大学 |
| Place of Conferral | 北京 |
| Degree Name | 理学硕士 |
| Degree Discipline | 天文技术与方法 |
| Keyword | 异构,数字终端,环形缓冲区,实时传输 |
| Abstract | 随着天文学研究不断深入和数字技术的迅速发展，天文观测对信号接收与处理系统设备性能的要求也在不断提升。超宽带、多波束接收技术使得天文观测过程产生的数据量成倍增长，海量天文数据流高速传输、预处理及存储已成为目前大型射电观测设备运行中急需解决的问题。射电天文数字终端系统作为射电望远镜的重要组成部分，承担信号采样及预处理和数据高速传输及存储功能，当前主流数字终端系统多采用FPGA+GPU+CPU混合架构模式，异构系统需要在不同设备间实现高效数据流转，以满足信号高速采样、实时处理与分析的科研需求。针对新疆天文台南山观测站26米射电望远镜（NSRT）实际运行过程中遇到的问题及未来大口径射电望远镜多功能数字终端系统数据实时高效传输与预处理系统设备的研制需求，结合现阶段数字终端系统的发展趋势，本文对异构平台上如何实现高效天文数据传输与预处理技术进行了分析研究，主要完成了以下几方面工作：（1）在充分调研国内外主流数字终端系统架构、可重构硬件设备和软件平台的基础上，设计了基于异构平台的高速天文信号实时高效传输技术方案，构建了高效内存缓冲区算法，分析研究了关键技术的实现原理，并通过调整操作系统网络核心配置参数实现了传输优化。（2）研究并搭建了基于ROACH2+GPU的混合架构实验平台，使用Matlab和Xilinx完成了模拟信号输入，信号采样、滤波等预处理算法设计，利用ROACH2实现了仿真分析，生成UDP数据包并通过高速以太网实现了ROACH2到GPU服务器端的数据包传输以及不同设备间的信息交互，为信号的高速传输与处理等相关研究提供了实验平台。（3）构建先进先出环形缓冲区作为数据传输算法的核心数据结构，借助高可用数据传输引擎提高了异构平台上数据传输效率，实现了实时网络数据接收、设备间内存共享和数据实时高效交换，创建并行线程实现了数据的高效传输算法以及数据传输关键节点的监控管理。（4）对高速UDP数据包实时传输过程中的丢包问题进行了分析，传统数据包捕获过程中多次拷贝限制报文接收能力，通过引入高速数据包捕获方法PF_RING进行优化，减少报文拷贝次数有效降低了丢包率，显著提高数据传输性能。论文主要创新点如下：（1）仿真UDP数据包作为原始传输数据，设计实现了FPGA到GPU服务器的数据高速传输算法并通过异构平台10GbE接口实现了数据传输，通过添加计数器对数据包传输个数实时统计。（2）针对网络传输存在的丢包问题，在设计中设置NUMA机制实现进程与CPU绑定以降低访问时延，优化操作系统网络核心配置，有效控制了数据包丢包率。（3）针对现有数据传输引擎使用传统网络传输方式不能满足万兆网卡数据高速率传输的问题，引入PF_RING接口实现数据包传输环境优化，减少网络传输过程中数据冗余拷贝，显著提升了传输效率。本文分析了异构数字终端系统的实现方式，调研并选用性价比良好的硬件设备构建了实验平台，深入研究了异构平台上数据传输与预处理技术，通过实验测试表明构建的软硬件平台及实现的相关设计实现了高速数据流仿真、实时预处理及异构平台上的高效传输。对比现有数据传输管道软件，本文设计的算法优化了数据传输环境，有效提高了数据传输效率，大大降低了丢包率。 |
| Other Abstract | With the continuous deepening of astronomical research and the rapid development of digital technology, the requirements of astronomical observation on the equipment performance for signals receiving and processing system are also improving. Benefited from ultrawide band and multi-beam receiving technology, the amount of data generated by astronomical observation exponentially increases. At present, high-speed transmission, preprocessing and storage of massive astronomical data streams have become the urgent problem to be solved during the operation of large aperture radio observation equipment. Radio astronomy digital backend system, as an important part of radio telescope, undertakes the functions of signal sampling and preprocessing as well as data transmission and storage at high-speed. The current mainstream digital backend system mostly adopts the hybrid architecture mode of FPGA+GPU+CPU. Heterogeneous systems need to realize efficient data transfer among different devices to meet the research needs of high-speed sampling, real-time processing and analysis. In view of the problems encountered during the actual observation of the NanShan 26m Radio Telescope (NSRT) in XinJiang Astronomical Observatory as well as the development requirements for the real-time high-efficiency transmission and preprocessing system of the large aperture radio telescope multi-functional digital backend system in the future, combined with the development trend of the digital backend system at the present stage, in this thesis, the analysis were conducted on how to achieve efficient astronomical data transmission and preprocessing technology on heterogeneous platforms. The main results are as follows:(1) Based on the investigation of mainstream digital backend architecture as well as the reconfigurable hardware devices and software platforms, the real-time and efficient transmission technology schemes for high-speed astronomical signals based on heterogeneous platforms were designed, built an efficient algorithm of memory buffer, and optimized the transmission by adjusting the core configuration parameters of the operating system.(2) A hybrid architecture experimental platform based on ROACH2+GPU was studied and built, analog signals input and preprocessing algorithms such as signal sampling, filtering were completed by using Matlab and Xilinx. In addition, employed ROACH2 to realize the simulation analysis and generate UDP packets. Besides, the transmission of data packets from ROACH2 to GPU server through high-speed Ethernet were achieved, and the interaction of information between different devices were realized, which provided an experimental platform for the high-speed transmission and processing of signals.(3) FIFO ring buffers were constructed as the core data structure of the data transmission algorithm. With the high-availability data transmission engine, the efficiency of data transmission on heterogeneous platforms was improved, thus realized network data reception in real-time, memory sharing, data real-time and efficient exchanged between devices. In addition, parallel threads were created to realize efficient data transmission algorithms and monitoring the process of data transmission.(4) The problem of packet loss during the real-time transmission of high-speed UDP packets was analyzed. In the traditional packet capture process, multiple copies restricted the reception of message. By introducing the high-speed packet capture method PF_RING, the research effectively reduced the times of message copies and the packet loss rate as well as significantly improved the data transmission performance.The main innovations of the thesis are as follows:(1) As the original data transmission, the simulation UDP data packet was designed to realize the high-speed data transmission algorithm from FPGA to GPU server and the data transmission was completed through the heterogeneous platform 10GbE interface. The number of data packets transmitted was counted in real time by adding counters.(2) Towards the packet loss problem in network transmission, NUMA mechanism was set in the design to realize the process and CPU binding to reduce the access delay, and the network core configuration of the operating system was optimized to effectively control the packet loss rate.(3) In view of the problem that the existing data transmission engine cannot satisfy the high data transmission rate of the 10GbE network by using the traditional network transmission method, PF_RING interface was introduced to optimize the data packet transmission environment, which reduces the redundant data copy in the network transmission process and significantly improves the transmission efficiency.This paper analyzed the implementation of heterogeneous digital backend system and constructed an experimental platform. The technology of data transmission and preprocessing on heterogeneous platform was deeply studied. The experimental test showed that the software and hardware platform constructed and the related design realized high-speed data flow simulation, real-time preprocessing and efficient transmission on heterogeneous platform. Compared with the existing data transmission pipeline software, the algorithm designed in this paper optimized the data transmission environment, effectively improved the data transmission efficiency, and greatly reduced the packet loss rate. |
| Pages | 57 |
| Language | 中文 |
| Document Type | 学位论文 |
| Identifier | http://ir.xao.ac.cn/handle/45760611-7/4141 |
| Collection | 研究生学位论文 |
| Affiliation | 中国科学院新疆天文台 |
| First Author Affilication | Xinjiang Astronomical Observatory, Chinese Academy of Sciences |
| Recommended Citation GB/T 7714 | 张萌. 基于异构平台的高速射电天文信号实时传输算法研究[D]. 北京. 中国科学院大学,2020. |
| Files in This Item: | ||||||
| File Name/Size | DocType | Version | Access | License | ||
| 张萌.pdf（3380KB） | 学位论文 | 暂不开放 | CC BY-NC-SA | Application Full Text | ||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment