XAO OpenIR  > 研究生
新疆天文台可视化数据传输系统设计与实现
Alternative TitleDesign and implementation of Xiniiang Astronomical Observatory Observation data transmission visualization
朱艳
Subtype硕士
Thesis Advisor张海龙
2019
Degree Grantor中国科学院大学
Place of Conferral北京
Degree Name理学硕士
Degree Discipline天文技术与方法
Keyword可视化,数据传输,数据库,日志 Visualization,Data transmission,Database,Log
Abstract随着多波束接收技术的发展与天文观测设备采样精度的提高,天文观测设备的数据收集能力也得到了大幅提升,天文数据已经进入了PB(Petabyte千万亿字节)量级时代。由于天文观测的特殊性,望远镜观测台站与天文数据存储中心往往相距遥远。新疆天文台望远镜产生的观测数据需通过数据专线传输到台本部数据中心进行长期存储。数据传输专线长度在百公里以上,由于线路长期暴露在野外,导致数据传输过程经常出现问题甚至传输中断。建立一套拥有可视化界面的数据传输系统实时显示数据传输情况,及时反馈相关信息给专业人员,会大大提高分析与解决问题的效率。本文在调研与分析了新疆天文台观测数据传输的现实需求后,设计并实现了新疆天文台可视化数据传输系统。 可视化数据传输系统主要由日志子系统、数据传输子系统、控制子系统及可视化子系统组成。日志子系统部分,设计了日志系统的整体架构,实现了数据传输过程的日志收集与存储,采用shell多线程技术加速日志信息入库速度,开发了日志查询工具供管理员使用。数据传输子系统部分,以rsync传输框架为基础,封装相关shel命令实现上层调用。控制子系统部分,设计了自动化运行的互相监控脚本体系,控制日志记录与数据传输请求,嵌入了告警系统。可视化子系统部分,搭建了展示数据传输过程中传输速率、网络状态、服务器硬盘存储情况等功能的可视化界面,采用柱状图、饼状图、折线图等多种形式进行数据展示。 本文采用rsymc镜像备份工具作为底层数据传输框架,选择开源关系型数据库MySQL存储日志信息,利用shell多线程技术编写了数据获取及整合程序。基于B/S架构,使用Qt框架开发了供管理员使用的日志查询工具,基于C/S结构,利用Ajax、JavaScript、Html5、PHP等技术和语言开发了具有多项查询和展示的可视化界面。论文最后对系统的功能模块进行了测试,总结了研究内容、分析了系统不足与下一步工作展望。
Other AbstractWith the increase of the receiving area of astronomical observation equipment and the development of multi-beam receiving technology, the data collection capability of astronomical observing equipment has also been greatly improved, and astronomical data has entered the era of PB(Petabyte). Due to the particularity of astronomical observations, telescope observing stations are often far from astronomical data centers. The observation data generated by the Xinjiang Astronomical Observatory (XAO) telescope is transmitted to the data center at XAO headquarters for long-term storage through the dedicated transfer line. The length of the data transmission line is more than 100 kilometers. Due to the long-term exposure of the line in the field, the data transmission process often has problems or even transmission interruption. Establishing a data transmission system with a visual interface to display data transmission status in real time and timely feedback relevant information to professionals will greatly improve the efficiency of analysis and problem solving. After researching and analyzing the actual needs of data transmission in Xinjiang Observatory, this paper designs and implements the Xinjiang Observatory's visual data transmission system. The system is mainly composed of a log subsystem,a data transmission subsystem,a control subsystem and a visualization subsystem. In terms of log subsystem, the overall architecture of the log system is reasonably designed, the log collection and storage of the data transmission process is realized, the shell multi-threading technology is used to accelerate the log information storage speed, and the log query tool is developed for the administrator to use. In terms of data transmission, the data transmission subsystem is based on the rsync transmission framework, and encapsulates shell commands to implement upper-layer calls. In the control subsystem part, the mutual monitoring script system of automatic operation is designed, the log record and data transmission request are controlled, and the alarm system is embedded. In terms of visualization subsystem,a visual interface for displaying functions such as transmission rate, network status, and storage of the server hard disk during data transmission is constructed, and various forms such as a histogram,a pie chart, and a line chart are used for data display. In this paper, the rsync baclkup tool is used as the basic framework of the underlying data transmission. Choose open source relational database MySQL to store log information, and use shell multi-threading technology to write data acquisition and integration procedures. Based on the B/S architecture, the Qt framework was used to develop a log query tool for administrators. Based on the C/S structure,a visual interface with multiple queries and presentations was developed using Ajax, JavaScript, Html and other technologies and languages. At the end of the thesis, the functional modules of the system were tested, the research contents were summarized, and the system deficiencies and future work prospects were analyzed.
Pages61
Language中文
Document Type学位论文
Identifierhttp://ir.xao.ac.cn/handle/45760611-7/2714
Collection研究生
Affiliation中国科学院新疆天文台
First Author AffilicationXinjiang Astronomical Observatory, Chinese Academy of Sciences
Recommended Citation
GB/T 7714
朱艳. 新疆天文台可视化数据传输系统设计与实现[D]. 北京. 中国科学院大学,2019.
Files in This Item:
File Name/Size DocType Version Access License
朱艳-硕士学位论文最终版.pdf(3773KB)学位论文 暂不开放CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[朱艳]'s Articles
Baidu academic
Similar articles in Baidu academic
[朱艳]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[朱艳]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.