查询结果:   陈晴,杨明,肖云,吴京生,滕舟.云数据存储技术在气象数据存储中的应用[J].计算机应用与软件,2018,35(8):124 - 127,158.
中文标题
云数据存储技术在气象数据存储中的应用
发表栏目
应用技术与研究
摘要点击数
677
英文标题
APPLICATION OF CLOUD DATA STORAGE TECHNOLOGY IN METEOROLOGICAL DATA STORAGE
作 者
陈晴 杨明 肖云 吴京生 滕舟 Chen Qing Yang Ming Xiao Yun Wu Jingsheng Teng Zhou
作者单位
浙江省气象信息网络中心 浙江 杭州 310017 武汉科技大学计算机科学与技术学院 湖北 武汉 430065    
英文单位
Zhejiang Provincial Meteorological Information and Network Center, Hangzhou 310017, Zhejiang,China College of Computer Science and Technology,Wuhan University of Science and Technology, Wuhan 430065, Hubei,China    
关键词
云存储 分布式关系型数据库 NoSQL数据库存储 数据分块和压缩 数据的时间度和使用度
Keywords
Cloud storage Distributed relational database NoSQL database Block and compress the data Time and use degree of the data
基金项目
国家自然科学基金项目(61702381);浙江省气象局重点项目(2016ZD15-1)
作者资料
陈晴,工程师,主研领域:计算机网络技术,高性能计算,气象数据加工处理。杨明,工程师。肖云,高工。吴京生,高工。滕舟,工程师。魏爽,工程师。胡永亮,高工。庞俊,讲师。 。
文章摘要
气象数据具有体量巨大、种类繁多、数据响应速度要求快等特点,传统的数据存储模式难以适应气象大数据的需求。基于云分布式存储技术设计一套数据存储方法,提高了气象数据存储传输能力。将气象数据分成结构化、半结构化和非结构化三种结构类型,针对不同结构类型的数据分别选择云存储模型,结构化数据采用分布式关系型数据库存储,半/非结构化数据采用NoSQL数据库存储。在对半/非结构化数据存储设计中,对数据进行主键表及属性表设计,对属性表的数据进行分块和压缩,提高大数据的存储读取性能。根据数据的时间度和使用度原则阐述了不同类型气象数据间的转换。实验表明,设计的基于云分布式存储技术的数据存储方法是可行的,在一定程度上提高了气象数据存储传输能力,为气象数据的云化建设提供一定的借鉴和参考。
Abstract
The meteorological data have the characteristics of huge volume, various types and fast response to the data. The traditional data storage mode is difficult to meet the demand of large meteorological data. A set of data storage methods was designed based on cloud distributed storage technology to improve the storage and transmission capacity of meteorological data.The meteorological data were divided into three structural types: structured, semi-structured and unstructured. Different structural types of data chose different cloud storage models. Structured data were stored in distributed relational database. Semi-structured data and unstructured data were stored in NoSQL database. In the design of Semi-structured data and unstructured data storage, we designed the primary key table and the attribute table, blocked and compressed data of the attribute table to improve the storage and reading performance of large data.Moreover,the conversion of different types of meteorological data was explained according to the time and used the degree of the data. Experiments show that the data storage method based on cloud distributed storage technology is feasible. To a certain extent, it improves the transmission capacity of meteorological data storage.The data storage method discussed in the paper provides a reference for the cloud construction of meteorological data.
下载PDF全文