可扩展的数据管理大数据的应用程序是一项艰巨的任务。它把对持久内存墙的问题,这使得数据访问的高性能计算(HPC)突出的性能瓶颈更大的压力,并改变HPC的利益HPDP(高性能数据处理)。 HPC是著名的大规模并行架构。一种自然的方式来实现HPDP是增加和利用存储并发的水平与HPC相称。我们认为,大量的内存并发存在于当前存储系统的每一层,但它并没有被充分利用。在这次演讲中,我们重新评估存储系统,引进了新型的C-AMAT模型并发数据访问的系统设计分析。 C-AMAT是一个范式转变,以支持持续的数据从一个数据中心的视图访问。对C-AMAT的力量在于它开辟了新的方向,以减少数据访问延迟。在一个理想的并行存储器系统,该系统将明确地表示,并利用并行数据访问。这种认识主要是从当前的存储系统缺失和目前的架构和算法设计缺失。我们会检讨在现代内存的系统中可用的并发性,提出了C-AMAT的概念,并讨论了考虑和优化的大数据应用的并行数据访问的可能性。我们也将提出我们的一些最新研究成果而量化和利用并行I / O以下为HPDP并行存储的概念。
孙贤和,美国伊利诺伊理工学院计算机科学系教授、系主任,2012年当选国际电子电气工程师学会(IEEE)院士。他曾在美国能源部艾姆斯国家实验室,在ICASE,美国航空航天局兰利研究中心,在路易斯安那州立大学巴吞鲁日工作,并且是ASEE海军研究实验室研究员。
原文:Scalable data management for big data applications is a challenging task. It puts even more pressure on the lasting memory-wall problem, which makes data access the prominent performance bottleneck for high performance computing (HPC), and has changed the interest of HPC to HPDP (High Performance Data Processing). HPC is known for its massively parallel architectures. A natural way to achieve HPDP is to increase and utilize memory concurrency to a level commensurate with that of HPC. We argue that substantial memory concurrency exists at each layer of current memory systems, but it has not been fully utilized. In this talk we reevaluate memory systems and introduce the novel C-AMAT model for system design analysis of concurrent data accesses. C-AMAT is a paradigm shift to support sustained data accessing from a data-centric view. The power of C-AMAT is that it has opened new directions to reduce data access delay. In an ideal parallel memory system, the system will explicitly express and utilize parallel data accesses. This awareness is largely missing from current memory systems and missing from current architecture and algorithm design. We will review the concurrency available in modern memory systems, present the concept of C-AMAT, and discuss the considerations and possibility of optimizing parallel data access for big data applications. We will also present some of our recent results which quantize and utilize parallel I/O following the parallel memory concept for HPDP.