• August 2010
    M T W T F S S

Data World Record Falls as Computer Scientists Break Terabyte Sort Barrier.

UCSD News (CA) (07/27/10) Kane, Daniel

University of California, San Diego (UCSD) researchers recently broke the terabyte barrier by sorting more than 1 terabyte of data in 60 seconds. The researchers also were able to sort 1 trillion data records in 172 minutes. “In data centers, sorting is often the most pressing bottleneck in many higher-level activities,” says UCSD professor Amin Vahdat. To break the terabyte barrier the researchers built a system consisting of 52 computer nodes. Each node is a commodity server with two quad-core processors, 24 GB of memory, and 16 500-GB disks. “If a major corporation wants to run a query across all of their page views or products sold, that can require a sort across a multi-petabyte dataset and one that is growing by many gigabytes every day,” Vahdat says. “Companies are pushing the limit on how much data they can sort, and how fast. This is data analytics in real time.”