Confining the data explosion

More simulation also requires more space on the hard disk – no prob-lem for the new compression software. The volume of data for crash and weather simulations is shrunk to as little as a tenth of its original size.

“Let’s talk about the weather,” says Clemens-August Thole of the Fraunhofer Institute for Algorithms and Scientific Computing SCAI in St. Augustin. “As of 2008 Germany’s national meteorological service, the Deutscher Wetterdienst, will each day produce about 20 forecasts of the weather for the next few days. That will generate about 2.5 petabytes of data a year.”

One petabyte is equivalent to something like 2,000 hard disks, each with 500 gigabytes of memory. To diminish this volume, Thole and his team have developed a software program that reduces meteorological data in GRIB format by a factor of 2.5.

“Germany is covered with a network of grid points for these simulations,“ explains Prof. Rudolph Lorentz. “The results of simulations at neighboring points are not independent of one another. All we need is a good prediction of the values at any grid point in order to reduce the data volume significantly“.

Simulations have become indispensable in the automobile industry, too. However, even a virtual crash takes up several gigabytes of memory. “On the computer, any one of these models crashes into the wall 100 to 150 times a day,” says Rodrigo Iza-Teran. That quickly adds up to 100 terabytes or more of data. Prior to compression, the development engineers specify the required accuracy of the stored data, for example one millimeter in the case of vehicle crashes. In making this quantization, they are deliberately accepting a loss of data. All other stages of compression are loss-free, which means the data can be reconstructed 1:1.

The scientists at SCAI can offer compression factors of 10, 7 or 5, depending on the needs of the users. “Our approach is rather like video coding,” says Thole. “One of the tricks is to find out which components move without deforming. Another is to predict the changing content as accurately as possible.”

Nearly all German car manufacturers use the software, and with good reason: They can save valuable memory space, and the exchange of data runs smoothly. What is more, compressed data can be visualized faster and more easily. The decompression software has meanwhile been integrated in many post-processors, enabling files to be opened without the intermediate step of unzipping them.

Rodrigo Iza-Teran, Prof. Dr. Rudolph Lorentz and Clemens-August Thole have been awarded the Joseph-von-Fraunhofer Prize 2007 for their work on the compression of simulation results. The jury was impressed not only by the outstanding mathematical methods employed, but also by the great success that the compression method has scored with users.

Source: Fraunhofer-Gesellschaft

Related stories:

New Sony Cyber-Shot T500 Camera's Video Capture Goes High Def
Sony is adding high-definition movie recording to its digital camera line with the introduction of the Cyber-shot DSC-T500 model.
Sign language over a mobile phone
A group at the University of Washington has developed software that for the first time enables deaf and hard-of-hearing Americans to use sign language over a mobile phone. UW engineers got the phones working together this spring, and recently received a National Science Foundation grant for a 20-person field project that will begin next year in Seattle.
Samsung Launches New Compact Camcorder with YouTube Connectivity
Samsung Electronics unveiled today the SC-MX20, the successor to the company’s popular SC-MX10 compact digital camcorder. The ideal camcorder for the exploding generation of consumers uploading video content to the Internet, the SC-MX20 features a customized recording mode optimized for YouTube or multi-media devices, further simplifying the way users integrate the camcorder into their digital lifestyles.
Turning conventional video coding wisdom on its head
A major drawback of the latest generation video products and applications has been the complex requirements for coding and decoding signals. An alternative put forward by European researchers turns the traditional video coding paradigm on its head.
High-definition television to go
New video compression technology and transmission/reception equipment means Europe is ready for commercially deploying multi-channel HDTV over terrestrial, satellite, cable or IPTV digital links.
The digital home: An all-in-one device
Cable clutter and interface problems will soon be a thing of the past. In the living room of the future, all devices are connected wirelessly and can be easily operated via the television set. Fraunhofer researchers and their partners will be presenting the first solutions at CeBIT in Hanover (Germany) on March 4 through 9.
Research Leads to Self-Improving Chips with Speed 'Warping'
Imagine owning an automobile that can change its engine to suit your driving needs – when you’re tooling about town, it works like a super-fast sports car; when you’re hauling a heavy load, it operates like a strong, durable truck engine. While this turn-on-a-dime flexibility is impossible for cars to achieve, it is now possible for today’s computer chips.
Sony Unveils 4x Blu-Ray Disc Writer Drive
Sony Electronics today announced its second generation internal Blu-ray Disc (BD) writer drive for the computer aftermarket, which boasts 4X BD-R and 16X DVD+R recording speeds.

News discussion:

Technology news