ORNL operates the world's brightest neutron source, the Spallation Neutron Source (SNS). Funded by the US DOE Office of Basic Energy Science, this national user facility hosts hundreds of scientists from around the world, providing a platform to enable break-through research in materials science, sustainable energy, and basic science. Computing and Data Science staff members have been engaged to help manage and analyze the large data sets (ranging in size from 100's of gigabytes to over 1 terabyte) generated by the intense pulses of neutrons.
Members of ORNL's Computing and Computational Sciences Directorate (CCSD) and SNS data specialists from the Neutron Data Analysis and Visualization Division have collaborated to successfully complete the Accelerating Data Acquisition, Reduction, and Analysis (ADARA) Lab-Directed Research and Development project to improve the production and analysis of these data sets. The ADARA project has brought together experts in high-performance file systems, parallel processing, cluster configuration and management, data management, and Neutron Science. As a result of the ADARA project, a new data infrastructure was created that enhances users' ability to collect, reduce, and analyze data as it is taken; create data files immediately after acquisition, regardless of size; reduce a data set in seconds after acquisition; and provide the resources for any user to do post-acquisition reduction, analysis, visualization, and modeling without requiring users to be on-site at the SNS facility.
ADARA is currently running on the HYSPEC beam line providing near real-time access to result data sets (both raw event data and reduced data) so that instrument scientists and users now obtain live feedback from their experiments. Moving forward, ADARA will be deployed in production across the beam lines at SNS. Due to the success of the ADARA LDRD, the project has transitioned to operations within the SNS and ongoing development activities are funded as part of ongoing SNS operations. For more information on the ADARA project please contact Galen Shipman (firstname.lastname@example.org).
Architectural Overview of ADARA
Galen Shipman (PI), Mark Hagen (co-PI), Thomas Proffen (NDAV Division Director), Ryan Adamson, Blake Caldwell, Stuart Campbell, Lloyd Clonts, David Dillow, Mathieu Doucet, Dave Giles, Melissa Graves-Brook, Gayle Greene, Matthew Gyurgyik, Steven Hartman, Matthew Hermanson, Steve Hicks, Kay Kasemir, Jason Kincl, Scott Koch, James Kohl, Chris Layton, Vickie Lynch, Robert McGreevy, Ross Miller, Sarp Oral, Andre Parizzi, Daniel Pelfrey, Pete Peterson, Doug Reitz, Shelly Ren, Michael Reuter, Charles Roberts, Andrei Savici, Sergey Shpanskiy, Brian Smith, Dale Stansberry, Madhan Sundaram, John Quigley, Carol Tang, Kevin Thach, Tara Thompson, Sudharshan Vazhkudai, Pedro Vicente, Feiyi Wang, Karen White, Barry Winn, Russell Taylor, Marie Yao, Piotr Zolnierczuk