TS Success Stories
From CEDPS
Workflow performance database for SCEC
The Southern California Earthquake Center (SCEC) CyberShake team performs detailed computer simulations of ruptures in the earth's crust to generate more accurate seismic hazard maps. An example map generated by the project is shown in Figure 1.
There are two phases to the CyberShake computation: one is a parallel MPI job; the other is a more data-intensive but loosely coupled bag of tasks. The second phase is implemented as a huge workflow using the Planning for Execution in Grids (Pegasus) workflow software. Millions of jobs are run on the TeraGrid. To monitor and manage the results of their computation, CyberShake is using the CEDPS monitoring tools to normalize and ingest the results of each run several times faster than their previous toolset. The CEDPS team is helping the CyberShake and Pegasus researchers understand the performance of their workflows with sophisticated database queries and the R statistical software.
Troubleshooting BeStMan performance for STAR
Throughput of the BeStMan software's data transfers between Oakland, CA and Brookhaven, NY for the STAR project was visualized for the first time in a way that the performance of individual transfers was clear, as well as their contribution to the overall performance. The performance was far below optimal, or even the anticipated necessary performance in the near future. Investigation into the root cause of the problem revealed outmoded hardware at PDSF, which was replaced. This produced the improvement shown in the Graph 1 below.
The CEDPS team worked with NERSC to measure the network with GridFTP and iperf tests to determine the optimal settings for BeStMan servers. Some of these measurements are [PDSF - BNL bandwidth measurements|hosted on the CEDPS wiki].
In the meantime, GridFTP transfers between the same two sites, in the other direction, using GridFTP directly are experiencing far better performance. This is shown in the Graph 2. The reasons for this asymmetry are being investigated. CEDPS analysis tools will be used to analyze the difference in performance, if any, found as the transfers transition from GridFTP to BeStMan (using GridFTP as a back-end).
