CEDPS Milestones
From CEDPS
CEDPS Milestones
Note to CEDPS project partners: Detailed tracking of milestones is on the Globus JIRA under the "CEDPS" project.
Schedule
"Team" format: Lead/Participants, e.g. A/IL. No team means everybody. Institution codes: A=ANL, F=FNAL, I=ISI, L=LBNL, W=UWisc.
Color codes indicate activity type: challenge problems, technology R&D, outreach activities, and application engagements.
| Team | Milestone | Date | Notes | Status | Details | |
|---|---|---|---|---|---|---|
| A | 2nd Cloud Computing and Applications (CCA) workshop | Oct 2009 | http://www.cca09.org | Completed | ||
| A/I | Globus.org: GSI, GridFTP, CLI, limited scalability | Nov 2009 | First availability to selected beta users | Completed | ||
| AIL | Globus.org: Prototype REST interface & monitoring console | Nov 2009 | For SC demo | Completed | ||
| A/I | Globus.org demonstration at SC conference | Nov 2009 | A first demonstration, with some simple clients | Completed | (Slides) | |
| L/A | Best-practices logging design for Globus.org | Dec 2009 | A precursor to integration | |||
| F/AW | Condor & Glide-in WMS integration w/ Globus.org design | Dec 2009 | A precursor to integration | Completed | (DocumentPage) | |
| A/IL | Challenge #1 (2 sites, 10Kx100MB=1TB, 3 days) | Dec 2009 | Validate reliability, limited scalability | Completed | (Report) | |
| A | GFDL data movement demonstration | Jan 2010 | Data from OLCF and/or ALCF to GFDL, modest rates | In progress | (Notes) | |
| A/IL | Globus.org: v1 of REST interface & simple monitoring console | Feb 2010 | Production version 1 | |||
| AIL | Challenge #2 (2 sites, 100Kx100MB=10TB, 3 days) | Feb 2010 | Increased transfer throughput (0.4/sec) | Completed | (Report) | |
| A/IL | GlobusWorld conference | Mar 2010 | Demonstrations and tutorials | Completed | (Program online) | |
| W | Condor interface to Globus.org prototype | Mar 2010 | Sandbox and output data movement | |||
| F/AW | Glide-in WMS with Globus.org prototype | Apr 2010 | Demonstrate ability to manage direct stage out | |||
| A | Submit tutorial proposal for SC10 conference | Apr 2010 | Due date April 5, 2010 | http://sc10.supercomputing.org/?pg=tutorials.html | ||
| A/IL | Globus.org: Mirroring integrated | Apr 2010 | First integration of ISI mirroring work | |||
| AIL | Challenge #3 (10 sites, 10 users, 100Kx200MB=20TB, 3 days) | Apr 2010 | Multi-sites, multi-users involved in transfers | |||
| I/A | SNS/APS mirror demonstration | Apr 2010 | Mirror SNS data at APS for collaboration and redundancy | |||
| W/A | Data caching service prototype | May 2010 | Space allocation, data movement, lifetime management | |||
| A/IL | Challenge #4 (2 sites, 10Kx2GB=20TB, 3 days) | Jun 2010 | Increase bandwidth to 300 megabit/sec | |||
| LW | OSG user activity: prototype | Jun 2010 | Monitoring to detect anomalies in OSG user activity | |||
| A | High Performance Distributed Computing conference | Jun 2010 | http://hpdc2010.eecs.northwestern.edu/ | |||
| A/all | Workshop on distributed data management for DOE | Jun 2010 | Outreach and leadership for DOE data problems | |||
| A/ILW | Globus.org: Megellan HDFS integration | Jul 2010 | Preparing for Glide-in WMS integration and Magellan work | |||
| All | Globus.org demonstration at SciDAC 2010 conference | Jul 2010 | We plan to do something impressive | |||
| A/L | ESG data movement to Magellan | Aug 2010 | Magellan, data mirroring, parallel file system integration | |||
| W/AL | Challenge #5 (10 sites to 1, 10Kx1GB=10TB, 10 hours, Condor) | Aug 2010 | Involves movement of Condor sandboxes | |||
| A/ILW | Globus.org: Data caching | Aug 2010 | ||||
| A/I | ESG data analysis on Magellan | Sep 2010 | Integration with Hadoop and/or Swift | |||
| A/IL | GFDL data movement operation | Sep 2010 | Data from OLCF and/or ALCF to GFDL, 10 TB/day | |||
| F/AW | Glide-in WMS + Globus.org with caching | Sep 2010 | Demonstrate ability to manage data staging via cache | |||
| A/IL | Globus.org: OpenID authentication, cache management | Oct 2010 | Preparation for Challenge #6 | |||
| F/AW | Glide-in WMS + Globus.org for 10K jobs | Oct 2010 | Scalability test | |||
| W/AL | Challenge #6 (2 sites + cache: 10Kx1GB=10TB, 10 hours + 3 days) | Oct 2010 | Involves use of cache service | |||
| A | 3rd Cloud Computing and Applications (CCA) workshop | Nov 2010 | ||||
| F/AW | Glide-in WMS+Globus.org applied to selected HEP user cases | Nov 2010 | First engagement with high energy physics community | |||
| A/All | Demonstrate at SC'10 conference | Nov 2010 | Potential bandwidth challenge participation | |||
| W/AI | Challenge #7 (2 sites + cache, 10Kx1GB=10TB, 4+10 hours) | Dec 2010 | Involves use of cache service | |||
| A/IL | Globus.org: Multi-site mirroring | Jan 2011 | Can engage heuristics for minimizing data movement | |||
| A/IL | Globus.org: End-to-end checksum | Feb 2011 | Reliability | |||
| AIL | Challenge #8 (1 site to 2 site mirror, 1Kx10GB=10TB, 10 hours) | Mar 2011 | Scale to more sites if possible | |||
| F/AW | Glide-in WMS + Globus.org for 100K jobs | Mar 2011 | Scalability test | |||
| AIL | Globus.org: Further scalability improvements | Apr 2011 | ||||
| A/I | ESG data analysis on Magellan | Sep 2010 | Large-scale analysis: 50+ terabytes | |||
| AIL | Challenge #9 (10 sites, 100 users 10Kx10GB=100TB, 10 hours) | Apr 2011 | Multi-user, 20 gigabit/sec aggregate | |||
| F/AW | Glide-in WMS + Globus.org production use for HEP | Jun 2011 | Production use | |||
| AIL | Challenge #10 (2 sites, 10Kx100GB=1PB, 3 days) | Jun 2011 | Uses Magellan, ANI; requires 31 gigabit/sec sustained | |||
| LW | OSG user activity, production | Jun 2011 | Monitoring to detect anomalies in OSG user activity | |||
| All | Globus.org demonstration at SciDAC 2011 conference | Jul 2011 | We will have a lot to show and talk about | |||
| A/IL | GFDL data movement operation | Jul 2010 | Data from OLCF and/or ALCF to GFDL, 40 TB/day | |||
| AIL | Globus.org: Further scalability improvements | Jul 2011 | Addressing in particular number of concurrent users |
