CEDPS Milestones

From CEDPS

Jump to: navigation, search

CEDPS Milestones

Note to CEDPS project partners: Detailed tracking of milestones is on the Globus JIRA under the "CEDPS" project.

Schedule

"Team" format: Lead/Participants, e.g. A/IL. No team means everybody. Institution codes: A=ANL, F=FNAL, I=ISI, L=LBNL, W=UWisc.

Color codes indicate activity type: challenge problems, technology R&D, outreach activities, and application engagements. We also add notes about additional tasks completed but not listed in the original milestones.

Team Milestone Date........ Notes Status....... Details
A 2nd Cloud Computing and Applications (CCA) workshop Oct 2009 http://www.cca09.org Completed
A/I Globus.org: GSI, GridFTP, CLI, limited scalability Nov 2009 First availability to selected beta users Completed
AIL Globus.org: Prototype REST interface & monitoring console Nov 2009 For SC demo Completed
A/I Globus.org demonstration at SC conference Nov 2009 A first demonstration, with some simple clients Completed (Slides)
L/A Best-practices logging design for Globus.org Dec 2009 A precursor to integration Completed Design as well as rewritten Best Practices Document with implementation notes.
F/AW Condor & Glide-in WMS integration w/ Globus.org design Dec 2009 A precursor to integration Completed (DocumentPage)
A/IL Challenge #1 (2 sites, 10Kx100MB=1TB, 3 days) Dec 2009 Validate reliability, limited scalability Completed (Report)
A GFDL data movement demonstration Jan 2010 Data from OLCF and/or ALCF to GFDL, modest rates Completed (Notes)
A/IL Globus.org: v1 of REST interface & simple monitoring console Feb 2010 Production version 1
AIL Challenge #2 (2 sites, 100Kx100MB=10TB, 3 days) Feb 2010 Increased transfer throughput (0.4/sec) Completed (Report)
A/IL GlobusWorld conference Mar 2010 Demonstrations and tutorials Completed (Program online)
W Condor interface to Globus.org prototype Mar 2010 Sandbox and output data movement
F/AW Glide-in WMS with Globus.org prototype Apr 2010 Demonstrate ability to manage direct stage out Completed (DocumentPage)
All Submit paper and/or tutorial for TG conference Apr 2010 Due date May 10, 2010 Completed Waiting to hear about acceptance
All Submit tutorial proposal for SC10 conference Apr 2010 Due date April 5, 2010 Completed Not accepted
All Submit tutorial proposal for SciDAC conference Apr 2010 Due date April 5, 2010 Completed
A/IL Globus.org: Mirroring integrated Apr 2010 First integration of ISI mirroring work (In Progress)
AIL Challenge #3 (10 sites, 10 users, 100Kx200MB=20TB, 3 days) Apr 2010 Multi-sites, multi-users involved in transfers Completed (Report)
I/A SNS/APS mirror demonstration Apr 2010 Mirror SNS data at APS for collaboration and redundancy
W/A Data caching service prototype May 2010 Space allocation, data movement, lifetime management
A/IL Challenge #4 (2 sites, 10Kx2GB=20TB, 3 days) Jun 2010 Increase bandwidth to 300 megabit/sec Completed (Report)
LW OSG user activity: prototype Jun 2010 Monitoring to detect anomalies in OSG user activity
A High Performance Distributed Computing conference Jun 2010 http://hpdc2010.eecs.northwestern.edu/ Completed
A/all Workshop on distributed data management for DOE Jun 2010 Outreach and leadership for DOE data problems Delayed
A/ILW Globus.org: Megellan HDFS integration Jul 2010 Preparing for Glide-in WMS integration and Magellan work
All Globus.org demonstration at SciDAC 2010 conference Jul 2010 No demo, but many useful discussions. Completed
F Glide-in WMS with multiple transfer protocols Jul 2010 Demonstrate ability to transfer output data using different transfer protocols (Report)
A/L ESG data movement to Magellan Aug 2010 Magellan, data mirroring, parallel file system integration
W/AL Challenge #5 (10 sites to 1, 10Kx1GB=10TB, 10 hours, Condor) Aug 2010 Involves movement of Condor sandboxes In progress, may be delayed somewhat by Condor integration work
A/ILW Globus.org: Data caching Aug 2010
A/I ESG data analysis on Magellan Sep 2010 Integration with Hadoop and/or Swift
A/IL GFDL data movement operation Sep 2010 Data from OLCF and/or ALCF to GFDL, 10 TB/day
F/AW Glide-in WMS + Globus.org with caching Sep 2010 Demonstrate ability to manage data staging via cache
A/IL Globus.org: OpenID authentication, cache management Oct 2010 Preparation for Challenge #6
F/AW Glide-in WMS + Globus.org for 10K jobs Oct 2010 Scalability test
W/AL Challenge #6 (2 sites + cache: 10Kx1GB=10TB, 10 hours + 3 days) Oct 2010 Involves use of cache service
A 3rd Cloud Computing and Applications (CCA) workshop Nov 2010 Delayed until Spring 2011
F/AW Glide-in WMS+Globus.org applied to selected HEP user cases Nov 2010 First engagement with high energy physics community
A/All Demonstrate at SC'10 conference Nov 2010 Potential bandwidth challenge participation
W/AI Challenge #7 (2 sites + cache, 10Kx1GB=10TB, 4+10 hours) Dec 2010 Involves use of cache service
A/IL Globus.org: Multi-site mirroring Jan 2011 Can engage heuristics for minimizing data movement
A/IL Globus.org: End-to-end checksum Feb 2011 Reliability
AIL Challenge #8 (1 site to 2 site mirror, 1Kx10GB=10TB, 10 hours) Mar 2011 Scale to more sites if possible
F/AW Glide-in WMS + Globus.org for 100K jobs Mar 2011 Scalability test
AIL Globus.org: Further scalability improvements Apr 2011
A/I ESG data analysis on Magellan Sep 2010 Large-scale analysis: 50+ terabytes
AIL Challenge #9 (10 sites, 100 users 10Kx10GB=100TB, 10 hours) Apr 2011 Multi-user, 20 gigabit/sec aggregate
F/AW Glide-in WMS + Globus.org production use for HEP Jun 2011 Production use
AIL Challenge #10 (2 sites, 10Kx100GB=1PB, 3 days) Jun 2011 Uses Magellan, ANI; requires 31 gigabit/sec sustained
LW OSG user activity, production Jun 2011 Monitoring to detect anomalies in OSG user activity
All Globus.org demonstration at SciDAC 2011 conference Jul 2011 We will have a lot to show and talk about
A/IL GFDL data movement operation Jul 2010 Data from OLCF and/or ALCF to GFDL, 40 TB/day
AIL Globus.org: Further scalability improvements Jul 2011 Addressing in particular number of concurrent users

Personal tools