Introduction

CASP experiments aim at establishing the current state of the art in protein structure prediction, identifying what progress has been made, and highlighting where future effort may be most productively focused.

There have been four previous experiments, in 1994, 1996, 1998 and 2000. Full details of these may be found at the CASP web site, http://predictioncenter.org/, and in the special issues of the journal PROTEINS: 23(5), 1995; Suppl 1, 1997; Suppl 3, 1999 and Suppl 5, 2001. In CASP4, 163 prediction groups from all around the world took part.

We now announce the fifth experiment. As before, the goal is to obtain an in-depth and objective assessment of our current abilities and inabilities in this area. To this end, participants will predict as much as possible about a set of soon to be known structures. These will be true predictions, not ‘post-dictions’ made on already known structures.

The experiment will begin in May 2002, when the first prediction targets will be made available. The prediction season will run through August. There will be a meeting at the Asilomar Conference Center in California December 1 - 5^th, to discuss the results.

In more detail:

Scope of CASP5 and Related Experiments

As in earlier CASPs, all types of methods for predicting protein structure will be considered, ranging from comparative modeling through fold recognition and 'new fold' prediction. Most emphasis will be on tertiary structure prediction but secondary structure and contact prediction methods will also be included. A new category, prediction of disordered regions in proteins, will be introduced. In addition, there will be additional activities included in CASP5, which will extend the scope substantially:

1. CAFASP3: In the era of genome sequencing, rapid protein structure modeling methods have a critical role to play. This experiment, led by Dani Fischer, will evaluate automatic methods of predicting protein structure, using the CASP targets. All targets will be processed through prediction servers that register for the experiment. Models will be evaluated by the same numerical criteria as used for CASP5, as well as the CAFASP criteria, and a session at the CASP5 meeting will compare performance through the servers with that obtained when full human participation is allowed. Further details of this experiment may be found at: http://www.cs.bgu.ac.il/~dfischer/CAFASP3.

2. Large scaling benchmarking: It is hoped that the results of well run benchmarking experiments such as EVA and LIVEBENCH will also be included in the CASP meeting and publications.

3. Ten Most Wanted: Results of the community effort to produce useful models of ten proteins of particular biological or medical interest will also be discussed at the meeting.

Experiment

The broad goals of the CASP5 experiment are to address the following questions about the current state of the art in protein structure prediction:

1. Are the models produced similar to the corresponding experimental structure?
2. Is the mapping of the target sequence onto the proposed structure (i.e. the alignment) correct?
3. Have similar structures that a model can be based on been identified?
4. Are the details of the models correct?
5. Has there been progress from the earlier CASPs?
6. What methods are most effective?
7. Where can future effort be most productively focused?

In addition, CASP5 will focus particularly on areas of prediction that previous CASPs have shown to be current bottlenecks to progress. Suggested problem areas are:

1. Alignment of a sequence onto a template fold.
2. Model refinement - improving the accuracy of an initial model.
3. Accurately modeling regions of insertion and deletion relative to a template structure.
4. Improved fold recognition, particularly for analogous, analogous/new fold targets.
5. Improved New Fold methods.
6. Bold new methods aimed at removing one of the current bottlenecks to progress.

To facilitate progress in some of these areas, partly built models will be provided where necessary, for example, correct alignment as a starting point for loop building, and full approximate models as a starting point for refinement. The MD community is encouraged to participate, making use of these starting points.

The set of problem areas may be revised, following discussion with the consultancy groups. Participants are advised to check for revisions.

Prediction Targets

For the experiment to succeed, it is essential that we obtain the help of the experimental community. As in previous CASPs, we will invite protein crystallographers and NMR spectroscopists to provide details of structures they expect to have made public before 1st October 2002. Prediction targets will be made available through the web site. All targets will be assigned an expiry date, and predictions must be received and accepted before that expiration date.

Participation

Participation is open to all. Those interested in receiving mailings concerning progress of the experiment may also register as 'observers'. CAFASP predictors must register at that web site. Participation in both experiments (initial submission of server generated models to CAFASP and subsequent submission of models based on human interaction to CASP) is encouraged. Note that separate registrations are required for CASP and CAFASP participation.

Assessment of Predictions

As in previous CASPs, independent assessors will evaluate the predictions. There will be three assessors, representing expertise in the comparative modeling, fold recognition and new fold prediction areas. Assessors will be provided with the results of numerical evaluation of the predictions, and will judge the results primarily on that basis. They will be asked to focus particularly on the effectiveness of different methods.

As CASP has grown, the work of the assessors has become more and more demanding. To help them with their task, predictors and target submitters will be asked to assist in the evaluation of models.

Numerical evaluation criteria will as far as possible be very similar to those used in CASP4, although the assessors may be permitted to introduce some additional ones. Appropriate members of the prediction community will be asked to develop methods of assessing the statistical significance of performance ranking, a controversial area in the past.

Release of Results

1. All CASP predictions and evaluations will be made available through this web site, shortly before the meeting.

2. The proceedings of the meeting will be published. In recent CASPs, the large number of predictors together with the limited number of published predictor papers has been one of the main causes of excessive focus on winners and losers. To combat that, the mix of papers in the special issue will altered. There will not be a predefined number of 'winner' group papers. Instead, papers that focus on problem areas and solutions will be included.

3. In a further change to increase recognition of the contributions of more predictors, a new web site will be developed, allowing all participants to report their work if they wish, and encouraging vigorous discussion of the results.

Meeting

A meeting will be held 1-5 December, 2002 at Asilomar, California, USA to evaluate the results of the prediction experiment. The meeting will be limited to about 200 participants and precedence will be given to active predictors. It is hoped that some financial assistance will be available for the more successful predictors. It is expected that the format of the meeting will be changed from previous CASPs, concentrating more on progress in problem areas, rather than the best performers in each section, irrespective of whether there has been a significant change since last time.

Organizing Committee

John Moult, CARB, University of Maryland, USA, (jmoult@tunc.org)
Krzysztof Fidelis, Lawrence Livermore National Laboratory, USA, (fidelis@llnl.gov)
Adam Zemla, Lawrence Livermore National Laboratory, USA, (adamz@llnl.gov)
Tim Hubbard, Sanger Centre, Hinxton, UK, (th@sanger.ac.uk)

Support

The organizers gratefully acknowledge support for the CASP5 experiment from the NIH and the Department of Energy. Additional industrial support is being sought.