|
|
| CASP16 Experiment |
| Registration |
Targets |
Predictions |
|
|
|
|
|
Detailed description of the experiment
CASP (Critical Assessment of Structure Prediction) is a community wide experiment to determine and advance the state of the art in computational structural biology. Every two years, participants are invited to submit models for a set of macromolecules and macromolecular complexes (proteins, RNA, ligands) for which the experimental structures are not yet public. In the most recent CASP round, CASP15 in 2022, nearly 100 groups from around the world submitted more than 53,000 models on 127 modeling targets in five modeling categories. Independent assessors then compare the models with experiment. Assessments and results are published in a special issue of the journal PROTEINS (check the latest CASP15 issue here).
The goals of CASP are to provide rigorous assessment of computational methods for modeling macromolecular structures and complexes so as to advance the state of the art. Recent CASPs saw enormous jumps in the accuracy of computed structures, first in CASP14 (2020) for single proteins and domains, with many models competitive in accuracy with experiment, and second, in CASP15 (2022), with a large increase in the accuracy of protein complexes. These advances are primarily the result of the successful application of deep learning methods, particularly AlphaFold2 and other methods built around it.
Major interest in the field is now centered around the further potential impact of deep learning methods. In response to that, in the 2022 CASP15, modeling categories were realigned. We are maintaining those categories in CASP16. We will also continue our close collaborations with CAPRI (for protein complexes) and RNA puzzles (for RNA structure).
CASP16 categories are as follows:
- Single Proteins and Domains
As in previous CASPs, the accuracy of single proteins and where appropriate single protein domains will be assessed, using the established metrics. The major emphasis is now on the fine-grained accuracy of models, whether limitations related to sequence alignment depth and target size are surmounted, and whether interdomain relationships are accurately captured. There is also interest how well the many new deep learning methods perform, including those using large language models.
- Protein Complexes
As in recent CASPs, the ability of current methods to correctly model subunit-subunit and protein-protein interactions will be assessed. We will again work in close collaboration with our CAPRI partners. There was enormous progress in this category in the last CASP, but accuracy was not yet as high as for single proteins, so there is substantial room for a further advance. New in this CASP is the option of predicting stoichiometry. Where possible, targets will initially be released without that information, models collected, followed by re-release with that data provided.
- Accuracy Estimation
Members of the community will again be invited to submit accuracy estimates for multimeric complexes and inter-subunit interfaces provided by others. There is no longer a category for general methods of estimating single protein structure accuracy, since in recent CASPs estimates provided by model builders have been consistently more reliable. However, there will be an emphasis on the reliability of accuracy estimates provided with submitted structures, both overall and at the individual amino acid level. Note that all accuracy estimates are in plddt units, not Angstroms.
- Nucleic acid (NA) structures and complexes
An RNA structure category was introduced in the previous CASP and the results were interesting and provocative. In particular, it appeared that deep learning methods were not yet as effective as more traditional ones for this type of macromolecule. Has that now changed? This CASP we expect to include RNA and DNA single structures and complexes, and complexes of these with proteins.
- Protein - organic ligand complexes
The last round of CASP included this category for the first time. Results indicated that, as with RNA structure, deep learning methods were not yet competitive with more traditional approaches. So there is considerable interest in whether that has now changed. In addition to ligands integral to protein targets, we expect to have several target sets related to drug design.
- Macromolecular conformational ensembles
Following the success of deep-learning methods for single structures, it is increasingly important to assess methods for predicting structure ensembles, and CASP included this category for the first time in 2022. While it was clear deep learning methods have considerable potential for generating ensembles, the best procedures are still hotly debated with many new papers appearing. In CASP16, we expect to have a variety of targets for both protein and RNA ensembles.
- Integrative modeling
Deep learning methods combined with sparse experimental data such as SAXS and chemical crosslinking are now being used extensively to obtain the structure of large marcomolecular complexes. To assess effectiveness of these approaches, CASP is reintroducing this category of modeling, provided appropriate targets will be available.
- April 2, 2024 - Start of the registration for CASP16 prediction experiment.
- April 16, 2024 - Start of the testing of server connectivity ("dry run" for server predictors).
- May 1, 2024 - Release of the first CASP16 modeling targets.
- June/July 2024 - Early bird registration for the December CASP16 conference.
- July 31, 2024 - Last date for releasing targets.
- August 31, 2024 - End of the modeling season.
- Early September 2024 - Collection of abstracts describing the methods used in CASP16.
- August-October 2024 - Evaluation of predictions.
- November 2024 - Invitations to groups with the most accurate models
and the most interesting methods to give talks at the CASP16 conference.
- November 2024 - Program of the conference finalized.
- December 2024 - CASP16 Conference (tentatively Nov 30 - Dec 3).
Participation is open to all.
CASP16 registration opens on April 2, 2024.
If you are new to CASP and don't have an account with the Prediction Center, you will have to
register with the Prediction Center first and only then proceed to
CASP16 registration page, which will be available here on April 2, 2024.
If you already have an account with the Prediction Center,
you can go directly to the
CASP16 registration page.
Please check, though, that your basic registration information is
current. If it has changed - please update it through the My Personal
Data link from the main Menu.
Participants with servers are requested to register before April 9, 2024 as
we are planning to start checking servers' format and connectivity thereafter.
CASP16 modeling targets will be announced through the Target List page from
the main CASP16 webpage.
The success of CASP is dependent on the generous help of the experimental community in providing targets. As in previous CASPs, protein crystallographers, NMR spectroscopists and cryo-EM scientists are asked to provide details of structures they expect to have made public before September 15, 2024. All types of macromolecular structures may be good modeling targets, but those that are membrane-related and/or complexes, including combinations of protein, RNA, and DNA components, are particularly needed. Immune-related complexes proved challenging last time, and so targets of that type and also viral-host complexes will be informative. For the protein-organic ligand category, sets of drug design-related complexes are most appropriate, but other complexes are also welcome. Success of the macromolecular ensembles category depends on the availability of targets with multiple experimentally observed conformations. The last day for suggesting proteins as CASP targets is July 20, 2024.
A target submission form is available here.
Details on the target collection and release procedures are available at our
Q&A page.
Models can be submitted through the Prediction Submission form available from
this web site or by the email provided in the
CASP16 format page . Please comply with the instructions on
submission procedures and format provided there.
Server predictions will be made publicly available shortly after the closing of the prediction
window for a specific target.
As is the practice in CASP, assessment of the results will be made by the independent assessor teams. Assessment criteria will be based on those previously developed in CASP, but assessors may add new metrics they consider appropriate. Where possible, results will also be evaluated using criteria from the previous CASP, so the effects of any changes in criteria can be appreciated.
The CASP16 Assessors are as follows:
- Protein structure prediction (single proteins and protein assemblies) - Nick Grishin (University of Texas, Southwestern, USA)
- Model accuracy estimation - Randy Read (University of Cambridge, UK)
- Ligands - Michael Gilson (University of California, San Diego, CA, USA)
- RNA - Rhiju Das (Stanford University, CA, USA) and Eric Westhof (Université de Strasbourg, France)
- Ensembles - Gaetano Montelione (Rensselaer Polytechnic Institute, Troy, NY, USA)
Click here
for the list of assessors in all CASPs held so far.
In accordance with CASP policy, assessors cannot take part in the relevant parts of the experiment as predictors. Participants must not contact assessors directly with queries, but rather these should be sent to the
email
address.
All CASP predictions and results of numerical evaluation will be made available through
this web site shortly before the meeting.
The proceedings will be published in a scientific journal
(see
publications of previous experiments).
All participants will also be required to describe their methods
in the abstracts (published locally at our web site) and encouraged to
discuss them on the
DISCORD forum.
The conference to discuss results of the CASP16 experiment is planned to be held in Caribbean in early December 2024 (tentatively November 30 - December 3, 2024).
John Moult, CASP chair and founder; IBBR, University of Maryland, USA
Krzysztof Fidelis, founder, University of California, Davis, USA
Andriy Kryshtafovych, University of California, Davis, USA
Torsten Schwede, University of Basel, Switzerland
Maya Topf, Centre for Structural Systems Biology, Hamburg, Germany
Minkyung Baek, Seoul University, South Korea
David Baker, University of Washington, USA
Charlotte Dean, University of Oxford, UK
Nick Grishin, University of Texas, USA
Andrzej Joachimiak, Argonne National Lab, USA
David Jones, University College, London, UK
John Jumper, Google Deepmind, London, UK
|
|
|
|
|
