Detailed description of the experiment
The main goal of the CASP experiments is to obtain an in-depth and
objective assessment of our current
abilities and inabilities in the area of protein structure modeling.
To this end, participants will produce models of soon to be released
experimental structures. These models will be true predictions,
not ‘post-dictions’ made on already known structures.
In addition to the traditional themes of CASP, there will be a
strong focus on new methods for predicting three-dimensional contacts,
modeling heterocomplexes and multimers (in collaboration with CAPRI),
and assessment of the extent to which a model can help addressing
specific biological questions, or sparse experimental data and
predicted contacts can improve the accuracy of models.
CASP12 will address the following questions:
-
How similar are the models to the corresponding experimental structure?
-
Are domain orientations, subunit interactions and the protein initeractions
in complexes modeled correctly?
-
How much better are template-based models than those that
can be obtained by simply copying the best template?
-
How reliable are overall, residue, and atomic level error estimates?
-
How much can current refinement methods improve the accuracy of models?
-
How effective are newly emerging methods at predicting protein three dimensional contacts?
-
How well do the models help answering relevant biological questions?
-
How helpful is additional information, such as sparse NMR data, chemical cross-linking
or SAXS in structure modeling?
-
What methods are most effective?
-
Has there been progress since the last CASP?
-
Where can future effort be most productively focused?
-
The High Accuracy Modeling category will include domains
where majority of submitted models are of sufficient accuracy for
detailed analysis.
For CASP12, established numerical methods will be used to evaluate
main chain, side chains, atomic accuracy, and contacts, as well as
hydrogen bonds and covalent geometry. This category replaces the
previous Template Based Modeling category.
- The Biological Relevance category will assess models
on the basis of how well they provide answers to biological questions.
This category builds on the CASP11 pilot assessment. Target providers
will be asked to say what questions prompted the determination of the
experimental structure, and the ability of models to provide answers
to those questions will be compared with the extent to which the
experimental structure can do so in addition to assessing aspects of
accuracy that include sequence alignment, backbone accuracy, and side
chain placement.
-
The Topology category (formerly Free Modeling) will assess
domains where all submitted models are of relatively low accuracy
using the established CASP metrics together with assessor judgment.
-
The Data Assisted category will assess how much the accuracy
of models is improved by the addition of sparse data. Targets for which
such data are available will be re-released after initial data independent
models have been collected, together with the available data.
Data types are expected to include simulated and actual sparse NMR data,
crosslinking data, and low angle X-ray scattering data.
-
The Contact Prediction category will assess the ability of methods
to predict three dimensional contacts in targets structures.
-
The Refinement category will analyze success in refining models
beyond the accuracy obtained in the initial submissions. Selected targets
from among those released in the main modeling experiment will be included.
We will select one of the best models received during the prediction season,
and reissue it as a starting structure for refinement.
-
The Assembly category will assess how well current methods
can determine domain-domain, subunit-subunit, and protein-protein interactions.
As in CASP11, we expect to work closely with CAPRI in this category.
-
The Accuracy Estimation category will assess the ability to provide
useful accuracy estimates for models at the overall, residue, and atomic levels.
-
April 2016 - Start of the registration for CASP12 prediction experiment.
-
April 20, 2016 - Start of the testing of server connectivity ("dry run" for server predictors).
-
May 1, 2016 - Release of the first CASP12 prediction target.
-
July 15, 2016 - Last date for releasing regular prediction targets.
-
July 31, 2016 - End of the regular prediction season.
-
August 18, 2016 - End of the refinement and data-assisted prediction season.
-
July/August 2016 - Early bird registration for the December predictors' meeting.
-
September 2016 - Collection of abstracts describing the methods tested in CASP12.
-
October/November 2016 - Invitation of groups with the most accurate predictions
and interesting methods to give talks at the meeting.
-
Novermber 2016 - Publishing of the program of the meeting.
-
December 2016 - Predictors' meeting to discuss the results of the experiment.
Participation is open to all.
-
If you are new to CASP and don't have an account with
the Prediction Center, you will have to
register with the Prediction Center first and only then
proceed to
CASP12 registration.
-
If you already have an account with the Prediction Center,
you can go directly to the CASP12 registration page.
Please check, though, that your basic registration information is
current. If it has changed - please update it through the My Personal
Data link from the main Menu.
Predictors with servers are requested to register before April 19, 2016 as
we are planning to start checking servers' format and connectivity on that day.
Targets suggested for prediction in CASP12 can be found on the
Target List page.
Details on the target collection and release procedures are available at our
Q&A page.
For the experiment to succeed, it is essential that we obtain the help
of the experimental
community. As in previous CASPs, we invite protein crystallographers and NMR
spectroscopists to provide details of structures they expect to have
made public before September 15, 2016. The last day for suggesting
proteins as CASP targets is July 14, 2016.
A target submission form is available here.
Predictions can be submitted through the Prediction Submission form available from
this web site or by the email provided on the
CASP12 format page . Please comply with the instructions on
submission procedures and format provided there.
Server predictions will be made publicly available shortly after the closing of the prediction
window for a specific target.
As in previous CASPs, independent assessors will evaluate the predictions.
Assessors will be provided with the results of numerical prediction evaluations performed
at the Prediction Center, and will judge the results primarily on that basis.
They will be asked to focus particularly on the effectiveness of different methods.
Evaluation criteria will as far as possible be similar to those used in previous CASPs,
although the assessors are welcome to introduce additional measures.
There will be five assessors, focusing on the following areas of prediction:
-
Biological Relevance of models - Russ Altman (Stanford University, USA)
-
Topology and Data-Assisted modeling -
Matteo Dal Peraro (EPFL, Lausanne, Switzerland)
-
Contacts -
Alexandre Bonvin (University of Utrecht, Netherlands)
-
Refinement -
Francesco Luigi Gervasio (University College London, UK)
-
Assembly (quaternary structure and complexes) -
Guido Capitani (Paul Scherrer Institut, Switzerland)
High accuracy models and estimates of model accuracy will be assessed with standard CASP metrics.
Click here
for the list of assessors in all CASPs held so far.
In accordance with CASP policy, assessors are not directly
involved in the organization of the experiment, nor can they take part
in the relevant parts of the experiment as predictors. Predictors must not
contact assessors directly with queries, but rather these should be
sent to the
email
address.
All CASP predictions and results of numerical evaluation will be made available through
this web site shortly before the meeting.
The proceedings of the meeting will be published in a scientific journal
(see
publications of previous experiments).
All participants will also be required to describe their methods
in the abstracts (published locally at our web site) and encouraged to
discuss them on the
FORCASP forum.
These contributions will be discussed and scored
by other predictors, and this material will be taken into account in
choosing some presentations at the meeting. Also, predictors
presenting posters at the meeting should be prepared to give a short
presentation, as some talks will be invited during the meeting based on the
discussion of poster sessions.
The meeting to discuss results of the experiment will be held
at Hotel Serapo in Gaeta, Italy in December 2016.
The meeting will start at 6pm on December 10 and run through noon of December 13.
The total cost of the meeting, including the early registration fee
and an all-inclusive lodging fee (room, all meals and coffee breaks
for 3 nights) is 850 EURO per person in a single room and 750 EURO in double.
Some financial assistance may be available for the most successful
predictors and students.
Registration for the meeting will open in
July/August 2016.
John Moult, CASP chair and founder; IBBR, University of Maryland, USA
Krzysztof Fidelis, University of California, Davis, USA
Andriy Kryshtafovych, University of California, Davis, USA
Torsten Schwede, University of Basel, Switzerland
Anna Tramontano, University of Rome, Italy
David Baker, University of Washington
Nick Grishin, University of Texas
David Jones, University College, London
Justin MacCallum, University of Calgary
Michael Sternberg, Imperial College, London
|
