Assessment of novel fold targets in CASP4: Predictions of three-dimensional structures, secondary structures, and interresidue contacts

In the Novel Fold category, three types of predictions were assessed: three-dimensional structures, secondary structures, and residue-residue contacts. For predictions of three-dimensional models, CASP4 targets included 5 domains or structures with novel folds, and 13 on the borderline between Novel Fold and Fold Recognition categories. These elicited 1863 predictions of these and other targets by methods more general than comparative modeling or fold recognition techniques. The group of Bonneau, Tsai, Ruczinski, and Baker stood out as performing well with the greatest consistency. In many cases, several groups were able to predict fragments of the target correctly - often at a level somewhat larger than standard supersecondary structures - but were not able to assemble fragments into a correct global topology. The methods of Bonneau, Tsai, Ruczinski, and Baker have been successful in addressing the fragment assembly problem for many but not all the target structures. In the case of secondary structure predictions, few groups performed consistently well, independently of the set of targets considered, and according to different evaluation schemes. CASP4 results can be interpreted as a reasonably accurate snapshot of the current status of the field at the time of the CASP experiment. Ambiguity in quantifying progress in the field is not so much due to the limited number of targets available for a CASP experiment, but to the huge disparity in the degree of difficulty of these targets. This finding suggests that an evaluation scheme should be calibrated to take this variation into account. This variation in prediction accuracy as a function of target difficulty is a general problem for all types of predictions. We propose a possible solution and apply it to secondary structure prediction results. The best residue-residue contact predictions showed significantly improved results relative to CASP3.