Quantitative assessment of relationship between sequence similarity and function similarity
© Joshi and Xu; licensee BioMed Central Ltd. 2007
Received: 05 July 2006
Accepted: 09 July 2007
Published: 09 July 2007
Comparative sequence analysis is considered as the first step towards annotating new proteins in genome annotation. However, sequence comparison may lead to creation and propagation of function assignment errors. Thus, it is important to perform a thorough analysis for the quality of sequence-based function assignment using large-scale data in a systematic way.
We present an analysis of the relationship between sequence similarity and function similarity for the proteins in four model organisms, i.e., Arabidopsis thaliana, Saccharomyces cerevisiae, Caenorrhabditis elegans, and Drosophila melanogaster. Using a measure of functional similarity based on the three categories of Gene Ontology (GO) classifications (biological process, molecular function, and cellular component), we quantified the correlation between functional similarity and sequence similarity measured by sequence identity or statistical significance of the alignment and compared such a correlation against randomly chosen protein pairs.
Various sequence-function relationships were identified from BLAST versus PSI-BLAST, sequence identity versus Expectation Value, GO indices versus semantic similarity approaches, and within genome versus between genome comparisons, for the three GO categories. Our study provides a benchmark to estimate the confidence in assignment of functions purely based on sequence similarity.
Large-scale genome sequencing projects have discovered many new proteins. Of all the proteins whose sequences are known, functions have been experimentally determined for only a small percentage . Annotation of a genome involves assignment of functions to proteins in most cases on the basis of sequence similarity. Protein function assignments based on postulated homology as recognized by sequence identity or significant expectation value of alignment are used routinely in genome analysis. Over the past years, many computational methods [2–11] have been developed to predict function through identifying sequence similarity between a protein of unknown function and one or more proteins with experimentally characterized or computationally predicted functions. However, it is widely recognized that functional annotations should be transferred with caution, as the sequence similarity does not guarantee evolutionary or functional relationship. In addition, if a protein is assigned an incorrect function in a database, the error could carry over to other proteins for which functions are inferred by sequence relationship to the protein with errant function assignment [12–14].
A number of studies in sequence-function relationship have been carried out. Shah et al.  showed that many EC (Enzyme Commission) classes could not be perfectly discriminated by sequence similarity at any threshold. Pawlowski et al.  have studied the relation between sequence similarity and functional similarities based on the EC classification for the E. coli genome. However, this study is limited only to within genome comparisons and lacks any analysis based on inter-genome comparisons. Devos et al.  have studied the complexity in transferring function between similar sequences. Their study shows that binding site, keywords, and functional class annotations are less conserved than EC numbers, and all of them in turn are less conserved than protein structure. Wilson et al. showed that percent identity in sequence alignment is more effective at quantifying functional conservation of their simple classification of SCOP domains than modern probabilistic scores . However, all these studies did not use a broad definition of functions for a systematic large-scale analysis. In this paper, we will build a comprehensive and systematic benchmark for the sequence-function relationship using four model organisms (Arabidopsis thaliana, Saccharomyces cerevisiae, Caenorrhabditis elegans, and Drosophila melanogaster) and controlled vocabularies of function annotation terms in the Gene Ontology  from three different perspectives, i.e., biological process, molecular function, and cellular component.
2. Results and discussion
It has been long recognized that genome annotations using computational methods produce many false function assignments. Many of these methods have been applied to function prediction. They often provide valuable hypotheses, but none are perfect. As a result, it is known that many databases contain incorrect function assignments, and these erroneous assignments propagate from one database to another. Nevertheless, up until now there has been no systematic study for this critical issue. The question whether two proteins are functionally similar is very complex to answer. Function is a very complex notion involving many different aspects including chemical, biochemical, cellular, organism mediated, and developmental processes. Qualitatively it is expected that with higher sequence similarity, the two proteins are more likely to have related functions. However, quantitatively the relationship between function similarity at the different categories and sequence similarity has not been studied deeply. Such a quantitative study is fundamentally important, as it can provide assessment of gene function prediction quality and insights into the underlying mechanisms of new evolving functions through changes in sequence [25, 26].
Our study confirms that sequence comparison often provides good suggestions for gene functions or related functions. These suggestions serve as useful hypotheses for further experimental work to confirm, refine or refute the predictions. Such a process can substantially increase the speed of biological knowledge discovery. On the other hand, when assigning function based purely on similarity to proteins of known function (as annotated in databases), it is important to be aware of incomplete or wrong annotations. Given the value of computational function annotation, our study also shows that a significant portion of gene annotations of biological process, molecular function, and cellular component based solely on sequence similarity, in particular, when the sequence similarity is low, are unreliable. Our study also provides a numerical benchmark for the extent to which one can trust computational annotation. It is possible that a confidence score can be derived from our study for any annotation based on sequence similarity. With this score in the annotation file, the user can have a better insight about the quality of the annotations. Furthermore, our analyses highlights the different sequence-function relationships identified from BLAST versus PSI-BLAST, sequence identity versus Expectation value, GO indices versus semantic similarity approaches and within genome versus between genome comparisons, for the three GO classification types.
There are some limitations in our current study. Our study can only reflect certain aspect of protein function. Protein function variations may result from factors other than sequence, such as alternative splicing and post-translational modification, and our method does not address these factors. Another limitation is that when we assess gene function prediction, we only consider one hit at a time in a database. In many cases, sequence comparison yields multiple hits for one query protein and these hits may have different functions. In our future study, we will develop a new method to assess the function prediction for a query protein by combining the functions of multiple hits while considering the dependence among these functions and the E-values of the hits.
4.1 Protein sequence databases
Details about the four genomes and number of functional annotations in biological process, molecular function and cellular component assigned based on experimental or sequence similarity evidence
# of annotations verified by experimental evidence
# of annotations based on computational methods
# of ORFs
4.2 Protein functional classification
Example of GO index and the corresponding GO ID and functional category
Functional category and GO ID
cellular process (GO:0009987)
cell communication (GO:0007154)
signal transduction (GO:0007165)
cell surface receptor linked signal transduction (GO:0007166)
G-protein coupled receptor protein signaling pathway (GO:0030454)
We assume that the functional relationship between two proteins is reflected by the number of index levels that they share. We have demonstrated the usefulness of such an assumption in our early studies for gene function prediction [28, 29]. We acquired the GO annotations for all the genes in the four genomes and for the three functional categories from GO Website . A gene can (and usually does) belong to multiple indices at various levels in the graph, as proteins may be involved in multiple functions in a cell. Different indices could correspond to the same GO term as well.
Gene Ontology annotation is based on various evidences to annotate functional categories. Towards quality control, all the plots (except for Figure 6B) presented in this paper are based on the annotations with actual experimental evidences such as IDA (inferred from direct assay), IEP (inferred from expression pattern), IGI (inferred from genetic interaction), IMP (inferred from mutant phenotype), IPI (inferred from physical interaction), RCA (inferred from reviewed computational analysis) and TAS (traceable author statement). We performed some comparisons using annotations assigned purely based on computational methods such as ISS (inferred from sequence similarity) and IEA (inferred from electronic annotation), but the plots are not presented here. We have removed the functional annotations that were purely based on evidences such as ND (no biological data available) and NAS (non-traceable author statement.
4.3 Protein functional similarity
Within each family of proteins with similar sequences, functional similarity between proteins is expressed as the number of common roots shared by their functional classification other than the first level, which represents a classification of biological process, molecular function and cellular component. In the case of proteins with multiple functional assignments, the maximum indices of overlap are considered. For example, consider a gene pair ORF1 and ORF2, both annotated proteins. Assume ORF1 has a function represented by GO INDEX 1-1-3-3-4 and ORF2 has a function 1-1-3-2. When compared with each other for the level of matching GO INDEX, they match through INDEX level 1 (1-1) and level 2 (1-1-3) and will have functional similarity equal to 2. The functional similarity defined this way can assume values from 1 to 12.
SS (t1,t2) = -ln p ms (t1,t2)
where, p ms (t1, t2) is the probability of the minimum subsumer for terms t1 and t2. The minimum subsumer for terms t1 and t2 is defined as the common parent of the deepest GO Index level shared by t1 and t2.
4.4 Protein subcellular localization
The subcellular distribution of proteins within a proteome is useful and important to a global understanding of the molecular mechanisms of a cell. Protein localization can be seen as an indicator of its function. Localization data can be used as a means of evaluating protein information inferred from other resources. Furthermore, the subcellular localization of a protein often reveals its activity mechanism. The subcellular localization information was predicted using SubLoc [32, 33, 41]. The five main subcellular localization categories as predicted by SubLoc are Cytoplasmic, Nuclear, Mitochondrial, Transmembrane, and Extracellular. The total numbers of proteins with predicted subcellular localization are 6323 in Saccharomyces cerevisiae, 27,288 in Arabidopsis thaliana, 21,588 in Caenorrhabditis elegans, and 18,498 in Drosophila melanogaster. It is worth mentioning that the subcellular localization predictions were not based on sequence similarity.
4.5 Protein sequence similarity search
The sequence similarity search was done using tools such as BLAST , FASTA [34, 35] and PSI-BLAST . BLAST is the most widely used sequence comparison tool, particularly for genome annotation. FASTA is more sensitive in accuracy but slower than BLAST. Both FASTA and BLAST were developed for pairwise local alignment, with heuristics used. PSI-BLAST is used to identify remote homology based on iterative BLAST searches.
We compared the sequences for within as well as between genome sequence similarities. Each protein sequence was compared against the complete set of proteins for the same genome for within genome comparisons. For between genome comparisons, a pair of similar protein pair was identified using the reciprocal search method , i.e., the two proteins in the pair are the best hits in each other's genome from sequence search. Intra-genome sequence comparison would reflect the sequence similarity between the paralogs; while the inter-genome comparison would partially highlight the orthologous sequence similarities.
To assess the significance of a sequence comparison, an expectation value or E-value can be calculated. This value represents the number of different alignments with the observed alignment score or better that are expected to occur in the database search simply by chance. The E-value is a widely accepted measure for assessing potential biological relationship, as it is an indicator of the probability for finding the match by chance. Smaller E-values represent more likelihood of having an underlying biological relationship. In this study, we will use both E-value and sequence identity as parameters to quantify sequence similarity. On the other hand, E-values depend on a number of computational factors, such as the length of the query protein and the size of search database. The issues prevent the E-value from being a reliable indicator for homology, as addressed in Fig. 1 and related discussions.
The data and results are publicly available at our website .
This research is supported by USDA/CSREES-2004-25604-14708 and NSF/ITR-IIS-0407204. We like to thank the anonymous reviewers for their helpful suggestions.
- Andrade MA, Sander C: Bioinformatics: from genome data to biological knowledge. Current Opinion in Biotechnology. 1997, 8: 675-683. 10.1016/S0958-1669(97)80118-8.PubMedView Article
- Koonin EV, Bork P, Sander C: Yeast chromosome III: new gene functions. The EMBO Journal. 1994, 13: 493-503.PubMed CentralPubMed
- Casari G, Sander C, Valencia A: A method to predict functional residues in proteins. Nature Structural Biology. 1995, 2: 171-178. 10.1038/nsb0295-171.PubMedView Article
- Ouzounis C, Casari G, Sander C, Tamames J, Valencia A: Comparisons of Model Genomes. Trends in Biotechnology. 1996, 14 (B): 280-285. 10.1016/0167-7799(96)10043-3.PubMedView Article
- Schneider R, Casari G, Antoine DD, Bremer P, Schlenkrich M: GeneCrunch: Experiences on the SGI POWER CHALLENGE array with bioinformatics applications. Supercomputer 1996: Anwendungen, Architekturen, Trends. 1997, , 109-119.
- Bork P, Ouzounis C, Sander C: From genome sequences to protein function. Curr Opin Struct Biol. 1994, 4: 39-403. 10.1016/S0959-440X(94)90109-0.View Article
- Bork P, Koonin EV: Predicting functions from protein sequences-where are the bottlenecks?. Nat Genet. 1998, 18: 313-318. 10.1038/ng0498-313.PubMedView Article
- Bork P, Dandekar T, Diaz-Lazcoz Y, Eisenhaber F, Huynen M, Yuan Y: Predicting function: from genes to genomes and back. J Mol Biol. 1998, 283: 707-725. 10.1006/jmbi.1998.2144.PubMedView Article
- Tamames J, Ouzounis C, Casari G, Sander C, Valencia A: EUCLID: Automatic Classification of Proteins in Functional Classes by Their Database Annotations. Bioinformatics. 1998, 14: 542-543. 10.1093/bioinformatics/14.6.542.PubMedView Article
- Andrade MA, Brown NP: "Automated genome sequence analysis and annotation". Bioinformatics. 1999, 15: 391-412. 10.1093/bioinformatics/15.5.391.PubMedView Article
- Koonin EV: "Computational genomics". Curr Biol. 2001, 11: R155-158. 10.1016/S0960-9822(01)00081-1.PubMedView Article
- Brenner SE: Errors in genome annotation. Trends Genet. 1999, 15: 132-133. 10.1016/S0168-9525(99)01706-0.PubMedView Article
- Karp PD: A protocol for maintaining multidatabase referential integrity. Pac Symp Biocomput. 1996, 438-445.
- Karp P: What we do not know about sequence analysis and sequence databases. Bioinformatics. 1998, 14: 753-754. 10.1093/bioinformatics/14.9.753.PubMedView Article
- Pawlowski K, Jaroszewski L, Rychlewski L, Godzik A: Sensitive sequence comparison as protein function predictor. Pac Symp Biocomput. 2000, 42-53.
- Rost B, Valencia A: Pitfalls of protein sequence analysis. Curr Opin Biotechnol. 1996, 7: 457-461. 10.1016/S0958-1669(96)80124-8.PubMedView Article
- Levitt M, Gerstein M: A unified statistical framework for sequence comparison and structure comparison. Proc Natl Acad Sci USA. 1998, 95: 5913-5920. 10.1073/pnas.95.11.5913.PubMed CentralPubMedView Article
- Whisstock JC, Lesk AM: Prediction of protein function from protein sequence and structure. Q Rev Biophys. 2003, 36: 307-40. 10.1017/S0033583503003901.PubMedView Article
- Ponting C: Issues in predicting protein function from sequence. Brief Bioinform. 2001, 2: 19-29. 10.1093/bib/2.1.19.PubMedView Article
- Shah I, Hunter L: Predicting enzyme function from sequence: a systematic appraisal. Proc Int Conf Intell Syst Mol Biol. 1997, 5: 276-283.PubMed CentralPubMed
- Devos D, Valencia A: Practical limits of function prediction. Proteins. 2000, 41 (1): 98-107. 10.1002/1097-0134(20001001)41:1<98::AID-PROT120>3.0.CO;2-S.PubMedView Article
- Wilson CA, Kreychman J, Gerstein M: Assessing annotation transfer for genomics: quantifying the relations between protein sequence, structure and function through traditional and probabilistic scores. J Mol Biol. 2000, 297 (1): 233-49. 10.1006/jmbi.2000.3550.PubMedView Article
- Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ: Basic local alignment search tool. J Mol Biol. 1990, 215: 403-410.PubMedView Article
- Altschul SF, Madden TL, Schaffer AA, Zhang J, Zhang Z, Miller W, Lipman DJ: Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res. 1997, 25 (17): 3389-3402. 10.1093/nar/25.17.3389.PubMed CentralPubMedView Article
- Thornton JM, Orengo CA, Todd AE, Pearl FM: Protein folds, functions and evolution. J Mol Biol. 1999, 293: 333-342. 10.1006/jmbi.1999.3054.PubMedView Article
- Thornton JM: From genome to function. Science. 2001, 292: 2095-2097. 10.1126/science.292.5524.2095.PubMedView Article
- The Gene Ontology Consortium. Nature Genetics. 2000, 25: 25-29. 10.1038/75556.
- Joshi T, Chen Y, Becker JM, Alexandrov N, Xu D: Genome-Scale Gene Function Prediction Using Multiple Sources of High-Throughput Data in Yeast. Saccharomyces cerevisiae. 2004, OMICS: A Journal of Integrative Biology, 8 (4): 322-333.
- Chen Y, Xu D: lobal Protein Function Annotation through Mining Genome-Scale Data in Yeast Saccharomyces cerevisiae. G. Nucleic Acid Research. 2004, 32: 6414-6424. 10.1093/nar/gkh978.View Article
- Lord PW, Stevens RD, Brass A, Goble CA: Investigating semantic similarity measures across the Gene Ontology: the relationship between sequence and annotation. Bioinformatics. 2003, 19 (10): 1275-83. 10.1093/bioinformatics/btg153.PubMedView Article
- Resnik P: Semantic similarity in a taxonomy: an information based measure and its application to problems of ambiguity in natural language. J Artif Intelligence Res. 1999, 11: 95-130.
- Guo T, Hua S, Ji X, Sun Z: DBSubLoc: database of protein subcellular localization. Nucleic Acids Research. 2004, D122-D124. 10.1093/nar/gkh109. 32 Database
- Hua S, Sun Z: Support vector machine approach for protein subcellular localization prediction. Bioinformatics. 2001, 17: 721-728. 10.1093/bioinformatics/17.8.721.PubMedView Article
- Pearson WR, Lipman DJ: Improved Tools for Biological Sequence Analysis. Natl Acad Sci USA. 1988, 85: 2444-2448. 10.1073/pnas.85.8.2444.View Article
- Pearson WR: Rapid and Sensitive Sequence Comparison with FASTP and FASTA. Methods in Enzymology. 1990, 183: 63-98.PubMedView Article
- Tatusov RL, Galperin MY, Natale DA, Koonin EV: The COG database: a tool for genome-scale analysis of protein functions and evolution. Nucleic Acids Res. 2000, 28: 33-36. 10.1093/nar/28.1.33.PubMed CentralPubMedView Article
- Clusters of Orthologous Groups (COGs). [http://www.ncbi.nlm.nih.gov/COG/]
- The Gene Ontology. [http://www.geneontology.org/]
- The Arabidopsis Information Resource (TAIR). [ftp://ftp.arabidopsis.org/]
- National Center for Biotechnology Information(NCBI). [ftp://ftp.ncbi.nih.gov/]
- Subcellular Localization Prediction of Eukaryotic Proteins (SubLoc). [http://www.bioinfo.tsinghua.edu.cn/SubLoc/eu_predict.htm]
- Data and Results website. [http://digbio.missouri.edu/sfsimilarity]
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.