Peptide spectrum matching, comparison between search engines by using a large synthetic peptide library.

Abstract

Comprehensive peptide identification is an essential part of mass spectrometry-based proteomics. Multiple tools exist to cross-correlate a reference library to spectra obtained via mass spectrometry. Using raw data from a synthetic peptide library containing normal and phospho peptides as a benchmark, we could compare the annotated MS/MS spectra between multiple search engines (MaxQuant, MSFragger, SEQUEST, Mascot, and Comet). Data analysis revealed that assessing the quantity of annotated PSMs is not an accurate representation of the quality of search engines, for example, actual false discovery rate (FDR) is much higher than selected in search engine setup. Or because some search engines annotates additional PSMs for already annotated peptide feature, leading to a three-fold decrease in unique peptides. Comparison of the overlap of unique peptides between search engines revealed that only 51.7% of normal peptides and 41.8% of phospho peptides are shared between three or more search engines. Additionally we did calculate phospho posttranslational modification (PTM) localization score using the same algorithm for all search engines to assess the accuracy of assignment of the phospho PTM position. We did observe distinct pattern of increased number of false positives among the assigned locations for spectra with lower PTM localisation probabilities. The overall assessment of search engine comparison is that each annotates slightly different set of PSMs and that to improve confidence in annotations we recommend to analyse your data with software tools.