The application of LLM prompt engineering to optimize title screening

Abstract

Screening papers for systematic reviews is a resource-intensive and time-consuming process. This study aims to reduce the necessary resources by automating the title screening phase using Large Language Models (LLMs). Initially, prompt engineering is employed to identify the optimal prompt for the LLM. Subsequently, the performance of the LLM is evaluated against simpler machine learning models to determine its effectiveness in excluding irrelevant papers without false exclusions. The findings indicate that the Large Language Model outperforms simpler machine learning models in the title screening phase, accurately excluding 60% of the papers with only one false exclusion. These promising results suggest that LLMs can assist researchers in the title screening phase, significantly reducing time and costs while maintaining high screening quality.