| dc.description.abstract | Personal values are referenced in natural language text through subtle cues that indicate a person’s priorities and beliefs. Understanding these values requires advanced natural language understanding to correctly interpret subtleties and nuances. Existing research on extracting personal values from text often utilizes methods that lack sufficient natural language understanding and do not consider the context of a text. In this study, we present VIVE (Value Identification and Value Extraction), a novel end-to-end method for the identification and extraction of context-specific personal values from natural language text. VIVE leverages a hybrid intelligence approach to identify which values are particularly important in a given context (Value Identification) and utilizes the natural language understanding capabilities of state-of-the-art large language models (LLMs) to extract the identified values from text (Value Extraction). To evaluate VIVE, we conduct a case study with the Netherlands Red Cross in which we elicit the requirements of humanitarian organizations with regard to processing feedback data from humanitarian programs. We apply VIVE to the context of a humanitarian program within which the Red Cross collects chat messages from Telegram groups, written by Ukrainian refugees or internally displaced people. VIVE is used to 1) identify a set of context-specific personal values for the data set of Ukrainian Telegram messages and 2) extract these values from the messages. We evaluate the accuracy, precision, recall, and F1 score of the value extraction and we conduct a user study with Red Cross analysts to evaluate the usefulness of VIVE. We find that large language models can accurately extract personal values from text and outperform a traditional dictionary-based approach. Based on this result, we make a comparison of three state-of-the-art LLMs and find no significant difference in their accuracy for value extraction. Furthermore, we show that representing personal values not only through names but also with natural language descriptions significantly improves the accuracy of value extraction and we present a value representation format that is suitable for an LLM-based value extraction. | |
