DeepWeight - An Investigation Towards Improving Functional Semantic Composition

Abstract

Natural Language Processing inquires about the meaning of lexical items and text segments. In vector semantics, the meaning of words and word combinations is represented by vectors. In natural language, words can be combined. When compared to their components, combined words can represent a different meaning. In an attempt to estimate vectors which represent the meaning of word combinations, using composition models, a set of composition functions is applied to the vectors of the components. Introduced by Dima et al., 2019, compared to its predecessors, the composition model TransWeight achieves the highest accuracy rates of correctly estimating word composition vectors. For this thesis, it is investigated if further increasing the complexity of the TransWeight model improves on its accuracy. Two experiments are conducted with this investigation. First, a new layer is added to a copy of TransWeight, producing the new investigative model DeepWeight. Second, the length of TransWeights transformations axis is increased to relatively higher values. Subsequently, the evaluation results of both experiments are compared to an evaluation of TransWeight. For both experiments, evaluated estimation ranks were near equal to those of the TransWeight model, possibly suggesting that TransWeight is already complex enough. However, potentially, by using different approaches towards increasing model complexity, future research may unravel significant ways to improve accuracy by further increasing the model complexity.