Lukovnikov, Denis: Deep Learning Methods for Semantic Parsing and Question Answering over Knowledge Graphs. - Bonn, 2022. - Dissertation, Rheinische Friedrich-Wilhelms-Universität Bonn.
Online-Ausgabe in bonndoc: https://nbn-resolving.org/urn:nbn:de:hbz:5-66709
@phdthesis{handle:20.500.11811/9810,
urn: https://nbn-resolving.org/urn:nbn:de:hbz:5-66709,
author = {{Denis Lukovnikov}},
title = {Deep Learning Methods for Semantic Parsing and Question Answering over Knowledge Graphs},
school = {Rheinische Friedrich-Wilhelms-Universität Bonn},
year = 2022,
month = may,

note = {Recently, the advances in deep learning have lead to a surge in research on semantic parsing and question answering over knowledge graphs (KGQA). Significant improvements in these fields and in natural language processing (NLP) in general have been achieved thanks to the use and better understanding of training neural-networks-based models. Particularly important in training any model for any task is their generalization ability. While the generalization ability of machine learning models can be improved with general techniques (e.g. dropout), semantic parsing and KGQA present unique generalization challenges that have been a focal point of research in the field. Other important aspects when using machine learning are its computational efficiency and response time, as well as the ability to measure the reliability of the predictions on given inputs.
In this thesis, we explore some questions regarding the generalization challenges in semantic parsing and KGQA. We also explore the tasks of out-of-distribution (OOD) detection for semantic parsing models, as well as the challenge of reducing the number of decoding steps.
In particular, we investigate zero-shot or out-of-vocabulary generalization in KGQA with simple questions, which require only a single triple pattern to find the answers. Here, we are concerned with the ability to generalize to entities and relations that were not observed during training. Another question we investigate is the ability to detect compositionally OOD examples. Recent work has shown that standard neural semantic parsers fail to generalize to novel combinations of observed elements, which humans can easily do. While different works have investigated specialized inductive biases and training techniques, to the best of our knowledge, none have focused on detecting whether the inputs are compositionally OOD, which is the focus of our work. The third question we focus on is transfer learning in the context of KGQA, where we investigate its impact on both simple questions and more complex questions. Since the emergence of large-scale pre-trained language models (PLM), transfer learning from PLM's has been shown to significantly improve accuracy on various NLP tasks. In this thesis, we look at transfer learning from PLM's, additionally providing a qualitative analysis of the model and an investigation of data efficiency. We also look at transfer learning between KGQA tasks. A unique aspect that can be present in tasks requiring the generation of formal languages is order-invariance in the queries, which is the fourth point we focus on in this thesis. For example, in SQL, the order in which the conditions appear in the WHERE clause does not matter: the meaning of the query remains the same. Nevertheless, when training, typically only one possible linearization of the query is used, which can lead to the learning of spurious patterns that do not generalize. In this thesis, we investigate whether the standard training can be problematic and also explore an order-invariant training method. Finally, we also explore insertion-based decoding in semantic parsing. Usually, semantic parsing is performed in an auto-regressive left-to-right manner, which requires as many decoding steps as there are tokens in the decoded sequence. In this thesis, we explore alternative decoders that rely on insertion, and can thus achieve a sub-linear number of decoding steps and have different independence assumptions between output variables. In addition, we propose a novel insertion-based decoder for trees.},

url = {https://hdl.handle.net/20.500.11811/9810}
}

Die folgenden Nutzungsbestimmungen sind mit dieser Ressource verbunden:

Namensnennung - Weitergabe unter gleichen Bedingungen 4.0 International