Open Access

Sabrina Dill

• In this thesis we investigate the task of automatically detecting phases in surgical workflow in endoscopic video data. For this, we employ deep learning approaches that solely rely on frame-wise visual information, instead of using additional signals or handcrafted features. While previous work has mainly focused on tool presence and temporal information for this task, we reason that additional global information about the context of a frame might benefit the phase detection task. We propose novel deep learning architectures: a convolutional neural network (CNN) based model for the tool detection task only, called Clf-Net, as well as a model which performs joint (context) feature learning and tool classification to incorporate information about the context, which we name Context-Clf-Net. For the phase detection task lower-dimensional feature vectors are extracted, which are used as input to recurrent neural networks in order to enforce temporal constraints. We compare the performance of an online model, which only considers previous frames up to the current time step, to that of an offline model that has access to past and future information. Experimental results indicate that the tool detection task benefits strongly from the introduction of context information, as we outperform both Clf-Net results and stateof-the-art methods. Regarding the phase detection task our results do not surpass state-of-the-art methods. Furthermore, no improvement of using features learned by the Context-Clf-Net is observed in the phase detection task for both online and offline versions