Bitte benutzen Sie diese Referenz, um auf diese Ressource zu verweisen:
doi:10.22028/D291-36738
Titel: | Connections Between Numerical Algorithms for PDEs and Neural Networks |
VerfasserIn: | Alt, Tobias Schrader, Karl Augustin, Matthias Peter, Pascal Weickert, Joachim |
Sprache: | Englisch |
Titel: | Journal of Mathematical Imaging and Vision |
Verlag/Plattform: | Springer Nature |
Erscheinungsjahr: | 2022 |
Freie Schlagwörter: | Numerical algorithms Partial differential equations Neural networks Nonlinear diffusion Stability |
DDC-Sachgruppe: | 004 Informatik 510 Mathematik |
Dokumenttyp: | Journalartikel / Zeitschriftenartikel |
Abstract: | We investigate numerous structural connections between numerical algorithms for partial differential equations (PDEs) and neural architectures. Our goal is to transfer the rich set of mathematical foundations from the world of PDEs to neural networks. Besides structural insights, we provide concrete examples and experimental evaluations of the resulting architectures. Using the example of generalised nonlinear diffusion in 1D, we consider explicit schemes, acceleration strategies thereof, implicit schemes, and multigrid approaches. We connect these concepts to residual networks, recurrent neural networks, and U-net architectures. Our findings inspire a symmetric residual network design with provable stability guarantees and justify the effectiveness of skip connections in neural networks from a numerical perspective. Moreover, we present U-net architectures that implement multigrid techniques for learning efficient solutions of partial differential equation models, and motivate uncommon design choices such as trainable nonmonotone activation functions. Experimental evaluations show that the proposed architectures save half of the trainable parameters and can thus outperform standard ones with the same model complexity. Our considerations serve as a basis for explaining the success of popular neural architectures and provide a blueprint for developing new mathematically well-founded neural building blocks. |
DOI der Erstveröffentlichung: | 10.1007/s10851-022-01106-x |
URL der Erstveröffentlichung: | https://link.springer.com/article/10.1007/s10851-022-01106-x |
Link zu diesem Datensatz: | urn:nbn:de:bsz:291--ds-367386 hdl:20.500.11880/33382 http://dx.doi.org/10.22028/D291-36738 |
ISSN: | 1573-7683 0924-9907 |
Datum des Eintrags: | 11-Jul-2022 |
Fakultät: | MI - Fakultät für Mathematik und Informatik |
Fachrichtung: | MI - Informatik MI - Mathematik |
Professur: | MI - Prof. Dr. Joachim Weickert MI - Keiner Professur zugeordnet |
Sammlung: | SciDok - Der Wissenschaftsserver der Universität des Saarlandes |
Dateien zu diesem Datensatz:
Datei | Beschreibung | Größe | Format | |
---|---|---|---|---|
Alt2022_Article_ConnectionsBetweenNumericalAlg.pdf | 1,63 MB | Adobe PDF | Öffnen/Anzeigen |
Diese Ressource wurde unter folgender Copyright-Bestimmung veröffentlicht: Lizenz von Creative Commons