Please use this identifier to cite or link to this item:
http://dx.doi.org/10.25673/112072
Title: | Model order reduction methods applied to neural network training |
Author(s): | Freitag, Melina A. Nicolaus, J. M. Redmann, Martin |
Issue Date: | 2023 |
Type: | Article |
Language: | English |
Abstract: | Neural networks have emerged as powerful and versatile tools in the field of deep learning. As the complexity of the task increases, so do size and architectural complexity of the network, causing compression techniques to become a focus of current research. Parameter truncation can provide a significant reduction in memory and computational complexity. Originating from a model order reduction framework, the Discrete Empirical Interpolation Method is applied to the gradient descent training of neural networks and analyze for important parameters. The approach for various state-of-the-art neural networks is compared to established truncation methods. Further metrics like L2 and Cross-Entropy Loss, as well as accuracy and compression rate are reported. |
URI: | https://opendata.uni-halle.de//handle/1981185920/114030 http://dx.doi.org/10.25673/112072 |
Open Access: | Open access publication |
License: | (CC BY-NC 4.0) Creative Commons Attribution NonCommercial 4.0 |
Journal Title: | Proceedings in applied mathematics and mechanics |
Publisher: | Wiley-VCH |
Publisher Place: | Weinheim |
Volume: | 23 |
Issue: | 3 |
Original Publication: | 10.1002/pamm.202300078 |
Page Start: | 1 |
Page End: | 8 |
Appears in Collections: | Open Access Publikationen der MLU |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
Proc Appl Math and Mech - 2023 - Freitag - Model order reduction methods applied to neural network training.pdf | 459.88 kB | Adobe PDF | View/Open |