Skip to Main content Skip to Navigation
Journal articles

On a multilevel Levenberg–Marquardt method for the training of artificial neural networks and its application to the solution of partial differential equations

Abstract : In this paper, we propose a new multilevel Levenberg–Marquardt optimizer for the training of artificial neural networks with quadratic loss function. This setting allows us to get further insight into the potential of multilevel optimization methods. Indeed, when the least squares problem arises from the training of artificial neural networks,the variables subject to optimization are not related by any geometrical constraints and the standard interpolation and restriction operators cannot be employed any longer. A heuristic, inspired by algebraic multigrid methods, is then proposed to construct the multilevel transfer operators. We test the new optimizer on an important application: the approximate solution of partial differential equations by means of artificial neural networks. The learning problem is formulated as a least squares problem, choosing the nonlinear residual of the equation as a loss function, whereas the multilevel method is employed as a training method. Numerical experiments show encouraging results related to the efficiency of the new multilevel optimization method compared to the corresponding one-level procedure in this context.
Document type :
Journal articles
Complete list of metadata

Cited literature [58 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02956018
Contributor : Open Archive Toulouse Archive Ouverte (oatao) Connect in order to contact the contributor
Submitted on : Friday, October 2, 2020 - 12:23:02 PM
Last modification on : Wednesday, June 9, 2021 - 10:00:35 AM
Long-term archiving on: : Monday, January 4, 2021 - 8:56:00 AM

File

Calandra_26744.pdf
Files produced by the author(s)

Identifiers

Citation

Henri Calandra, Serge Gratton, Elisa Riccietti, Xavier Vasseur. On a multilevel Levenberg–Marquardt method for the training of artificial neural networks and its application to the solution of partial differential equations. Optimization Methods and Software, Taylor & Francis, 2020, pp.1-26. ⟨10.1080/10556788.2020.1775828⟩. ⟨hal-02956018⟩

Share

Metrics

Record views

60

Files downloads

97