A Novel Fixed-Point Based Two-Step Inertial Algorithm for Convex Bilevel Optimization in Deep Learning Data Classification

Abstract.

In this paper, we present a new two-step inertial algorithm for finding a common fixed-point of a countable family of nonexpansive mappings. We establish a strong convergence theorem for the proposed method under mild conditions. We then apply this algorithm to solve the convex bilevel optimization problem, demonstrating its versatility in handling complex hierarchical optimization tasks. Additionally, we extend the application of the proposed method to the field of data classification using deep learning, specifically utilizing a multihidden-layer extreme learning machine (MELM). Our numerical tests indicate that the proposed method outperforms existing methods in the literature achieving higher classification accuracy. The empirical findings demonstrate our method’s efficacy and efficiency and indicate its potential for deep machine learning and optimization applications.