Estimating the probability of success of a simple algorithm for switched linear regression

Fabien Lauer (1)

1 ABC - Machine Learning and Computational Biology

This paper deals with the switched linear regression problem inherent in hybrid system identification. In particular, we discuss k-LinReg, a straightforward and easy to implement algorithm in the spirit of k-means for the nonconvex optimization problem at the core of switched linear regression, and focus on the question of its accuracy on large data sets and its ability to reach global optimality. To this end, we emphasize the relationship between the sample size and the probability of obtaining a local minimum close to the global one with a random initialization. This is achieved through the estimation of a model of the behavior of this probability with respect to the problem dimensions. This model can then be used to tune the number of restarts required to obtain a global solution with high probability. Experiments show that the model can accurately predict the probability of success and that, despite its simplicity, the resulting algorithm can outperform more complicated approaches in both speed and accuracy.

Keywords: switched regression; system identification; switched linear systems; piecewise affine systems; sample size; nonconvex optimization; global optimality

Domains: Machine Learning [cs.LG]; Automatic; Optimization and Control [math.OC]

Fabien Lauer. Estimating the probability of success of a simple algorithm for switched linear regression. Nonlinear Analysis: Hybrid Systems, 2013, 8, pp.31-47. ⟨10.1016/j.nahs.2012.10.001⟩. ⟨hal-00743954⟩