Sub-quadratic Markov tree mixture learning based on randomizations of the Chow-Liu algorithm

Sourour Ammar (1) , Philippe Leray (1) , François Schnitzler (2) , Louis Wehenkel (2)

1 LINA - Laboratoire d'Informatique de Nantes Atlantique
2 Institut Montefiore - Department of Electrical Engineering and Computer Science

The present work analyzes different randomized methods to learn Markov tree mixtures for density estimation in very high-dimensional discrete spaces (very large number n of discrete variables) when the sample size (N ) is very small compared to n. Several sub-quadratic relaxations of the Chow-Liu algorithm are proposed, weakening its search procedure. We first study naîve randomizations and then gradually increase the deterministic behavior of the algorithms by trying to focus on the most interesting edges, either by retaining the best edges between models, or by inferring promising relationships between variables. We compare these methods to totally random tree generation and randomization based on bootstrap-resampling (bagging), of respectively linear and quadratic complexity. Our results show that randomization becomes increasingly more interesting for smaller N/n ratios, and that methods based on simultaneously discovering and exploiting the problem structure are promising in this context.

Domaines: Intelligence artificielle [cs.AI]; Apprentissage [cs.LG]

Sourour Ammar, Philippe Leray, François Schnitzler, Louis Wehenkel. Sub-quadratic Markov tree mixture learning based on randomizations of the Chow-Liu algorithm. PGM 2010, Sep 2010, Helsinki, Finland. pp.17-25. ⟨hal-00568028⟩