The Nesting-Depth of Disjunctive mu-Calculus for Tree Languages and the Limitedness Problem
Résumé
In this paper we lift the result of Hashiguchi of decidability of the restricted star-height problem for words to the level of finite trees. Formally, we show that it is decidable, given a regular tree language~$L$ and a natural number~$k$ whether $L$ can be described by a disjunctive $\mu$-calculus formula with at most $k$ nesting of fixpoints. We show the same result for disjunctive $\mu$-formulas allowing substitution. The latter result is equivalent to deciding if the language is definable by a regular expression with nesting depth at most~$k$ of Kleene-stars. The proof, following the approach of Kirsten in the word case, goes by reduction to the decidability of the limitedness problem for non-deterministic nested distance desert automata over trees. We solve this problem in the more general framework of alternating tree automata.