On the Size of Homogeneous and of Depth-Four Formulas with Low Individual Degree

: Let r ≥ 1 be an integer. Let us call a polynomial f ( x 1 , x 2 ,..., x N ) ∈ F [ x ] a multi-r -ic polynomial if the degree of f with respect to any variable is at most r . (This generalizes the notion of multilinear polynomials.) We investigate the arithmetic circuits in which the output is syntactically forced to be a multi-r -ic polynomial and refer to these as multi-r -ic circuits. We prove lower bounds for several subclasses of such circuits, including the following.


Introduction
Arithmetic models of computation.An arithmetic circuit computes a polynomial function over some underlying field F via a sequence of operations involving + and × starting from its inputs x 1 , x 2 , . . ., x N .We typically allow arbitrary constants from F on the incoming edges to a + gate so that a + gate can in fact compute an arbitrary F-linear combination of its inputs.The complexity of a circuit is measured in terms of its size 1 and depth. 2 Being the most natural and intuitive way to compute polynomials, arithmetic circuits have been widely investigated (see for example the book by Bürgisser, Clausen and Shokrollahi [3] or the more recent surveys by Shpilka and Yehudayoff [32] and by Saptharishi [31] for an overview of the problems and results in this area).A central open problem in this area is to prove arithmetic circuit lower bounds (for some explicit family of polynomials).Progress on it has been made in the form of lower bounds for some subclasses of circuits, one of the most significant ones being Raz's lower bound for multilinear 3 formulas4 [28].We study formulas that are a natural generalization of the class of multilinear formulas.We now give some more motivation before giving a precise definition of the relevant circuit subclasses studied and the lower bounds obtained here.
Background.Motivated by the question of whether computation can be efficiently parallelized, one thread of work in this area [11,36,1,29,20,34,9] gives the loss in size incurred in transforming a general circuit or formula into one of low depth (sometimes with additional structural restrictions on the resulting low-depth circuits).In particular, these results say that proving sufficiently strong lower bounds for (subclasses of) low-depth circuits implies lower bounds for arbitrary circuits as well.Low-depth circuits being easier to analyze, this might be a potential pathway to general lower bounds.Somewhat promisingly, a lot of new lower bounds have recently been proved for various subclasses of arithmetic circuits, particularly for low-depth subclasses [12,8,13,6,4,22,14,24,17,23,21].
However, most of the present lower bounds can only handle subclasses of circuits having formal degree which is rather low (equal to or sometimes slightly larger than the number of variables).To make progress and remove the limitations of multilinearity and low formal degree, some recent work [18,27] looks at a model that generalizes multilinear circuits/formulas.Such a generalization also appears in the hardness versus randomness trade-off result for bounded-depth circuits [5].
The multi-r-ic circuit model.Intuitively, a multi-r-ic circuit is an arithmetic circuit in which the output polynomial is syntactically constrained to have degree at most r with respect to any individual variable.We now make this precise.Definition 1.1.Define the formal degree of a node α with respect to a variable x i in an arithmetic circuit inductively as follows.For a leaf α it is 1 if α is labelled with x i and zero otherwise; for an internal node α labelled with × (labelled with +) it is the sum (the maximum, respectively) of the formal degrees of the children with respect to x i , The formal degree of a circuit is the formal degree of its output node.We call an arithmetic circuit a multi-r-ic circuit if the formal degree of the output node with respect to each variable is at most r.
The formal degree of a circuit represents what the degree of the output would have been if there were no cancellations and is always an upper bound on the degree of the output.Note that the formal (total) degree of a N-variate multi-r-ic circuit can be (r • N) which is asymptotically larger than N when r = ω (1).In this work, we prove lower bounds for several subclasses of multi-r-ic circuits.Now, once one has a lower bound for some explicit polynomial f (x) ∈ F[x] against a circuit subclass C, it is desirable to try to make such an f come from as small a class D as possible. 5Following this general theme, we have tried to minimize the complexity of our target polynomial f .
A polynomial is called homogeneous if all its monomials have the same total degree.A circuit is called homogeneous if all its internal nodes compute a homogeneous polynomial.
Our results.Our lower bounds hold over any field unless mentioned otherwise explicitly.Our first result is a superpolynomial lower bound for homogeneous multi-r-ic formulas.
Theorem 1.2.Homogeneous multi-r-ic formulas.Let r = r(N) be any integer.There exists an explicit family of N-variate multi-r-ic polynomials P N,r such that any homogeneous multi-r-ic formula computing P N,r must have a size of at least 2 Ω(log 2 (N)) .Moreover, if the field F contains r distinct r-th roots of unity and has a size of at least (2Nr) then P N,r can be computed by a poly(Nr)-sized (nonhomogeneous) depth-three multi-r-ic formula.
We probe a bit further in this direction and obtain improved lower bounds for homogeneous multi-r-ic formulas of low depth.
Theorem 1.3 (Constant-depth homogeneous multi-r-ic formulas).Let r = r(N) be an integer and let p be an integer.If p ≤ 0.9 log N log r + log log N (1.1) then there exists an explicit family of N-variate multi-r-ic polynomials P N,r such that any homogeneous multi-r-ic formula of product-depth p computing P N,r must have a size of at least Moreover, if the field F contains r distinct r-th roots of unity and has a size of at least (2Nr) then P N,r can be computed by a poly(Nr)-sized (nonhomogeneous) depth-three multi-r-ic formula.
Let us notice that, unlike Theorem 1.2, the previous bound degrades with the parameter r.
The proofs of these two results follow the strategy of first facilitating the analysis by reducing the depth 6 and then proving lower bounds against the resulting low-depth formula.As one hopes to be able to implement some such strategy to obtain lower bounds against more powerful subclasses of circuits (maybe even general arithmetic circuits), it makes sense to prove lower bounds against low-depth multi-r-ic circuits (without the homogeneity restriction).Also, the "moreover" part of the above theorems shows that nonhomogeneous depth-three multi-r-ic formulas are superpolynomially more powerful than homogeneous multi-r-ic formulas of arbitrary depth.This further motivates our next few results on lower bounds for low-depth multi-r-ic formulas without the homogeneity restriction.We represent a circuit of depth p with a sequence of p alternating symbols (either Σ or Π) wherein the leftmost symbol denotes the nature of the output gate.So for example, a ΣΠΣΠ circuit is a depth-four circuit where the output gate is an addition gate.We denote by IMM n,d the (1, 1)-th entry of the product of d matrices of size n × n each.
Theorem 1.4 (Depth-four multi-r-ic formulas computing IMM).For any integer r = r(n) such that r 1.1 n and for any d r, any multi-r-ic ΣΠΣΠ circuit computing IMM n,d where d ≥ log 2 n must have a size of at least7 (n/r 1.1 ) Ω( Moreover, one can notice that the reduction [35] of IMM n,d as a projection of Det n•d maintains the multi-r-icity.Note that the target polynomials, namely IMM n,d and Det n , in the above theorems are multilinear polynomials.If we allow our target polynomial to be a multi-r-ic polynomial then we can obtain a lower bound that does not degrade at all as r increases. Theorem 1.6 (Depth-four multi-r-ic formulas computing a multi-r-ic polynomial).For any positive integer r = r(N) there exists an explicit family {G N,r } of multi-r-ic N-variate polynomials such that any multi-r-ic ΣΠΣΠ-circuit computing G N,r must have a size of at least 2 Ω( √ N) .Moreover, one can even choose such a family G N,r so that it can in fact be computed by a poly(Nr)-sized multi-r-ic algebraic branching program.
The three previous bounds hold for ΣΠΣΠ-circuits.Moreover, we can improve them in the case of ΣΠΣ-circuits.
Theorem 1.7 (Depth-three multi-r-ic formulas computing IMM).For any integer r = r(n) such that r n, any multi-r-ic ΣΠΣ-circuit computing IMM n,d has a size of at least (n/r) Ω(d) .
Corollary 1.8 (Depth-three multi-r-ic formulas computing the determinant).For any integer r = r(n), any multi-r-ic ΣΠΣ-circuit computing Det n has a size of at least 2 Ω(n/r) .Theorem 1.9 (Depth-three multi-r-ic formulas computing a multi-r-ic polynomial).For any positive integer r = r(N) there exists an explicit family {G N,r } of multi-r-ic N-variate polynomials such that any multi-r-ic ΣΠΣ-circuit computing G N,r must have a size of at least 2 Ω(N) .Moreover, one can choose such a family G N,r so that it can in fact be computed by a poly(Nr)-sized ΠΣΠ-circuit.
One can notice that for any constant r the lower bound 2 Ω(N) in Theorem 1.9 and the lower bound 2 Ω( √ N) in Theorem 1.6 are optimal since there is a depth-three circuit of size 2 O(N) in the former case and a depth-four circuit in the latter case, computing the target polynomial.
Comparison to previous work.As mentioned earlier, most prior work failed to yield (superpolynomial) lower bounds when the degree of the polynomial being computed and/or the formal degree of the circuit was significantly larger than the number of variables. 8In this particular aspect, the above results represent significant progress.For example, Theorem 1.2 and Theorem 1.6 yield (superpolynomial) lower bounds against natural subclasses of circuits wherein the formal degree is allowed to be arbitrarily larger than the number of variables.These results also yield improved lower bounds for some previously studied subclasses.
1. Multilinear ΣΠΣΠ Circuits.While the focus of this work is on multi-r-ic formulas for r > 1 for which lower bounds were previously not known, our results have interesting consequences for the much more well-studied special case of r = 1 corresponding to multilinear formulas.Previously, the best known 9 lower bound against multilinear-ΣΠΣΠ circuit computing any explicit N-variate polynomial of degree d was 2 Ω( , where d N. Note that this does not increase with N. 10 The special case of Theorem 1.4 for r = 1 yields a n Ω( √ d) = (N/d) Ω( √ d) lower bound for multilinear-ΣΠΣΠ circuits computing IMM n,d .
2. Multi-r-ic ΣΠΣ-circuits.Multi-r-ic depth-three circuits were recently studied in [17] and a lower bound of 2 Ω(N/2 r ) was obtained for an explicit multi-r-ic N-variate polynomial.In particular, no superpolynomial lower bound seems to have been known when r = ω(log N).In comparison, Theorem 1.9 gives an exponential lower bound which is independent of r.
8 Sometimes, as over finite fields [7], high formal degrees are allowed but these models can be "easily" transformed (and this is how the proofs of these lower bounds work) into ones of low formal degrees.Notable exceptions could be found in other recent works trying to break this barrier [16,23,25]. 9Actually a lot of the work on multilinear formulas deal with polynomials such as the determinant and the permanent where the number of variables N is a fixed function of d, e. g., N = d 2 in the case of the permanent and the determinant.Therefore the statements of the results themselves do not reveal the structure of the lower bound as a function of both N and d.It seems that the proof technique employed in Raz [28] and Raz-Yehudayoff [30] only yields a lower bound that is independent of N for when N is much larger than d (in a multilinear polynomial N cannot be smaller than d), a key initial step in their argument involving random restrictions kills off the extra variables so that the number of surviving variables is comparable to the degree and then works with this restricted polynomial. 10The work of [6] looks at multilinear-ΣΠΣΠ-circuits with the additional restriction of homogeneity and obtains a n Ω( lower bound for IMM n,d . THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 Moreover, Theorem 1.2 and Theorem 1.3 are a generalization (from r = 1 to arbitrary values of r) of the work of [10] and follows the same overall proof strategy.The main difference appears at the step of monomials counting.They directly bound the number of monomials which appear in the formula whereas we bound the number of occurrences of a particular subset of these monomials.

Proof overview
In this section we give an overview of the proof of some of these lower bounds with an emphasis on those ingredients of the proof that are new here as compared to prior work in the area.
Homogeneous multi-r-ic formulas.Our proof is a generalization (from r = 1 to arbitrary values of r) of the work of [10] and follows the same overall proof strategy of first doing a depth reduction 11 in order to make the resulting expression easier to analyze and then proving lower bounds on the size of such expressions.Roughly, if f is any N-variate polynomial computed by a multi-r-ic formula Φ of size s then f can be written as where each T i is a multi-r-ic polynomial that can be expressed as a product of (log N)-many homogeneous polynomials where each Q i j has at least √ N-many fresh variables (2.2) (see Lemma 4.2 and the preceding discussion for the precise definitions and statements 12 ).Moreover, if the formula Φ is also homogeneous then each Q i j is homogeneous as well.We then carefully choose a subset of multi-r-ic monomials 13 which we refer to as extremal monomials (see Definition 4.5 for the precise statement) and employ a result from extremal combinatorics called Sperner's theorem to upper bound the number of extremal monomials in a homogeneous multi-r-ic term T of the form given by Equation (2.2).We then choose our target polynomial f to have the maximum possible number of extremal monomials (and also to be easily computed by a nonhomogeneous multi-r-ic depth-three circuit).Finally looking at the ratio of the number of extremal monomials in f to that in T yields the stated lower bound.
Depth-Four Circuits.The proof for multi-r-ic depth-four circuits builds on some of the recent work [14,24] on homogeneous depth-four circuits and shares many ingredients with these.Let f be a polynomial computed by a small multi-r-ic depth-four circuit.We first employ random restrictions to reduce the support size 14 of the monomials appearing in our depth-four circuit.We then get a representation of the following form: where each term T i is a multi-r-ic polynomial of the form every monomial in each Q i j has a relatively small number of variables.Since each T i is multi-r-ic, the number D of factors in it can at most be (N • r) and in general this upper bound is tight.Now such representations did occur at the intermediate stages in some recent pieces of work [14,24] and a complexity measure called dimension of projected shifted partials was devised to analyze these.It involves looking at all the low-order partial derivatives of f , multiplying these with monomials of a suitable degree, applying a carefully chosen linear operator π : F[x] → F[x] and looking at the dimension of the resulting set of polynomials.It turns out that when the number of factors D significantly exceeds the number of variables N the bounds on the dimension of shifted partials that were proved in earlier works do not seem to yield any nontrivial lower bound overall.The key observation here is that one can get around this difficulty using a complexity measure that is some sort of a hybrid of the shifted partials measure with what is used in the work of [28,30].Specifically, we partition our set of variables x into two sets x = y z where the size of y is significantly larger compared to the size of z. 15 We observe that if instead of taking all the (low order) derivatives, we take the (low order) derivatives with respect to only the y-variables and subsequently set them to zero then effectively the number of factors becomes more like |z| • r which is much smaller than (|x| • r) = (N • r) while still giving us a large enough space of partial derivatives to work with.In order to highlight this idea and illustrate its power in a simpler situation, in Theorem 6.4 we first show how this can be used to obtain a (N/r 3 ) Ω(d) lower bound against multi-r-ic depth-three circuits computing an explicit multilinear polynomial of degree d on N = Θ(d 3 ) variables (by considering f d,d/3 in Theorem 6.4).We then show how some of the other ingredients from some recent lower bound proofs [12,8], combined with some judicious bounds on the dimension of some relevant sets of polynomials (Lemma 7.4) can be used to obtain a 2 Ω( √ N) lower bound for multi-r-ic depth-four circuits computing an explicit polynomial (Theorem 1.6).This lower bound degrades with r in case the target polynomial is multilinear (Theorem 1.4).

Notation
[n] shall denote the set of first n positive integers, i. e., [n] = {1, 2, . . ., n}.For any finite set A and any integer k ≤ |A|, A k shall denote the set of all subsets of A of size exactly k.We will often use boldfaced letters for tuples of variables or numbers.For example x will usually denote an N-tuple of variables (x 1 , x 2 , . . ., x N ) while r = (r 1 , r 2 , . . ., r N ) will usually denote an N-tuple of non-negative integers (the dimension N will usually be clear from context). 14The support size of a monomial is the number of distinct variables appearing in it, i. e., if N is a monomial then the support size of m, denoted by |Supp(m)| is the size of the set {i : e i ≥ 1} ⊆ [N]. 15For comparison, [28,30] also partition the variables into two sets but it is crucial to their argument that the two sets have nearly the same size and that the partition is chosen randomly.In contrast, we choose the partition deterministically and it is crucial to our argument that the two parts have rather unequal sizes.THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 Some numerical estimates.We will employ the following well-known numerical estimates: (given by Stirling's formula).
e x ≥ 1 + x for all x ∈ R, and Let x = (x 1 , x 2 , . . ., x N ) be an N-tuple of formal variables.We work with the ring of polynomials F[x] over some underlying field F. We will use the following notation related to (sets) of polynomials and maps between them.
Sets of polynomials.For a subset of variables z ⊂ x, we will denote by z = the set of all monomials in the z variables of degree exactly .Thus if z = (z 1 , z 2 , . . ., z m ) then Similarly, z ≤ shall denote the set of all monomials over the z-variables of degree at most .Let y ⊆ x be a subset of variables of x. ∂ ∂ ∂ =k y f shall denote the set of all k-th order partial derivatives of f with respect to the y variables, i. e., For two sets of polynomials A, B ⊆ F[x], the set A • B shall be the set of pairwise products, i. e., For a set of polynomials A ⊆ F[x], the dimension of A will denote the dimension of the F-vector space generated by A.
Degree with respect to a subset of variables.Let f (x) ∈ F[x] be a polynomial and z ⊆ x be a subset of variables.The z-degree of f , denoted by deg z ( f ), is the degree of f when viewed as a polynomial over the z-variables with coefficients from the function field F(y), where y = x \ z.
be a monomial.The support of m, denoted by Supp(m), is the subset of variables appearing in it.The z-support of m, denoted by Supp z (m)), is the subset of z-variables appearing in it.Formally, Isomorphic polynomials.We will say that two N-variate polynomials f (x), g(x) ∈ F[x 1 , x 2 , . . ., x N ] are isomorphic if one can be obtained from the other via a renaming of the variables, i. e., there exists a permutation π ∈ S N such that Linear maps and homomorphisms.We will sometimes need to take a subset y ⊆ x of variables from x and set them to zero in some relevant collection of polynomials.We employ the following notation in this regard.For a subset y ⊆ x, σ y : shall denote the homomorphism corresponding to setting the variables in y to zero.It is formally defined as 16 : For any set of polynomials S ⊆ F[x] and any (linear) map σ : shall denote the set of polynomials obtained by applying σ to every polynomial in S, i. e., The Iterated Matrix Multiplication.Let n, d ≥ 2 be two parameters.We consider polynomials defined on variable sets x 1 , . . ., x d .For i ∈ [d] \ {1, d}, let x i be the set of variables x i, j,k for j, k ∈ [n]; for i = 1, let x 1 be the set of variables x 1,1, j and for i = d, let x d be the set of variables x d, j,1 where j ∈ The Iterated Matrix Multiplication polynomial on x, denoted by IMM n,d , is defined to be Note that the polynomial IMM n,d is the value of the product of times), and n × 1).

Multi-r-ic models
Let Φ be an arithmetic formula.Φ ∈ F[x] will denote the output polynomial computed by Φ while Size(Φ) will denote the size (the number of leaves) of Φ.If α is a node of Φ, we will denote by Φ α the sub-formula rooted at α.In analyzing formulas in which the formal degree of every variable is bounded, it is naturally convenient to keep track of the degree of each variable in the polynomials computed at intermediate nodes of the formula.We will employ the following notation for this purpose.
1. Support of r.The support of r is the set of indices of the non-zero coordinates.More formally, 3. Multi-r-ic formulas.We say an arithmetic formula Φ is a multi-r-ic formula if for all i ∈ [N], the formal degree of Φ with respect to the variable x i is at most r i .
For conciseness, a multi-(r, r, . . ., r)-ic polynomial 17 (resp.formula) will be referred (as defined before) to simply as a multi-r-ic polynomial (resp.formula).In particular, multi-1-ic polynomials are exactly multilinear polynomials and multi-1-ic formulas are syntactically multilinear formulas.This notational device will aid us in analyzing formulas via the labelling of every node α with an N-tuple that upper bounds the syntactic degree of the polynomial computed at α with respect to the various formal variables.We refer to such labelled formulas as certified multi-r-ic formulas and the precise properties that such labellings satisfy are captured in the definition below.Definition 3.3 (Certified multi-r-ic formulas).Let r = (r 1 , r 2 , . . ., r N ) be an N-tuple of nonnegative integers.A certified multi-r-ic formula is an arithmetic formula such that each gate α is labelled by an N-tuple d α = (d 1 , . . ., d N ) of non-negative integers such that • the output is labelled by r, • and if α is a multiplication gate of children β 1 , β 2 , . . ., β p then18 It is readily verified that a natural top-down labelling procedure provides such a labelling to any multi-r-ic formula.Proposition 3.4.Let r = (r 1 , r 2 , . . ., r N ) be an N-tuple of nonnegative integers.If Φ is a certified multi-ric formula, then the formal degree of Φ with respect to the variable x i is at most r i .Conversely, if Φ is a multi-r-ic arithmetic formula, then there exists a labelling of the vertices such that the labelled formula is certified multi-r-ic.
Proof.The forward direction is immediate by definitions.Now consider the other direction.We will assign a labelling so that if a gate α is labelled by d = (d 1 , d 2 , . . ., d N ), then the formal degree of α with respect to x i is at most d i .Let us show it by descending induction on Φ.If α is the output, we label it by r.By assumption we have that the degree with respect to each variable x i is bounded by r i .If α is an addition gate labelled by d α such that its children are not labelled, we label all the children by d α .As the formal degrees of the children are bounded by the degree of α, the condition is satisfied.Otherwise α is a multiplication gate labelled by let the formal degree of β j with respect to x i be e ji .Then the formal degree of α with respect to x i equals e 1i + • • • + e pi and is upper bounded by d i .Then for all j ≥ 2 we assign the label to the child β j and to the first child β 1 .Thus the sum of the d β j equalsd α and it is easy to check that the i-th coordinate of d β j is an upper bound for the formal degree of β j with respect to x i (for all j ∈ [p] and i ∈ [N]).

Homogeneous multi-r-ic formulas
In this section we implement the strategy outlined in section 2 to obtain superpolynomial lower bounds for homogeneous multi-r-ic formulas.

Log-product decomposition
In this section we show that if a multi-r-ic polynomial f (x) is computed by a multi-r-ic formula Φ of size s then f can be written as a sum of s polynomials having a rather special structure that we will exploit.We first capture the structure of the summands in the following definition.
Definition 4.1.Let d = (d 1 , . . ., d N ) be an N-tuple of nonnegative integers and T (x) ∈ F[x] be a polynomial on N variables.Let v, L ≥ 0 be integers.We will say that T has a where each d j ∈ Z N ≥0 is an N-tuple and each g j (x) is a multi-d j -ic homogeneous polynomial such that: THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 Intuitively, the first three conditions specify that T is a multi-d-ic polynomial that is a length-v product of multi-d i -ic polynomials.The fourth condition intuitively says that each factor g i contains at least L fresh variables, i. e., variables which do not occur in the previous g j 's.factors g j .Lemma 4.2.Let v ≥ 1 be an integer, d = (d 1 , . . ., d N ) be an N-tuple of non-negative integers and f be a N-variate polynomial computed by a homogeneous multi-d-ic formula of size s.
We will need a suitable adaptation of an observation that is common to many depth-reduction results for arithmetic (and even Boolean) formulas.Proposition 4.3.Let f (x) be a polynomial computed by a certified multi-d-ic formula Φ and let α be any node in Φ having a label d α .Then there exist certified formulas Ψ and Λ such that where 3. If Φ is homogeneous then Ψ, Λ are also homogeneous formulas.
4. If Φ is of product-depth p then Ψ, Λ are of product-depth at most p.
Proof.Let β 0 = α, β 1 , β 2 , . . ., β t be the sequence of nodes in the (unique) path from α to the root node of Φ.So β 1 is the parent of α, β 2 is the grandparent of α and so on and β t is the root node of Φ.It suffices to prove (via induction) that for every i ≥ 0 The base case i = 0.This is easy to verify.The inductive step.Let β i+1 have children β i and γ 1 , . . ., γ q with labels d β i and d γ 1 , . . ., d γ q , respectively.There are two cases.The node β i+1 is a + node.Then by definition we have The other case of the inductive step is where β i+1 is a × gate.Then by definition we have THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 Now define with the natural labelling to the root nodes of Ψ i+1 and of Λ i+1 that this definition entails.Then Λ i+1 is computed by a certified multi-d β i+1 -ic formula while Ψ i+1 is computed by a certified multi-e-ic formula, where e = (d Finally note that in the two cases we have: This completes the proof of the inductive step.The last two properties of Ψ, Λ are also easily checked from their definitions.This completes the proof of the proposition.
Here we see how to use it to get the desired depth reduction.
Proof of Lemma 4.2.By Proposition 3.4 we can assume that the formula Φ is in fact a certified multi-d-ic homogeneous formula.We can also assume without loss of generality 19 that every node in the formulas has fanin20 at most 2. We prove the result by induction on v and s.
If s = 1, the formula is just a leaf node and f depends on at most one variable-say x i .Now since the support of d is large (at least Then, it is easy to check that f is in (d, v, √ N )-form via the pairs That concludes this case.
Induction step.Let us prove the lemma for a fixed value of (v, s) with v ≥ 2. By hypothesis, f is computed by a homogeneous certified multi-d-ic formula Φ.There are two cases.
To see this first note that since α is a leaf node so Φ α depends on at most one variable-say x a .Now since the support of Notice that d v has no restriction about the size of its support and it just contains the remainder of the support.Also we have With the above facts in hand, it is easy to check that That concludes this case.
• Otherwise all the leaves have support smaller than |Supp(d)|/3.As the fan-in of every gate is bounded by 2, there exists a node α in Φ with label d α such that By Proposition 4.3, there exist certified formulas Ψ and Λ such that where size of Φ α and of Λ is at most (s − 1) each.So we apply the induction hypothesis on Φ α and Λ.Now Λ is a certified multi-d-ic formula so by induction hypothesis THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 where each T i has a (d, v, √ N )-form and s 1 ≤ Size(Λ).Now Ψ is a homogeneous multi- By the induction hypothesis, where each Since it suffices to show that for all i the polynomials This completes the inductive step and hence proves the lemma.

Counting extremal monomials
From the log-product decomposition described in the previous section, our problem boils down to understanding the sums of (d, v, L)-forms.Our next definition will help us describe the weakness of such summands that is being exploited here.In this subsection, we will show that a term of our log-product decomposition (Corollary 4.4) has a relatively small number of extremal monomials.Specifically, Lemma 4.6 (Upper bound on the number of extremal monomials in a term).Let T (x) be an N-variate polynomial and d ∈ Z N ≥0 be an N-tuple of non-negative integers.If T is homogeneous and has a (d, v, L)-form then the number of d-extremal monomials in T is at most The rest of this subsection is devoted to a proof of this lemma.We will need the following result from extremal combinatorics due to Sperner [33] (a proof of this can be found in the book [2]).
In the following, 2 A will denote the set of all subsets of A.
Let us consider the set This contradicts the premise that g is homogeneous.
The final inequality is given by Proposition 3.1.
Lemma 4.9.Let d 1 , d 2 , . . ., d v ∈ Z N ≥0 be N-tuples and g 1 (x), g 2 (x), . . ., g v (x) ∈ F[x] be N-variate polynomials such that the i-th polynomial g i (x) (4.23) If all the g i are homogeneous polynomials then the number of d-extremal monomials in T is at most Proof.We prove it by induction on v.We can first assume without loss of generality that Supp(d) = [N] (otherwise for any i ∈ [N] \ Supp(d) the corresponding variable x i does not appear in any monomial in T and we can effectively remove this variable from consideration).If v = 1, the result is directly given by Lemma 4.8.Now consider the case Let us first assume that the claim is true.In this case, by induction hypothesis the number of e-extremal monomials in g is at most 2 Fix an e-extremal monomial m 1 in g.We observe that there is a one-one correspondence between the set of all possible monomials m 2 such that (m 1 • m 2 ) is d-extremal and subsets of Z.To see this observe that fixing m 1 completely determines the degree of m 2 with respect to any Y -variable 21 x j -if j ∈ Y then For the remaining variables, i. e., the Z-variables, the degree of m 2 with respect to x j must be either 0 or (d j − e j ) = d j .Define the subset of Z corresponding to m 2 as We now use the homogeneity of g v+1 and proceed as in the proof of Lemma 4.8 (by considering here the degree with respect to Z-variables) to deduce that the set forms an antichain in 2 Z .So by Sperner's theorem I m 1 can have a size of at most

.34)
This proves the claim and hence the lemma as well.
The upper bound on the number of d-extremal monomials in a (d, v, L)-form follows immediately.
Proof of Lemma 4.6.Since T is in (d, v, L)-form by definition there exist pairs and Furthermore, since T is homogeneous, all the g i are homogeneous as well.Applying Lemma 4.9 we get that the number of d-extremal monomials in T is at most as required.

Putting things together
Our target polynomial is a multi-r-ic adaptation of the elementary symmetric polynomial obtained by raising every variable to the r-th power.Specifically, let Then P N,r is a homogeneous multi-r-ic polynomial of degree rN/2 which contains N N/2 extremal monomials.A relatively straightforward adaptation of an observation attributed to Michael Ben-Or [26] implies that P N,r is easy to compute.Proposition 4.11.Let F be a field which contains r distinct rth roots of unity and size of F is at least (2Nr).The polynomial P N,r defined above can be computed by a multi-r-ic (nonhomogeneous) ΣΠΣ circuit of size O(N 2 r) on F.
Proof.Let X 1 , . . ., X N ,t be N + 1 variables.Let ξ 1 , . . ., ξ r be the r distinct rth roots of the unity.We consider the polynomial For all i ≤ r and all j ≤ N, we know that (X j − ξ i t) is a factor of (X r j − t r ) and so Then, for every integer p such that 0 ≤ p ≤ N, the coefficient of t pr in Q(X,t) is Consequently, P N,r = (−1) N/2 [t Nr/2 ]Q(X,t) .Moreover, if q is an integer which is not a multiple of p, then the coefficient of t q in Q(X,t) is zero.Finally, we can extract any coefficient by interpolation: let a 1 , . . ., a N+1 be elements of F such that the elements a r 1 , . . ., a r N+1 are pairwise distinct (this is possible since the size of F is at least r(N + 1)).We get where λ 1 , . . ., λ N+1 are constants of F. The formula is of the required form.
We can finally prove Theorem 1.2.THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 Proof of Theorem 1.2.Our target polynomial is the polynomial P N,r defined above.Suppose that it is computed by a homogeneous multi-r-ic formula Φ of size s.Let r = (r, r, . . ., r).By Corollary 4.4, Φ can be written as a sum, of size at most s, of homogeneous (r, log(N)/4 , √ N )-form polynomials.By Lemma 4.6 each one of these polynomials can compute at most 2 N /(2 log 2 N/16−log(N)/4 )-many r-extremal monomials.But P N,r has N N/2 -many r-extremal monomials.Therefore, using Proposition 3.1, we must have The moreover part follows from Proposition 4.11.

Constant-depth homogeneous multi-r-ic formulas
We follow the same overall proof strategy as in the previous section to obtain a lower bound for homogeneous multi-r-ic formulas of small product-depth.With definitions as in section 4, the depth reduction for low-depth formulas that we obtain is: Lemma 5.1.Let r ≥ 1 be an integer and r = (r, r, . . ., r).Let f be a degree-d polynomial computed by a homogeneous certified multi-r-ic formula Φ of size s and product-depth bounded by p.For any positive integer v ≤ 1 8r there exist s homogeneous (r, v, 2)-form polynomials T 1 , T 2 , . . ., T s such that f = This depth reduction proceeds in two stages.The following definition will help us describe the structure of the output of the first stage.Definition 5.2.Let d = (d 1 , . . ., d N ) be an N-tuple of non-negative integers.We will say that a polynomial T (x) is (d,t, L)-balanced if T is multi-d-ic and there exist t pairs (d 1 , g 1 ), (d 2 , g 2 ), . . ., (d t , g t ) where each g i is a multi-d i -ic polynomial such that Let us notice that if a polynomial is (d, a, L)-balanced for a parameter a > t, then we can easily get a (d,t, L)-balanced expression by grouping some factors.
The first stage flattens the formula into a sum of a small number of balanced polynomials and its proof idea comes from [10].Specifically, we have: Lemma 5.3.Let r ≥ 1 be an integer and let r = (r, r, . . ., r).Let Φ be a certified multi-r-ic homogeneous formula of product-depth p computing a polynomial of degree d ≥ 2r.
, then we can again set α = γ.Otherwise there exists γ i such that deg(γ i ) ≥ d/(4t).In this case, Φ γ i is of product-depth p < p and degree at least d/(4t).Using the hypothesis over t, we know that We now show that Φ α is (d α ,t, 2)-balanced.Let the children of α be β 1 , β 2 , . . ., β v .We greedily merge pairs of polynomials and also add the corresponding d β i and d β j .This means that Φ α can be written as (5.9) 22 Recall that for a node α, the total formal degree of α is denoted by deg(α) and that if a node alpha has certification (d 1 , . . ., d N ), then ∑ i d i is only an upper bound on deg(α).THEORY OF COMPUTING, Volume 14 (16), 2018, pp.This implies that a is at least t.Finally, rewriting Φ α as -balanced where a ≥ t.This proves the claim.
Let α be a node given by Claim 5.4.By Proposition 4.3, there exist a d-certified homogeneous formula Λ of product-depth at most p and a (d − d α )-certified homogeneous Ψ such that (5.10) By induction on size, Λ can be expressed as a sum of at most Size(Λ)-many (d,t, 2)-balanced polynomials.Now Φ α is (d α ,t, 2)-balanced and so ( Ψ • Φ α ) is (d,t, 2)-balanced (by grouping Ψ with another factor).
Altogether, Φ can be written as a sum of (d,t, 2)-balanced polynomials, the number of summands being at most 1 + Size(Λ) ≤ Size(Φ).This proves the lemma.
Proof.By the premise of the lemma, there exist pairs The proof is by reordering and regrouping the f i in a series of v steps.The following claim captures the invariants after the k-th step of this process.
Intuitively, the set A is the accumulator of good polynomials, B is the source of fresh polynomials and C denotes the set of throwaway polynomials.

Proof. Let us prove the claim by induction on k.
Base case k = 0.For the base case of k = 0, we can choose A = C = / 0 (so that e = (0, 0, . . ., 0)) and B = [t] and the claim is easily verified using Equation (5.11).Inductive step.Assume k < v and let [t] = A B C be the partition obtained after the k-th step.Let e = ∑ i∈A d i and let Z = [N] \ Supp(e) be the set of (indices of) variables which do not appear in A. We can associate to each f i with i ∈ B the support of d i in Z: (5.12) We choose a factor f u with u ∈ B such that the size of its corresponding support (Supp(d u )) in Z is minimum (as k < v, the set B is non-empty) and add it to A. We then remove from B the neighbouring factors F defined as The updated partition is given by To complete the induction, it suffices to verify that the size of B is large enough (the other conditions follow from the choice of u and F).In particular, by induction hypothesis, we know that |Supp | − 1 for all j ∈ F. Now counting the occurrences of the variables in Supp Z (d u ) in d u + ∑ j∈F d j and using the fact that T is multi-r-ic we get that (5.15) (5. 16) (5.17) This completes the proof of the inductive step and hence of the claim.
Let us come back to the proof of the lemma.The previous claim when k = v states that T can be written as it ensures that v is a positive integer.By Lemma 5.1, Φ can be written as a sum of at most s homogeneous (r, v, 2)-form polynomials.By Lemma 4.6 each one of these polynomials can compute at most 2 N /(2 v/2 )many r-extremal monomials.But P N,r has N N/2 -many r-extremal monomials.Therefore we must have (using the hypothesis over p) The moreover part follows from Proposition 4.11.
Remark 5.7.We noticed that for proving Theorem 1.3, we just need in fact the weaker hypothesis The hypothesis of the theorem ensures that the final lower bound is superpolynomial.
6 A lower bound for depth-three multi-r-ic circuits

The complexity measure
In this section we prove a lower bound for multi-r-ic depth-three circuits by employing some sort of a hybrid of the complexity measures used by Raz [28] and by Nisan and Wigderson [26] and recently introduced in [15] called dimension of skewed partials.We pick a suitable set of variables y ⊂ x, take k-th order derivatives with respect to these variables (for a suitably chosen value of k), then set the y-variables to zero and finally count the dimension of the resulting set of polynomials.In the notation introduced in section 3 our measure is the dimension of the set which we denote by and we sometimes refer to it as the skewed partials complexity of f (SkP y -complexity of f for short).

Upper bounding the SkP y -complexity of a depth-three circuit
We first observe that in order to upper bound the SkP y -complexity of a ΣΠΣ-circuit C, it suffices to upper bound the SkP y -complexity of a single term.
Proposition 6.1 (Sub-additivity of the complexity measure).For any pair of polynomials g(x), h(x) ∈ F[x] and any subset of variables y ⊆ x and any integer k ≥ 0 we have THEORY OF COMPUTING, Volume 14 (16), 2018, pp.This is easily verified.We next derive an upper bound for the SkP y -complexity of a term T of a multi-r-ic ΣΠΣ-circuit C by observing that the SkP y -complexity of T depends only on a relatively small subset of the affine forms in T .Lemma 6.2 (Upperbound on SkP y -complexity of a depth-three circuit).Let x = y z be any partition of the variable set with |z| = m.If C is a multi-r-ic depth-three circuit (i.e., C = T 1 + T 2 + • • • + T s , where each T j is a product of affine forms and every variable occurs in at most r affine forms within a given T j ) then Proof.Let x = y z, where |z| = m.Consider a term T of our circuit C. T is a product of affine forms.
Since the term T is assumed to be multi-r-ic, we have that each z-variable appears in at most r affine forms inside T .In particular, there are at most m • r affine forms in T which contain a z-variable.So let where t ≤ m • r, each i (y, z) is an affine form that depends on some z-variable and Q(y) is the product of all the affine forms in T which depend only on the y-variables.To prove the lemma it suffices (via Proposition 6.1) to show that It suffices to show that Since σ y : This last statement is easily seen by starting from the definition of T given by Equation (6.5) and computing the appropriate derivatives.

A multilinear polynomial with high SkP y -complexity
So our problem boils down to finding an explicit polynomial f (x) such that is large.Additionally, it is desirable that f should be as easy to compute as possible.We show that there is a polynomial f computed by a small ΠΣΠ-circuit 23 whose dim σ y ∂ ∂ ∂ =k y f -complexity is large.
Moreover, f n,k can be obtained as a restriction of IMM n,d+2k simply by substituting some subset of variables in IMM n,d+2k to zero/one values.
Proof.We first give the description of the family of polynomials { f n,k (x) : k ≥ 0} along with the associated partition of variables x = y z.From the description itself, it will be clear that f n,k has the required degree and number of variables and is computed by a Π [3] -circuit.We first partition our set of (n 2 k + 2nk) variables into two sets The y and z are further partitioned into k sets of equal size: where for each i ∈ [k] we have |y i | = n 2 and |z i | = 2n and the overall structure of f n,k is: where each g i (y i , z i ) is a degree three polynomial over the indicated set of variables.For each i ∈ [k], y i will consist of the n 2 variables {y i,a,b : a, b ∈ [n]} and z i consists of the 2n variables {z i,a,b : a Finally define It remains to show that dim To see this note that any k-th order derivative of f with respect to the y-variables is either zero or a monomial in the z-variables.There are (n 2 ) k nonzero derivatives of f with respect to the y variables (obtained by picking exactly one variable from each y i , i ∈ [k]) and moreover these give rise to distinct monomials in the z-variables and hence are linearly independent as well.Therefore, For the moreover part, first note that from the definition of the family f n,k it follows that it is computed by a multilinear-Π [k] Σ [n 2 ] Π [3] -circuit.So it is a restriction of IMM n,d+2k . 24

Putting things together
With the above upper and lower bounds in our hand, we are ready to prove a lower bound for multi-r-ic ΣΠΣ-circuits.
Theorem 6.4.There exists an explicit family f n,k (x) of multilinear polynomials of degree d = 3k on N = Θ(n 2 d) variables such that f n,k is computed by a poly(n, d)-sized multilinear ΠΣΠ-circuit 25 but any multi-r-ic ΣΠΣ-circuit computing f n,k must have top fanin at least . (6.17) Proof of Theorem 6.4.Let f n,k (y, z) be the polynomial (family) along with the indicated partition of variables as described in Lemma 6.3.Indeed, as it is noticed in the proof of Lemma 6.3, it is computed by a multilinear-Π Then by Lemma 6.2 and Lemma 6.3 we have as required.
As it was noticed before, the polynomial f n,k is a restriction of the iterated matrix multiplication IMM n,d+2k .So, Theorem 1.7 directly follows from Theorem 6.4.

Depth-four multi-r-ic circuits with low support
As mentioned in the overview, we prove a lower bound for multi-r-ic ΣΠΣΠ circuits by first using random restrictions to reduce the number of variables appearing in any monomial and then proving lower bounds against such representations.Let us give names to such polynomials and circuits.Definition 7.1 (Support size of a polynomial and low-support ΣΠΣΠ circuits).Let z ⊆ x be a subset of variables.The support size of a polynomial f (x) ∈ F[x] (resp.the z-support size of f ) is the maximum support size (resp.z-support size) of any monomial appearing in f .We will call a depth-four circuit C as a τ-supported depth-four circuit, denoted by ΣΠΣΠ {τ} , if it is of the following form: where each term T i is of the form We remark that the support size of f is the maximum of the sizes of the supports of the monomials of f , which can be really smaller than the size of the set of the variables which appear in f .

7.1
The complexity measure Definition 7.2 (Shifted Skewed Partials).Let x = y z be a partition of our set of variables into two parts y and z.For a polynomial f (x) ∈ F[x], define the dimension of shifted y-partials, SSP ,y,k ( f ) for short, as follows: SSP ,y,k ( f 7.2 Upper bounding the SSP-complexity of a multi-r-ic low support depth-four circuit Let C be a multi-r-ic depth-four circuit with z-bottom support bounded by τ.We now give an upper bound on SSP ,y,k (C).
We now provide the following upper bound on the SSP-complexity of a term of such a circuit.upper bound for a term T which is of the form where each Q i (y, z) is a polynomial of z-support at most τ, R(y) is an arbitrary polynomial over the indicated set of variables and T is multi-r-ic.Note that since each Q i has z-support at most τ and is multi-r-ic, the z-degree of each Q i is at most r • τ.We first do a preprocessing part in which we focus on the z-degree of the Q i and combine those with low z-degree.Specifically, if any two of the Q i have z-degree less than r • τ/2 we multiply them together to obtain a new combined factor having z-degree at most rτ.At the end, if it remains one Q i of z-degree less than rτ/2 and if we can multiply it with another Q i without exceeding the z-degree rτ, we merge them.After this preprocessing, all the Q i but at most one have z-degree greater than rτ/2 (and the last part cannot be merged), thus the average z-degree is at least rτ/2.Now, since the polynomial T is multi-r-ic, its z-degree is at most rm.Hence Now it suffices to show that In turn, it suffices to show that which in turn follows from .11)We show this last containment via induction on k.For k = 0, the containment in Equation (7.11) follows from the definition of T Equation (7.6) itself.For the inductive step, suppose that g(y, z) ∈ ∂ ∂ ∂ =k y T is a k-th order derivative of T .By the inductive assumption we have that g(y, z) can be expressed as a F-linear combination of polynomials of the form where THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 i 0 ∈ S and y ∈ y be a variable.Differentiating Equation (7.12) with respect to y we have In the third step above we use the fact that Q j has z-degree at most rτ and hence for all j we have that Q j as well as This completes the induction step for the proof of Equation (7.11) and hence proves the lemma.
As an immediate corollary, we obtain an upper bound on the SSP-complexity of a multi-r-ic circuit having bottom support bounded by τ.

A multilinear polynomial with high SSP-complexity
The explicit polynomial.Our explicit polynomial is a generalization of the one in section 6 for multir-ic depth-three circuits.Lemma 7.5.There exists an explicit family of polynomials F n,d,k (y, z) such that for any δ ≤ 1/5, n ≥ 3, d, k, we have where M = (n 2−δ /2) k .Moreover, F n,d,k can be obtained as a restriction of IMM n,d+2k simply by substituting some subset of variables in IMM n,d+2k to zero/one values.
THEORY OF COMPUTING, Volume 14 (16), 2018, pp.As the proof of the lemma is a bit technical, we postpone it at the end of the section (in subsection 7.6).We now describe the family F n,d,k of the lemma above.Let The polynomial F n,d,k is of the form: where the g i are polynomials over the indicated (disjoint) subsets of variables defined as so that the overall number of y-variables is |y| = (n 2 ) • (k) and the overall number of z-variables is The proof that this family has the properties claimed in Lemma 7.5 is very similar to a corresponding one in the work of Fournier, Limaye, Malod and Srinivasan [6].For the sake of completeness, we included a proof at the end of the section.

Putting things together
Armed with these upper and lower bounds on SSP-complexity, we are now ready to prove the lower bound for multi-r-ic ΣΠΣΠ {τ} -circuits.Our polynomial F n,d,k (x) is as described above.
Claim 7.7.For the above choice of parameters we have M Proof.We want to constraint β to ensure the claim, i. e., We have which is less than or equal to 1 as soon as .

A multi-r-ic polynomial with high SSP-complexity
We also observe here that if our target polynomial is also allowed to be multi-r-ic (so that the total degree of our target polynomial can be Θ(rN)) then we can obtain a significantly improved lower bound-a lower bound that is independent of r (and exponential in (N/τ)).Proof.Our target polynomial is just the same polynomial as in section 7.3 but wherein the z-variables are raised to the r-th powers.In more detail, for positive integers n, k, r and α define the polynomial F n,k,r,α as follows.
F n,k,r,α (y, z) where the g i are polynomials over the indicated (disjoint) subsets of variables defined as so that the overall number of y-variables is |y| = (n 2 ) • (k) and the overall number of z-variables is The analogous lower bound in this case is that for any δ ≤ 1/5, n ≥ 3, d, , k we have where M = (n 2−δ /2) k .Then for proving Equation (7.30) is larger or equal to M |z|+ |z| /2, we need rτ ≥ εβ ln n.Thus taking r ≥ 1, τ ≥ 2 is sufficient for the choice of parameters given below.Combining this with the upper bound for the circuit given by Lemma 7.4 and proceeding as in the proof of Theorem 7.6, we finally arrive at a lower bound of we get s = 2 Ω(N/τ) , as required.(7.33)We recall that a ΣΠΣΠ circuit is called τ-supported if the product gates in the top layer have a support of size at most τ.In particular, from a depth-3 ΣΠΣ circuit, it is possible to add a top layer of product gates of fan-in one (and so of support one).So, by choosing τ = 2 in Theorem 7.8, it directly implies Theorem 1.9.THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 7.6 Proof of Lemma 7.5 Lemma 7.9 (Restatement of Lemma 7.5).There exists an explicit family of polynomials F n,d,k (y, z) such that for any δ ≤ 1/5, n ≥ 3, d, k, we have where M = (n 2−δ /2) k .Moreover, F n,d,k can be obtained as a restriction of IMM n,d+2k simply by substituting some subset of variables in IMM n,d+2k to zero/one values.
Proof.The third step is due to the fact that the dimension of the vector space generated by a set of monomials is exactly the cardinality of this set.
In the following, we will consider a subset of {∂ ∂ ∂ 1.For each σ ∈ S, σ (h) is a polynomial isomorphic to the polynomial F n,d,k from section 7.3.
2. For any circuit C, σ (C) is a circuit of the same depth as C itself.Moreover, if C is multi-r-ic then so is σ (C).Let first define the set of the restrictions.In fact, we will give in the same time a renaming of the variables which highlights the isomorphism to F n,d,k .For any sequence of n-permutations π π π = (π 1,1 , . . ., π k,2α ) ∈ (S n ) k•2α we will define the restriction σ π π π .We want each restriction σ π π π (h) be a product of k polynomials.Remark 8.3.In the proof from [6], we can notice that the groups of two consecutive constant matrices (last matrix from a block and first matrix of the next block) are merged together.We choose this presentation since it highlights the fact that the restriction is a product of length k.Now, we want to define the restriction for the other matrices.
THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 Finally, let us prove the point 3).We uniformly randomly pick a sequence π π π of n-permutations.Let A and B be two subsets of S n .So, for i = j the events π i ∈ A and π j ∈ B are independent.Let x ⊆ x be the subset of the variables which can be transformed into a z-variable.More formally, x = {x i,a,b | ∃p, q ∈ N s.t.i = pL + q, p < k and (2 ≤ q ≤ α + 1 or α + 3 ≤ q ≤ 2α + 2)}.(8.14)For simplicity, we will also re-index the variables in x = (x i,q,a,b ) 1≤i≤k,1≤q≤2α by skipping the matrices which do not contain x -variables.
x i,q,a,b = x (i−1)L+q+1,a,b if q ≤ α, x (i−1)L+q+2,a,b otherwise.(8.15)Let t be a monomial computed at the first level of the circuit C of x -support larger than τ.Let us define C i,q = {(a, b) | x i,q,a,b is a variable of t}.(8.16) In particular, we have The restrictions σ π π π (t) will be sent to 0 as soon as one of the x -variable x i,q,a,b of t is sent to 0, i. e., as soon as there exists a x -variable x i,q,a,b in t such that π i,q (a) = b.Hence, P π π π [σ π π π (t) = 0] = P π π π π i,q (a) = b | ∀x i,q,a,b ∈ (x ∩ t) The last equality holds since the events are independent.Let us fix i and q, we will prove Claim 8.4.Proof.In the proof, c will denote the cardinality |C i,q |.The inequality is true if c = 0.Moreover, if (a, b 1 ) and (a, b 2 ) are in C i,q with b 1 = b 2 , then π i,q (a) = b 1 or π i,q (a) = b 2 and so P i,q = 0. Hence, we can assume that 1 ≤ c ≤ n.Let C i,q = {(a where the fifth step comes from the fact that the function with 1 ≤ x ≤ n is maximized when x = n. Finally, using Equation (8.17 So, the probability with which a term with x -support larger than τ is still present after the restriction is at most (e/n) τ .By hypothesis, there are at most s terms, so the union bound implies that the probability that the restriction of the circuit does not contain any term with z-support larger than τ is at least 1 − s • (e/n) τ .

Discussion
One motivation behind this work was to generalize the results of [28,30] from r = 1 which corresponds to multilinear formulas to arbitrary r.While we do make some progress towards this goal, the original problem(s) which motivated this work remain open: Open Problem 9.1.Prove super-polynomial lower bounds for constant-depth multi-r-ic formulas (for some explicit family of polynomials).

Corollary 1 . 5 (
Depth-four multi-r-ic formulas computing determinant).For any integer r = r(n), any multi-r-ic ΣΠΣΠ circuit computing Det n must have a size of at least 2 Ω( √ n/r 1.1 ) .

Definition 4 . 5 (
Extremal monomials).Let d = (d 1 , d 2 , . . ., d N ) ∈ Z N ≥0 be an N-tuple and m ∈ F[x] be a multi-d-ic monomial on N variables.We will call m as a d-extremal monomial if the degree of m with respect to any variable x j is either the minimum possible or the maximum possible amount, i. e., deg x j (m) ∈ {0, d j } for all j ∈ [N].

Theorem 4 . 7 ( 18 )Lemma 4 . 8 .
Sperner's Theorem).Let N be an integer and F ⊆ 2[N] be a set of subsets of[N].Such an F is called an antichain if and only if for all distinct I and J in F we have I ⊆ J and J ⊆ I.If F ⊆ 2[N] is an antichain then We use it to first bound the number of extremal monomials in any homogeneous polynomial.Let d ∈ Z N ≥0 be an N-tuple and g(x) ∈ F[x] be a multi-d-ic polynomial.Let N 1 def = |Supp(d)|.If g is homogeneous then the number of d-extremal monomials in g is at most 29) THEORY OF COMPUTING, Volume 14 (16), 2018, pp.1-46 which implies the result.Let us prove the claim.Fix any e-extremal monomial m 1 .Now there is a natural partition induced on the set of variables as follows: Y def = Supp(e) and Z def = (Supp(d v+1 ) \ Supp(e)) = [N] \Y.(4.30)

18 ) 2
2)-form, as required.Proof of Lemma 5.1.Let t = 2rv.Then we have So we can apply Lemma 5.3.We get some (r,t, 2)-balanced multi-r-ic polynomials T 1 , . . ., T s such that s ≤ Size(Φ) and f = T 1 + • • • + T s .By Lemma 5.5 each of these terms T i has a (r, v, 2)-form.Proof of Theorem 1.3.Our target polynomial is the polynomial P N,r defined in Equation (4.38) and used previously in the proof of Theorem 1.2.It has degree d = rN/2.Suppose that it is computed by a homogeneous multi-r-ic formula Φ of product-depth p and size s.Let r = (r, r, . . ., r).By Proposition 3.4, there exists a labelling of the vertices of Φ such that the formula becomes certified multi-r-ic.Let + log N 4 + log r , (5.20)THEORY OF COMPUTING, Volume 14(16), 2018, pp.1-46

Lemma 6 . 3 (
An explicit family with high SkP y -complexity).There is an explicit family of multilinear polynomials { f n,k (x) : n, k ≥ 0} of degree d = 3k on N = (n 2 • k + 2nk) variables such that there exists a partition x = y z with |z| = m = 2nk and |y| = N − m = n 2 k such that

Lemma 7 . 4 (
Upper bound on the SSP ,y,k -complexity of a low support circuit.).Let x = y z be any partition of variables.If C is a multi-r-ic ΣΠΣΠ-circuit of z-support at most τ and if C has top fanin s thenSSP ,y,k (C) ≤ s • 2•m τ k • m + + k • r • τ m ,where m = |z| .(7.4) Proof.By the subadditivity of our measure (Proposition 7.3), it suffices to prove a 2 • m/τ k • m + + k • r • τ m (7.5)

. 7 )
THEORY OF COMPUTING, Volume 14(16), 2018, pp.This yields the following upper bound on the number of factors D in T :

3 .
If C is a depth-four circuit of size s then for any τ ≥ 1, with probability at least p = 1 − s • e n τ over the uniform random choice of σ , (8.9) σ (C) is a depth-four circuit with z-bottom support at most τ.Proof.Let S n be the set of the permutations of [n] and let L = 2 + d/k = 2α + 3.

1 n
More formally, for any matrix M l with• l ≡ 1 mod L, we set x l,a,b = 1 if a = 1
Support and z-support.Let z ⊆ x be a subset of variables and m 3 where d α is the label of the node α.By Proposition 4.3 we have Φ = Ψ • Φ α + Λ where Λ is a multi-d-ic formula of size at most s − 1.Using the inductive hypothesis, it suffices to show that 19))Proof of Lemma 4.8.Let d = (d 1 , d 2 , . . ., d N ).We can first assume that d i > 0 and that deg x i (g) = d i for all i ∈ [N] (otherwise the corresponding variable x i does not appear in any d-extremal monomial in g and we can effectively remove this variable from consideration).Now looking at the support of the extremal monomials in g gives us a collection of subsets of [N].Specifically, for any d-extremal monomial m let: Via Sperner's theorem (Theorem 4.7), it is sufficient to prove that I g is an antichain.If it is not the case, it means that there exist d-extremal monomials m 1 and m 2 in g such that S m 1 is a proper subset of S m 2 .
Claim 4.10.If m 1 is an e-extremal monomial in g, the number of monomials m 2 from g v+1 such that e 1 , . . ., e N ) (4.26) so that T = g • g v+1 and d = e + d v+1 .Now note that a d-extremal monomial in T is a monomial from {e-extremal monomials in g} • {d v+1 -extremal monomials in g v+1 }. (4.27) If Φ is a certified multi-r-ic homogeneous formula of product-depth p and of formal degree d ≥ 2r, then there exists a product node α in Φ such that 22 deg(α) ≥ 2r and for every child β of α it holds that deg β < deg(α)/(4t).Moreover, Φ α is (d α ,t, 2)-balanced.