Faster Change of Order Algorithm for Gröbner Bases under Shape and Stability Assumptions

Solving zero-dimensional polynomial systems using Gröbner bases is usually done by, first, computing a Gröbner basis for the degree reverse lexicographic order, and next computing the lexicographic Gröbner basis with a change of order algorithm. Currently, the change of order now takes a significant part of the whole solving time for many generic instances. Like the fastest known change of order algorithms, this work focuses on the situation where the ideal defined by the system satisfies natural properties which can be recovered in generic coordinates. First, the ideal has a shape lexicographic Gröbner basis. Second, the set of leading terms with respect to the degree reverse lexicographic order has a stability property; in particular, the multiplication matrix can be read on the input Gröbner basis. The current fastest algorithms rely on the sparsity of this matrix. Actually, this sparsity is a consequence of an algebraic structure, which can be exploited to represent the matrix concisely as a univariate polynomial matrix. We show that the Hermite normal form of that matrix yields the sought lexicographic Gröbner basis, under assumptions which cover the shape position case. Under some mild assumption implying n≤t, the arithmetic complexity of our algorithm is O~(tω-1D), where n is the number of variables, t is a sparsity indicator of the aforementioned matrix, D is the degree of the zero-dimensional ideal under consideration, and ω is the exponent of matrix multiplication. This improves upon both state-of-the-art complexity bounds O~(tD2) and O~(Dω, since ω<3 and t≤D. Practical experiments, based on the libraries msolve and PML, confirm the high practical benefit.


INTRODUCTION
Context.A method of choice for solving polynomial systems of dimension zero with coefficients in some field K consists in computing a Gröbner basis for a degree-refining order (such as the degree reverse lexicographic one) using an algorithm such as Buchberger's, or Faugère's F 4 or F 5 algorithms [9,13,14] and next apply a change of order to obtain the lexicographic Gröbner basis, using the FGLM algorithm [16].We refer to [31,46] for an exposition on solving polynomial systems through algebraic methods, and some applications.
In this paper, we consider polynomial systems and ideals in K[ 1 , . . .,  −1 , ] as well as two classical assumptions (which are recalled with more details below).When a zero-dimensional ideal I under consideration is in shape position (see the shape lemma in [21,Lem. 1.4], and Eq.(1) below), the leading monomials of its reduced lexicographical Gröbner basis are  1 , . . .,  −1 ,   where  is the degree of the ideal.This shape position is important for solving polynomial systems, as it reduces multivariate to univariate solving.
The second assumption is a stability one which we also describe in detail below.Roughly, a consequence of this assumption is that the matrix encoding the multiplication by  in the quotient ring K[ 1 , . . .,  −1 , ]/I can be read on the reduced Gröbner basis of I for the degree-refining monomial order.Both the shape position and stability property are satisfied generically and, when the base field is large enough and the ideal under consideration is radical, they can be ensured through a generic linear change of coordinates.When these assumptions hold, Sparse-FGLM variant [17,18] can be applied and is faster than the classical FGLM algorithm for the change of order step.This paper aims at improving this change of order step, under assumptions similar to the ones of Sparse-FGLM, since despite the progress brought by [17,18], this step has become the bottleneck of polynomial system solving with Gröbner bases on a wide range of problems (see [8,Tbl. 1]).
Prior results.The original FGLM algorithm [16] uses  ( 3 ) operations in K without any assumption.We refer to [38,Thm. 1.7] for some improvements around this algorithm.
In [15], the authors consider the special case where ≼ 1 is the degree reverse lexicographic order ≼ drl and ≼ 2 is the lexicographic one ≼ lex , with  ≼ drl   and  ≼ lex   for 1 ≤  ≤  − 1.They show that the aforementioned multiplication matrix by  can be read on the ≼ drl -Gröbner basis, under some genericity assumptions and using results from [35].Assuming additionally the shape position, this multiplication matrix alone suffices to recover the ≼ lex -Gröbner basis, which is done in time ˜(  ) [15].
In [17,18], the authors follow on from the same assumptions: shape and stability.They observe and exploit the sparsity of the multiplication matrix thanks to Wiedemann's approach [48].Precisely, the matrix has about  nonzero entries out of  2 , where the parameter  is the number of polynomials in the ≼ drl -Gröbner basis whose leading monomial is divisible by .This leads to a complexity estimate of  ( 2 ) operations in K, which improves upon ˜(  ) when  is small compared to .This provides significant practical benefit for the change of order step of polynomial system solving in many cases [18,Tbl. 2], and this is the approach used by the state-of-the-art change of order implementation in msolve [8].
Contributions.We push forward the study of the properties of the multiplication matrix  by .Let B be the monomial basis of K[ 1 , . . .,  −1 , ]/I obtained from the reduced ≼ drl -Gröbner basis G of I.Under the stability position assumption, its columns are either unit vectors (for those  ∈ B such that  •  ∈ B) or vectors of coefficients of polynomials in G (for thos  ∈ B such that  •  is a leading monomial of one element in G).The latter ones are usually referred to as a "dense" columns [18,Sec. 5] [44,Sec. 4].This is a well-known matrix structure in K-linear algebra, called a shifted form in [39,40], and studied in particular in the context of the computation of the characteristic polynomial or the Frobenius normal form of a matrix over K (see [45,Sec. 9.1] and Section 3.3).
We exploit the algebraic structure itself and relate it to operations in a K[]-submodule of I. Following a classical construction in [45, Sec.9.1], instead of the multiplication matrix  which is in K × , we consider a univariate polynomial matrix  in K[]  × whose average column degree is /.This polynomial matrix can be seen as a "compression" of , or more precisely of the characteristic matrix   − , with smaller matrix dimension but larger degrees.
Our main result is that, if  is known and the lexicographic Gröbner basis satisfies some assumption which covers the shape position case, then this Gröbner basis can be directly retrieved from the Hermite normal form of  (Theorem 4.1).We also prove that the matrix  can be computed for free from some part of a border basis of I in general (Theorem 5.2) and, as a consequence under the stability assumption, from the input Gröbner basis (Corollary 5.4).Observe that both structural assumptions, of being stable and shape, are used independently.In particular it is expected that in some situations where the stability assumption is not satisfied,  may still be obtained efficiently, and then its Hermite normal form yields the lexicographic Gröbner basis if I is in shape position.
The Hermite normal form can be computed deterministically in ˜( −1 ) operations in K [32], which dominates the overall complexity of the change of order.Theorem 1.1.Let I ⊂ R = K[ 1 , . . .,  −1 , ] be a zero-dimensional ideal of degree .Let G ≼ drl (resp.G ≼ lex ) be the reduced ≼ drl -(resp.≼ lex -)Gröbner basis of I and B be the ≼ drl -monomial basis of R/I.Assume that  1 , . . .,  −1 are in B, and that for all monomials  ∈ B, either  •  is in B or it is the ≼ drl -leading monomial of an element in G ≼ drl .Assume that I is in shape position.Then, one can compute G lex using ˜( −1 ) operations in K, where  is the number of elements of G ≼ drl whose ≼ drl -leading monomial is divisible by .
Compared to the previous  ( 2 ), the speed-up factor is of the order of  2− .We give explicit complexity gains for families of polynomial systems for which closed formulas or asymptotic estimates for  and  are known [7,18].
We study the practical performance of the new approach.For this, we designed an efficient implementation of the Hermite normal form, which follows the approach of [32] but tailored to the matrices  encountered here, which have specific degree shapes.This implementation relies on the Polynomial Matrix Library [28] (PML) and on NTL [43].We show that it outperforms both the existing change of order algorithm in the current version of msolve [8], and an implementation of a block-Wiedemann approach in NTL.
Structure of the paper.Section 2 is devoted to preliminaries and detailed definitions of the shape position and stability assumptions.Section 3 shows how the aforementioned univariate polynomial matrix  can be obtained from a module-theoretic perspective.Section 4 establishes the connexion between the reduced ≼ lex -Gröbner basis of I and the Hermite normal form of .Section 5 shows how to compute  from a Gröbner basis.Finally, in Section 6, we discuss complexity results and report on practical performance.

NOTATION AND PRELIMINARIES
Consider the polynomial ring R = K[ 1 , . . .,  −1 , ].For a nonzero polynomial  ∈ R \ {0}, the support of  , denoted by supp( ), is the collection of all monomials appearing in  , with a nonzero coefficient.For a set of polynomials  ⊂ R, the ideal generated by Monomial orders, normal forms.For the definition of a monomial order ≼ on R, we refer to [10, Chap.2, §2].We recall that ≼ is a total order on the set of monomials, and we write ≺ for the corresponding strict order.Here, monomial orders are such that We will use the lexicographic order ≼ lex , and the degree reverse lexicographic order ≼ drl .We denote by lt ≼ ( ) the ≼-leading term of a nonzero polynomial  ∈ R, and by lt ≼ () the set of ≼-leading terms of all nonzero elements of a set  ⊆ R.
For a monomial order ≼ and an ideal I ⊂ R, consider the set B of monomials in R that are not in the ideal of leading terms ⟨lt ≼ (I)⟩.This set B is called the ≼-monomial basis of R/I: it is a basis of R/I as a K-vector space [5, Prop.6.52].For a polynomial  ∈ R, the ≼-normal form of  with respect to I, denoted by nf ≼,I ( ), is the unique polynomial whose support is in B and such that  − nf ≼,I ( ) ∈ I.
Gröbner bases, shape position.For the notion of (reduced) ≼-Gröbner bases of ideals in R, we refer to [10,Chap. 2].By definition, for a ≼-Gröbner basis G of I, we have G ⊂ I and ⟨lt ≼ (G)⟩ = ⟨lt ≼ (I)⟩ and the ≼-monomial basis B is also the set of monomials that are not multiples of an element in lt ≼ (G).
A proper ideal I ⊂ R is zero-dimensional [5,Def. 6.46] if and only if R/I has finite dimension  as a K-vector space [5,Thm. 6.54].In that case, following [4], I is said to be in shape position if its reduced ≼ lex -Gröbner basis has the form where Stability assumption.This assumption, mentioned in Section 1, concerning the stability of the ideal of ≼ drl -leading terms of I, is defined as follows.
Definition 2.1.For a set  of monomials in R, the statement S() is: "for any monomial  ∈  such that  divides , the monomial     belongs to  for all  ∈ {1, . . .,  − 1}".This is directly related to classical notions of stability of sets of monomials and of monomial ideals [26, Sec.4.2.2, 6.3 and 7.2.2], which arise notably through the Borel-fixedness of generic initial ideals [3,20].The next lemma states that when considering the monomials in a monomial ideal, the above statement holds if, and only if, it holds for the minimal generating monomials of that ideal.Lemma 2.2.Let J be a monomial ideal of R, and { 1 , . . .,   } be its minimal generating set.Let  be the set of monomials in J .Then, S() is equivalent to S( 1 , . . .,   ).
We do not prove this, as this is a direct consequence of [38,Lem. 2.2].For our purpose, we are mostly interested in the case J = lt ≼ (I) for some monomial order ≼ and some ideal I.The above lemma shows that, if lt ≼ (I) is known (for example via the ≼-leading terms of a ≼-Gröbner basis), then it is straightforward to check whether S(lt ≼ (I)) holds.
Example 2.3.Consider the ideal I of F 29 [ 1 ,  2 , ] generated by Observe that it has  = 3 polynomials whose ≼ drl -leading terms, in boldface font, are multiples of .The ≼ drl -monomial basis B of R/I is the set of monomials not in Finally, we verify that the stability property S(lt ≼ drl (I)) holds.As noted in Lemma 2.2, it is sufficient to check that for each minimal generator  of ⟨lt ≼ drl (I)⟩ such that  divides , the monomials  1   and  2   remain in lt ≼ drl (I).Thus we consider  ∈ { 2  2 ,  1  2 ,  4 }, and it is easily verified that The algorithmic approach in this paper makes use of a K[]-module denoted by M T,I , which is defined from an ideal I and a set of monomials T .Using the module addition of K[ 1 , . . .,  −1 , ], this module is a subset of I, which is sufficient to recover the ≼ lex -Gröbner basis of I in the shape position case, and which allows us to benefit from efficient algorithms for matrices over K[].

General definitions and properties
See [11,Chap. 10] for general definitions and properties of modules.Roughly, free modules are those which admit a basis, and since is commutative, all bases of a free K[]-module have the same cardinality which is called the rank of the module [11,Sec. 10.3].
For modules over a principal ideal domain such as K[], we refer to [11,Chap. 12].In particular, if N is a free K[]-module of rank  ∈ N and M is a K[]-submodule of N , then M is free and its rank  is at most  [11, Sec.12.1, Thm.4].As a result, M has a basis of cardinality , which can be represented as a matrix in K[] × .This matrix has full row rank, and its rows are the basis elements.Furthermore M has a unique basis in a specific form, at the core of this work: the Hermite normal form [25,29].When  = , a matrix •  is lower triangular; • the diagonal entries of  are monic; • in each column of , the diagonal entry has degree greater than the other entries, i.e. deg(   ) < deg(   ) for  ≠ .A typical example of ambient module is N = K[]  .Here we will also consider the K[]-module N = R T ⊂ R, defined as follows.Let T = ( 1 , . . .,   ) be a list of pairwise distinct monomials in K[ 1 , . . .,  −1 ], and consider the set of monomials of -multiples of a monomial in T .Then we define which is a free K[]-module of rank , with basis given by T .
Hereafter, for a finite set of polynomials  ⊂ R, the K[]-module generated by  will be denoted by ⎷⌄.

A module associated to the ideal
Consider the K[]-module R T as in Section 2, for some pairwise distinct monomials T = ( 1 , . . .,   ) in K[ 1 , . . .,  −1 ].This module is free of rank , with basis T .Then, for any ideal I of R, let By construction, we have the inclusion of ideals ⟨M T,I ⟩ ⊆ I.
Example 3.2.Let T be as in Example 3.1.Then M T,I is the set of polynomials in I which have degree at most 1 in each of the variables  −1 , . . .,  1 , and • for a zero-dimensional ideal I, if I is in shape position then ⟨M T,I ⟩ = I (see Lemma 5.1).□ The case of equality ⟨M T,I ⟩ = I is of particular interest: it ensures that no information is lost when restricting to polynomials with monomial support in T * .Our aim is to compute objects related to I, such as its ≼ lex -Gröbner basis, using only computations in the smaller submodule M T,I .The motivation behind this idea is that many efficient tools are known for computing with M T,I , thanks to the matrix representation explained below.
As seen in Section 3.1, as a K[]-submodule of R T , M T,I is free of rank , with  ≤ , and any basis of M T,I is a collection of  polynomials { 1 , . . .,   } ⊂ R T .Such a basis can be represented as a matrix of rank , whose row  is formed by the univariate polynomials Example 3.3 (following on from Example 2.3).Take T as the set of monomials in B which are not multiples of , that is, T = (1,  2 ,  1 ); observe that the cardinality of T is  = 3.As noted in Example 3.1, M T,I is then the set of polynomials in I which have degree at most 1 in  1 and in  2 .This is the case for 3 polynomials of G drl : Hence these polynomials are in M T,I ; note they are exactly the polynomials of G drl whose ≼ drl -leading terms are multiples of .In Section 5 we will prove that, since S(lt ≼ drl (I)) is satisfied (see Example 2.3), these polynomials form a basis of M T,I .
Representing these polynomials on the basis T of R T , we obtain the following matrix in Note that this matrix is directly read off from G drl .□ We end this section by showing that if I is zero-dimensional, then the bases  ∈ K[] × of M T,I are square, nonsingular matrices.This is implied by the first item of the following lemma, thanks to the fact that a zero-dimensional ideal contains a univariate polynomial in each variable [5,Lem. 6.50].For completeness, we also give a partial converse property in the second item.Proof.We already observed that  ≤ .First item: assuming the existence of ℎ, the set R T = {ℎ |  ∈ R T } is a K[]-module of rank , having ℎT as a basis.Since ℎ ∈ I, ℎR T is contained in M T,I , which implies  ≤  as recalled in Section 2. Hence  = .Second item: assuming  = , we define  ∈ K[]  × as in Eq. ( 4), from a basis { 1 , . . .,   } ⊂ R T of M T,I .Then, we let ℎ = det() ∈ K[], which is nonzero since  is nonsingular.By assumption, there exists  ∈ {1, . . .,  } such that   = 1.Then, by Cramer's rule, there are  1 , . . ., The choice of ordering of B makes the following structure obvious: this matrix has companion blocks on the diagonal, and its other blocks have zeroes everywhere but possibly on the last row.Note how the basis  from Example 3.3 can be built by replacing each block by a single polynomial in K[] (recall here K = F 29 ): • companion blocks are replaced by their respective characteristic polynomials, for example the first companion block becomes  4 − (26 3 + 26 2 + 7) =  4 + 3 3 + 3 2 + 22; • other blocks by are replaced by the opposite of the polynomial given by the last row, for example the block (3, 1) yields Both this type of structure for matrices over a field and the corresponding compact representation as univariate polynomial matrices have been studied, in particular concerning questions of matrix similarity.For example, the Frobenius normal form of  corresponds to the Smith normal form of  [45, Thm.9.1], whereas the shifted Hessenberg form of  corresponds to the Hermite normal form of  [45, Thm.9.5 and Lem.9.7].More recently, such matrix structures were instrumental in the design of fast algorithms for the Frobenius normal form of a matrix over a field [39,40].
However, to our knowledge, in the context of Gröbner basis change of order, this structure of the multiplication matrix had only been exploited through the sparsity it brings, in order to rely on (block-)Wiedemann techniques [18,27,44].

RETRIEVING LEXICOGRAPHIC GRÖBNER BASES FROM HERMITE NORMAL FORMS
From a matrix  as in Eq. ( 4), whose rows in K[] 1× represent a basis of M T,I , one can compute the reduced Gröbner basis of M T,I with respect to a chosen monomial order on K[] 1× ; see [12,Chap. 15] for Gröbner bases of submodules of a free module with basis.Here this ambient free module is R T ≃ K[] 1× , with K[] univariate: specific terminology and computational tools exist.In particular, classical reduced Gröbner bases are the Hermite normal form [25] (corresponding to the position-over-term order [30]), the Popov normal form [42] (corresponding to the term-over-position order [30]), and shifted variants of the latter [6] (corresponding to term-over-position orders with weights [36,Sec. 1.3.4]).The definition of Hermite normal forms was given in Section 2. However, these Gröbner bases of the submodule M T,I do not necessarily correspond to Gröbner bases of the ideal I, even when ⟨M T,I ⟩ = I.The next result states that, under the stability assumption, there is a correspondence between the lexicographic Gröbner basis of I and the basis of M T,I in Hermite normal form.(A link of this kind is not new [33,Sec. 5] [47,Sec. 7], yet we were not able to find a statement similar to the next one in the literature.)Assuming G lex ⊆ R T , then G lex can be read off from the rows of  .Explicitly, let  be an element of G lex and let  be the unique integer in {1, . . .,  } such that lm ≼ lex ( ) =     for some  ≥ 0. Then the th row of  has the form Proof.In this proof, ≼ stands for the lexicographic order ≼ lex .Let  be an element of G lex .Since G lex ⊆ R T , every monomial of  belongs to T * = {    | 1 ≤  ≤ ,  ≥ 0}.In particular, lm ≼ ( ) =     for some  in {1, . . .,  } and  ≥ 0 (and  is unique since the   's are pairwise distinct).
Since  ∈ R T , and T is a basis of R T as a K[]-module, there is a unique Let  ∈ { + 1, . . .,  }.We are going to prove   = 0. Recall that  ≺   for 1 ≤  ≤ −1, and that the monomial   only involves the variables  1 , . . .,  −1 .Besides, T being sorted increasingly ensures We first show that the th diagonal entry of  has degree  = deg(  ).On the one hand, the th row of  corresponds to a nonzero polynomial in I whose ≼-leading term is     .Then, having  <  would mean that lt ≼ ( ) is a strict multiple of the ≼-leading term of some element of I, which contradicts the definition of G lex .Thus  ≥ .On the other hand,  is in M T,I and therefore corresponds to a vector in the K[]-row space of  whose rightmost nonzero entry is at index .Then, the triangularity of  implies that for some  1 , . . .,   ∈ K[] and   ≠ 0. Using the triangularity again, we obtain  = deg(  ) = deg(  ) + , hence  ≤ .This yields  = .
Let  1 , . . .,  −1 ∈ N be the degrees of the first  − 1 diagonal entries of  .In this paragraph we show that, to conclude the proof, it is enough to prove deg(  ) <   for 1 ≤  < .Indeed, as seen above, the vector -row space of  , and has rightmost nonzero entry   at index , which has the same degree as the th diagonal entry of  and is monic by definition of a reduced ≼-Gröbner basis.Thus, if deg(  ) <   for 1 ≤  < , then this vector must be equal to the th row of  , by uniqueness of the Hermite normal form: otherwise one could replace the th row of  by this vector and get a different Hermite normal form for the same K[]-module.
Let ) is known as well, which is the case under the shape position assumption, then Theorem 4.1 indicates precisely which rows of  give the polynomials of G lex , without any further computation.Remark 4.3.Even when lt ≼ lex (G lex ) is unknown, G lex is easily found from  .Indeed,  yields polynomials ℎ 1 , . . ., ℎ  in M T,I ⊆ I, and Theorem 4.1 ensures that they include the polynomials of G lex .Thus {ℎ 1 , . . ., ℎ  } is a ≼ lex -Gröbner basis of I, and filtering out from it the polynomials which are not in G lex is easily done and computationally cheap, by following the classical procedure for transforming a non-minimal Gröbner basis into a minimal one.Explicitly: Note that the uniqueness of the indices  1 , . . .,   is ensured by the fact that  1 , . . .,   are pairwise distinct by construction.

CONSTRUCTING A BASIS OF THE MODULE FROM A KNOWN GRÖBNER BASIS
There are two missing ingredients in order to use the above framework to compute G lex .First, the assumptions of Theorem 4.1 must be satisfied.Second, we need an efficient method to compute the basis  of M T,I in Hermite normal form; for this, known methods require the knowledge of some basis  ∈ K[]  × of M T,I .
The assumption that I is zero-dimensional will be guaranteed from our context.The main constraint is therefore the choice of T in order to ensure that G lex ⊂ R T is satisfied.This relates to the more general equality ⟨M T,I ⟩ = I, via the following characterization: ⟨M T,I ⟩ = I if and only if there exists a generating set of I formed by polynomials in R T .Obviously, taking T large enough ensures G lex ⊂ R T ; yet a larger set T also means a larger matrix dimension  and thus more expensive computations to find  and deduce  .Now, focusing on the shape position case as explained in Section 1, all monomials occurring in G lex are either in { −1 , . . .,  1 } or in {  |  ≥ 0}.Hence the following lemma.Therefore, in the shape position case, the condition G lex ⊆ R T is easily satisfied, and the main missing ingredient is an efficient method for computing .The next theorem shows that, for any monomial order ≼, the knowledge of some ≼-border basis of I [34] directly provides a suitable set T and a corresponding basis  ∈ K[]  × of M T,I .Furthermore, this matrix  has a particular degree pattern related to the ≼-monomial basis of I.
In Corollary 5.4 we deduce that, under the stability assumption S(lt ≼ (I)), the knowledge of the reduced ≼-Gröbner basis of I is enough to find T and .Then, it will only remain to find the Hermite normal form of , which is the sought basis  : the efficient computation of  from  is discussed in Section 6.1.Theorem 5.2.Let ≼ be a monomial order such that  ≺   for 1 ≤  ≤  − 1.Let I be a zero-dimensional ideal in R and let B be the ≼-monomial basis of R/I.Let T = ( 1 , . . .,   ) be the monomials in B which are not divisible by , i.e.
Proof.First note that, since I is zero-dimensional, B is finite and therefore T is finite as well.Concerning the identity in Eq. ( 5) we first observe that, since B is finite and is the complement of lt ≼ (I), for each  ∈ {1, . . .,  } there is a unique  ∈ Z >0 such that  −1   ∈ B and     ∉ B. Conversely, for any  ∈ B such that  ∉ B, the integer  = 1 + max{  ∈ N |   divides } satisfies  1−  ∈ T , hence  =  −1   for some .This shows Eq. ( 5).Now, concerning P, its elements are in I by definition of the ≼-normal form (see Section 2), hence P ⊆ M T,I .Furthermore P has cardinality , which is the cardinality of T and therefore the rank of M T,I (see Lemma 3.4).Thus, to prove that P is a basis of M T,I it is sufficient to show that any polynomial  in M T,I is a R-linear combination of P, that is,  ∈ ⎷P⌄.Since  ∈ M T,I , On the other hand, as showed in Lemma 5.3,    +   −  , ∈ ⎷P⌄ for some  , ∈ Span K (B), for all 1 ≤  ≤  and  ∈ N. Altogether, this implies that  =  + , for some  ∈ Span K (B) and  ∈ ⎷P⌄.Since ⎷P⌄ ⊆ M T,I , we have  −  ∈ M T,I ⊆ I and therefore Finally, consider the matrix representation  of P. As seen in Section 3.2, the th row of  is the vector Proof.We prove this by induction on , noting that this property holds for  = 0 by definition of P. Now, consider  ∈ Z >0 and suppose the property holds for all integers up to  − 1.Let  ∈ {1, . . .,  }.By induction hypothesis there exists  ∈ ⎷P⌄ such that It remains to prove that  ,−1 is the sum of an element of ⎷P⌄ and one of Span K (B) (the latter must then be  , by uniqueness).This follows from the facts that and that the elements of P are {     −  ,0 | 1 ≤  ≤  } with  ,0 ∈ Span K (B); indeed these imply more precisely that  ,−1 is the sum of an element of Span K (P) and one of Span K (B).□ Corollary 5.4.Using notation from Theorem 5.2, assume further S(lt ≼ (I)), let G be the reduced ≼-Gröbner basis of I, and let  1 , . . .,   be the elements of G whose ≼-leading term is divisible by .Then { 1 , . . .,   } is a basis of M T,I as a K[]-module.

COMPLEXITY AND PERFORMANCE 6.1 Hermite normal form computation
We assume that the basis  ∈ K[]  × from Theorem 5.2 is known, and we review the computation of its Hermite normal form.This subsection ends with a proof of Theorem 1.1.
Reducing to average degree, and general algorithms.By Theorem 5.2, the matrix  has column degrees ( 1 , . . .,   ), and  = Hence  has average column degree   .Then, its Hermite normal form  can be found deterministically in Besides, it is showed in [32,Sec. 6] that computing  directly reduces to computing the Hermite normal form of a matrix which is built from  and has slightly larger size but with all entries of degree at most ⌈   ⌉.Since  ≤  here, ⌈   ⌉ ∈  (   ), hence the same cost ˜(    ) = ˜( −1 ) is obtained by the Las Vegas randomized algorithm in [22,24].Observe that, in both cases, the number of logarithmic factors in the cost bound is currently unknown.
In particular, we know the degree shape of the sought Hermite normal form.Finding these degrees is the first step of the fastest known Hermite normal form algorithm [32,Sec. 3], which can therefore be omitted: we directly use the second step in [32,Sec. 5].The advantage is that the latter boils down to one call to a row reduction algorithm, for which the cost bound is known including logarithmic factors: it is  (  M(   )(log() 2 + log(   ))), if one uses the fastest known deterministic algorithm [23,Thm. 18].Here, M(•) is a time function for the multiplication of univariate polynomials in K[], with usual assumptions recalled for example in [23,Sec. 2].
Observe that one may still follow this approach when it is unknown whether the ideal is in shape position.If the obtained matrix does not have the expected form described in Eq. ( 6), then the ideal is not in shape position, and one can restart computations using a more general, slower change of order algorithm.
Our implementation, on which we report in Section 6.3, is based on this approach.The advantage is that this uses a single call to the fast kernel basis algorithm of [50], for which a precise cost estimate is known.After the multiplication, whose cost is also well understood, we are left with the computation of a Hermite normal form of an  ×  matrix.In most interesting instances, this has negligible cost, since  ≪  (see for example Sections 6.2 and 6.3).
Here again, one does not have to assume that the ideal is in shape position: this can be detected from the degrees in , which in fact can be predicted from the degrees in the kernel basis .In the case where the degrees in  reveal that the ideal is not in shape position, one could switch to another more general method.
Proof of Theorem 1.1.Since S(lt ≼ (I)) is assumed (via Lemma 2.2), Corollary 5.4 ensures that the set of  elements of G drl whose ≼ drlleading term is divisible by  forms a basis  ∈ K[]  × of M T,I as a K[]-module, where T is built as in Theorem 5.2.
Besides, since the variables  1 , . . .,  −1 are in the ≼ drl -monomial basis B, they belong to T , which implies G lex ⊆ R T according to Lemma 5.1, since I is in shape position.Hence Theorem 4.1 states that the Hermite normal form  of  yields the sought lexicographical Gröbner basis.As we have seen above, computing  from  takes  ∼ ( −1 ) operations in K.

Extrinsic asymptotics of complexity gains
The estimate  ∼ ( −1 ) of Theorem 1.1 depends on the sparsity indicator  which is less than the degree  of the ideal.These are intrinsic to the ideal and the monomial order.On several important classes of problems, the asymptotics of  and  can be expressed as a function of the number of variables , the maximum degree of the input polynomials  and some other extraneous parameters.These results can then be used to make explicit the speed up  2−  we obtain from the complexity  ( 2 ) of [17,18].
These are given in Table 1.For generic systems of  polynomials of degree , [18, Tbl.2] provides such asymptotics ; we refer to these in the line (rand , ).In [7, Tbl.1] asymptotic values of ,  are given for systems defining the critical points of the restriction of a linear map to an algebraic set defined by  generic polynomials by means of the simultaneous vanishing of maximal minors of a truncated Jacobian matrix.We refer to these in the line (crit , , ).More recently, [19, Tbl.1], provides asymptotic estimates for ,  when considering the ( + 1)-minors of a polynomial symmetric matrix of size  in  = − +1 2 variables (symdet , , ,  ).For all these systems, the asymptotics of  2−  appear with a positive exponent of quantities greater than 1, showing that the new algorithm is asymptotically faster than Sparse-FGLM.Note that for 2 <  < 3, the complexity gain is exponential in  for most of them, in particular for rand ,  and crit , , .

Practical performance
We compare our implementation of the new change of order algorithm with two other algorithms: • The state-of-the-art implementation of the Sparse-FGLM algorithm [17,18], provided by msolve [8].This is based on the Wiedemann algorithm, with the core computational task consisting of a series of matrix-vector products.• A prototype implementation of a block-Wiedemann variant of Sparse-FGLM, whose core computational task consists of a series of matrix-matrix products.For the sake of comparison with our prototype PML/NTL implementation of the new algorithm, this was written with NTL using the linear algebra tools provided by its Mat<zz_p> module.Both implementations exploit the structure of the multiplication matrix of  in R/I written on the ≼ drl -monomial basis, as explained in Section 3.3.In this context, the expected advantage of the block variant comes from the greater efficiency of performing a single matrix-matrix product  • [ 1 • • •   ] versus performing several matrix-vector products   for 1 ≤  ≤ .
In our experiments on the block-Wiedemann approach, a blocksize  in the range between 64 and 128 appeared as a good compromise.When  is below 64, the benefit from matrix-matrix products remains limited.On the other hand, when  is above 128, although there could still be some gain by further increasing the matrix dimension, this is counter-balanced by the cost of the second step which starts to be non-negligible.This second step is a matrix fraction reconstruction, performed via an approximant basis of a (2) ×  matrix at order 2/, for which we used PML [28].
Note that, although block-Wiedemann approaches are often used for benefiting from multi-threaded or parallel computations, here only single-threaded performance is considered, and we keep the design of an optimized, multi-threaded implementation of our new change of order algorithm as a future perspective.Indeed, we expect it to also benefit from multi-threading, since the dominant part of its computations consists of multiplication and Gaussian elimination of large-dimension matrices over K.
We summarize our comparison in Table 2.All computations were performed on a single thread on a computer equipped with Intel ® Xeon ® Gold CPU 6246R v4 @ 3.40GHz and 1.5TB of RAM.
The base field is K = Z/Z with a 30-bit prime modulus .This choice comes from the fact that many application areas require Gröbner bases computations over large fields.This is the case for problems in multivariate cryptography and number theory [1,2,41].Furthermore, large computations over the rationals K = Q boil down to solving several instances over K = Z/Z, through the Chinese Remainder Theorem.We consider the 30-bit prime fields as a base case in this setting, since it allows us both to choose sufficiently many primes for large instances, and to avoid bad primes with higher probability than e.g.16-bit prime fields.It also seems to be the base case used in computer algebra software like Macaulay2, Maple and Singular and in the state-of-the-art change of order implementation in msolve which we compare to.
We observe that our implementation of the new algorithm is always faster than both other implementations, and that the gap is increasing with the size of the instances.For large instances, the speed-up factor is close to 5.
Let us notice that the block-Wiedemann approach also outperforms msolve for large sizes, as might be expected, yet only by a very small margin.One explanation can be that NTL does not seem to use AVX2 vectorization techniques for matrix multiplication over a 30-bit prime field, whereas msolve does for its matrix-vector products.Investigating this is a future perspective, and incorporating AVX2 may lead to further accelerations for the block-Wiedemann approach, but also for the new algorithm which makes an intensive use of the multiplication of matrices over K when multiplying univariate polynomial matrices.
Let us recall that when computing over the rationals, the common strategy through Chinese Remainder Theorem is to use F 4 with a tracer for the computation of G drl modulo each prime: perform a full F 4 algorithm modulo the first prime and learn which polynomials are used in the construction of each matrix and remove those that reduce to 0, and those used only in these reductions to 0. This allows one to minimize the computations modulo the subsequent primes.As a practical consequence, FGLM used to be slower than F 4 -tracer, but this is not the case anymore, we have reestablished a kind of balance.With the above perspective we expect the change of order step to be consistently faster than the F 4 -tracer step.Furthermore, even when computing over a 30-bit prime field, with only F 4 and no tracer, FGLM could take more than 25% of the total time, sometimes even close to 40% (see rand (4,7) or (4, 8)), now it is closer to negligible (often below or close to 10%).
Table 2: Timings in seconds for random square systems in  variables and degree , over a prime field Z/Z with a 30-bit modulus.F 4 is the algorithm of [13]." F 4 -tr" is the tracer algorithm after the initial learning phase [8,Sec. 4.3]."Wied." and "bl-Wied." are the change of order comparison points described in Section 6.3.They use respectively the Wiedemann-based Sparse-FGLM algorithm [18] and a folklore block-Wiedemann variant (see e.g.[27,44])."HNF" is the new algorithm, based on Hermite normal form, as described in Section 6.1.

Lemma 3 . 4 .
With the above notation, • If there exists a nonzero univariate ℎ ∈ I ∩ K[], then M T,I has rank  =  as a K[]-module.• If M T,I has rank  =  as a K[]-module and 1 ∈ T , then there exists a nonzero univariate ℎ ∈ I ∩ K[].

Theorem 4 . 1 .
Let I be a zero-dimensional ideal of R and let G lex be the reduced ≼ lex -Gröbner basis of I. Let T = ( 1 , . . .,   ) be pairwise distinct monomials in K[ 1 , . . .,  −1 ], sorted increasingly according to ≼ lex .Define R T and M T,I as in Eqs.(2) and (3).Let  ∈ K[]  × be the basis of M T,I in Hermite normal form.