Watersheding hierarchies

. The computation of hierarchies of partitions from the watershed transform is a well-established segmentation technique in mathematical morphology. In this article, we introduce the watersheding operator , which maps any hierarchy into a hierarchical watershed. The hierarchical watersheds are the only hierarchies that remain unchanged under the action of this operator. After deﬁning the watersheding operator, we present its main properties, namely its relation with extinction values and sequences of minima of weighted graphs. Finally, we discuss practical applications of the watersheding operator.


Introduction
In the context of image segmentation, hierarchies (of partitions) are sequences of nested partitions of image pixels.At the highest levels of a hierarchy, we have the most representative regions according to a given criterion, such as size and contrast.Hierarchies can be equivalently represented by saliency maps [1,6,18], in which the contours between regions are weighted according to their level of disappearance in the hierarchy.Thank to the bijection between hierarchies and saliency maps [6], we work indifferently with any of those notions in this study.
In mathematical morphology, hierarchies are often obtained from the watershed transform [4,2].With the definitions proposed in [4], hierarchical watersheds are optimal in the sense of minimum spanning forests.Furthermore, efficient algorithms to compute those hierarchies have been designed [5,17].The competitive performance of hierarchical watersheds is attested by the quantitative evaluation presented in [19].Moreover, watersheds can be linked to a broader family of combinatorial optimization problems, such as random walkers and graph cuts, as demonstrated in [3].
In this article, we propose the watersheding operator, which, given an edgeweighted graph ((V, E), w), transforms (the saliency map of) any hierarchy on V into (the saliency map of) a hierarchical watershed of ((V, E), w). Figure 1 illustrates an application of the watersheding operator.The goal is to obtain, from the image gradient G, a hierarchical watershed of G that highlights the circular regions of the image I.A straightforward method to achieve this goal is to compute the hierarchical watershed of G based on a (regularized) circularity attribute, which results in the hierarchy H c .We can observe that the hierarchy H c I G Hc Hcc Hw Fig. 1: An image I, the gradient G of I computed using the structured edge detector introduced in [8], the hierarchical watershed H c of G based on a regularized circularity attribute, a circularity based hierarchy H cc and the watersheding H w of H cc (for G).
does not succeed at highlighting all the main circular regions of the image I.
Alternatively, we can compute another hierarchy H cc by simply weighing each contour as the maximum circularity value among the regions that share this contour.The hierarchy H cc brings to the fore the main circular regions of I, however it is not a hierarchical watershed of G.By applying the watersheding operator on H cc , we obtain the hierarchical watershed H w .We can see that the circular regions of the image I are better highlighted in the result of the watersheding operator H w when compared to the straightforward approach H c .This article is organized as follows.We review basic notions on graphs, hierarchies and saliency maps in section 2.Then, we present the watersheding operator and some properties of this operator in section 3. Finally, we discuss applications of the watersheding of hierarchies in section 4.

Background notions
In this section, we first introduce hierarchies of partitions.Then, we review the definition of graphs, connected hierarchies and saliency maps.Subsequently, we define hierarchical watersheds.

Hierarchies of partitions
Let V be a set.A partition of V is a set P of non empty disjoint subsets of V whose union is V .If P is a partition of V , any element of P is called a region of P. Let V be a set and let P 1 and P 2 be two partitions of V .We say that P 1 is a refinement of P 2 if every element of P 1 is included in an element of P 2 .A hierarchy (of partitions on V ) is a sequence H = (P 0 , . . ., P n ) of partitions of V such that P 0 is an arbitrary partition of V , the partition P n is the set {V } and P i−1 is a refinement of P i , for any i ∈ {1, . . ., n}.For any i in {0, . . ., n}, any region of the partition P i is called a region of H.
Hierarchies of partitions can be represented as trees whose nodes correspond to regions, as shown in Figure 2. Given a hierarchy H and two regions X and Y of H, we say that X is a parent of Y (or that Y is a child of X) if Y ⊂ X and X is minimal for this property.In other words, if X is a parent of Y and if there is a region Z such that Y ⊆ Z ⊂ X, then Y = Z.It can be seen that any Fig. 2: A representation of a hierarchy of partitions H = (P 0 , P 1 , P 2 , P 3 ) on the set {a, b, c, d, e, f, g, h} and a weighted graph (G, w).
region X of H such that X = V has exactly one parent.Thus, for any region X such that X = V , we write parent(X) = Y where Y is the unique parent of X.
For any region R of H, if R is not the parent of any region of H, we say that R is a leaf region of H. Otherwise, we say that R is a non-leaf region of H.
In Figure 2, the regions of a hierarchy H are linked to their parents and children by straight lines.The partition P 0 of H contains all the leaf regions of H.

Graphs, connected hierarchies and saliency maps
A graph is a pair G = (V, E), where V is a finite set and E is a set of pairs of distinct elements of V , i.e., E ⊆ {{x, y} ⊆ V |x = y}.Each element of V is called a vertex (of G), and each element of E is called an edge (of G).To simplify the notations, the set of vertices and edges of a graph G will be also denoted by V (G) and E(G), respectively.
Let G = (V, E) be a graph and let X be a subset of V .A sequence π = (x 0 , . . ., x n ) of elements of X is a path (in X) from x 0 to x n if {x i−1 , x i } is an edge of G for any i in {1, . . ., n}.The subset X of V is said to be connected for G if, for any x and y in X, there exists a path from x to y.The subset X is a connected component of G if X is connected and if, for any connected subset Y of V such that X ⊆ Y , we have X = Y .In the following, we denote by CC(G) the set of all connected components of G.It is well known that this set CC(G) of all connected components of G is a partition of the set V .
Let G = (V, E) be a graph.A partition of V is connected for G if each of its regions is connected for G and a hierarchy on V is connected (for G) if each of its partitions is connected.For example, in Figure 2, the hierarchy H is connected for the graph (G, w).
Let G be a graph.If w is a map from the edge set of G to the set R + of positive real numbers, then the pair (G, w) is called an (edge) weighted graph.If (G, w) is a weighted graph, for any edge u of G, the value w(u) is called the weight of u (for w).
As established in [6], a connected hierarchy can be equivalently treated by means of a weighted graph through the notion of a saliency map.Given a graph G and a hierarchy H = (P 0 , . . ., P n ) connected for G, the saliency map of H (also known as ultrametric watershed [16] and ultrametric contour map [1]) is the map from E(G) to {0, . . ., n}, denoted by Φ(H), such that, for any edge u = {x, y} in E(G), the value Φ(H)(u) is the smallest value i in {0, . . ., n} such that x and y belong to a same region of P i .Therefore, the weight Φ(H)(u) of any edge u = {x, y} is the ultrametric distance between x and y on the hierarchy H.It follows that any connected hierarchy has a unique saliency map.Moreover, we can recover any hierarchy H connected for G from the saliency map Φ(H) of H through the notion of quasi-flat zones [15,13], as established in [6].For instance, in Figure 2, the weight map w is the saliency map of the hierarchy H.

Hierarchies of minimum spanning forests and watersheds
The watershed segmentation [4,2] derives from the topographic notion of watersheds lines and catchment basins.A catchment basin is a region whose collected precipitation drains to the same body of water, as a sea, and the watershed lines are the separating lines between neighbouring catchment basins.In [4], the authors formalize watersheds in the framework of weighted graphs and show the optimality of watersheds in the sense of minimum spanning forests.In this section, we present hierarchical watersheds following the definition of hierarchies of minimum spanning forests presented in [7].
Let G be a graph.We say that G is a forest if, for any edge u in E(G), the number of connected components of the graph 1. the graphs G and G have the same set of vertices, i.e., V (G ) = V (G); and 2. the graph G is a subgraph of G ; and 3. each connected component of G includes exactly one connected component of G ; and 4. the sum of the weight of the edges of G is minimal among all graphs for which the above conditions 1, 2 and 3 hold true.
Important notations and notions: in the sequel of this article, the symbol G denotes a tree, i.e., a forest with a unique connected component.This implies that any map from E(G) into Z + is the saliency map of a hierarchy which is connected for G, as established by Properties 9 and 10 of [6].To shorten the notation, the vertex set of G is denoted by V and its edge set is denoted by E. The symbol w denotes a map from E into R + such that, for any pair of distinct edges u and v in E, we have w(u) = w(v).Thus, the pair (G, w) is a weighted graph.Every hierarchy considered in this article is connected for G and therefore, for the sake of simplicity, we use the term hierarchy instead of hierarchy which is connected for G.
Intuitively, a drop of water on a topographic surface drains in the direction of a local minimum and there is a correspondence between the catchment basins of a surface and its local minima.As established in [4], in the context of cuts, a notion of watershed in the framework of edge-weighted graphs is characterized as a (graph) cut induced by a minimum spanning forest rooted in the minima of this graph.Let k be a value in R + .A connected subgraph G of G is a (regional) minimum (for w) at level k if: 1. for any edge u in E(G ), the weight of u is equal to k; and 2. for any edge {x, y} in E(G)\E(G ) such that |{x, y}∩V (G )| ≥ 1, the weight of {x, y} is strictly greater than k.
The sequence T = (CC(G 0 ), . . ., CC(G n−1 )) is called a hierarchical watershed of (G, w) for S. Given a hierarchy H, we say that H is a hierarchical watershed of (G, w) if there exists a sequence S = (M 1 , . . ., M n ) of pairwise distinct minima of w such that {M 1 , . . ., M n } is the set of all minima of w and such that H is a hierarchical watershed for S.
A weighted graph (G, w) and a hierarchical watershed H of (G, w) are illustrated in Figure 3.We can see that H is the hierarchical watershed of (G, w) for the sequence S = (M 1 , M 2 , M 3 , M 4 ) of minima of w.
Important notations and notions: since the weight of the edges of G are pairwise distinct, it follows that any minimum of G is a graph with a single edge.The number of minima of w is denoted by n.Moreover, for any sequence S of pairwise distinct minima of w, the hierarchical watershed of (G, w) for S is unique.Every sequence of minima of w considered in this article is a sequence of n pairwise distinct minima of w and, therefore, we use the term sequence of minima of w instead of sequence of n pairwise distinct minima of w.

Watersheding
In this section, we introduce the watersheding (operator), which maps any saliency map into the saliency map of a hierarchical watershed of (G, w).To present the watersheding, we first introduce binary partition hierarchies by altitude ordering, whose link with hierarchical watersheds is established in [7].

Binary partition hierarchies (by altitude ordering)
Binary partition trees [20] are widely used for hierarchical image representation.In this section, we describe the particular case where the merging process is guided by the edge weights [7].
Given any set X, we denote by |X| the number of elements of X.Let k be any element in {1, . . ., |E|}.We denote by u k the edge in E such that there are k − 1 edges in E of weight strictly smaller than w(u k ).We set , where u k = {x, y} and B x k−1 and B y k−1 are the regions of B k−1 that contain x and y, respectively.The binary partition hierarchy (by altitude ordering) of (G, w), denoted by B, is the hierarchy (B 0 , . . ., B |E| ).
Let B = (B 0 , . . ., B |E| ) be the binary partition hierarchy of (G, w) and let k be a value in {1, . . ., |E|}.Since G is a tree, given u k = {x, y}, it can be seen that B x k−1 and B y k−1 are disjoint.Thus B k is different from B k−1 .We can affirm that B x k−1 ∪ B y k−1 is not a region of B k for any k < k.Since B is a hierarchy, we say that u k is the building edge of the region Given any edge u in E, we denote by R u the region of B whose building edge is u.
In Figure 3, we present the binary partition hierarchy B of the graph (G, w).The building edge of each non-leaf region R of B is shown above the node that represents R.
Important notation: in the sequel of this article, the binary partition hierarchy by altitude ordering of (G, w) is denoted by B.
Let X and Y be two distinct regions of B. If the parent of X is equal to the parent of Y , we say that X is a sibling of Y , that Y is a sibling of X and that X and Y are siblings.It can be seen that X has exactly one sibling and we denote this unique sibling of X by sibling(X).

Watersheding operator
The idea underlying the watersheding operator is to invert the method to compute hierarchical watersheds proposed in [17,7].In order to present this method, we first present the definition of extinction values and extinction maps.
Definition 2 (extinction value and extinction map).Let S be a sequence of minima of w and let R be any region of B. The extinction value of R for S is zero if there is no minimum of w included in R and, otherwise, it is the maximum value i in {1, . . ., n} such that the minimum M i is included in R. Let P be a map from the regions of B into R + .We say that P is the extinction map for S if, for any region R of B, the value P (R) is the extinction value of R for S. We say that P is an extinction map if there exists a sequence of minima S such that P is the extinction map for S.
The following property characterizes extinction maps. 1roperty 3 (extinction map).Let P be a map from the regions of B into R + .The map P is an extinction map for w if and only if the following statements hold true: -∪{P (R) | R is a region of B} = {0, . . ., n}; for any two distinct minima M 1 and M 2 , P (M 1 ) = P (M 2 ); and for any region R of B, we have Let S be a sequence of minima of w.As established in [7], the saliency map f of the hierarchical watershed of (G, w) for S can be obtained through the following steps: 1. computation of the extinction map P for S; and 2. computation of f : for any edge u in E, the weight of Inspired by this simple and efficient method to compute hierarchical watersheds, we propose the watersheding of hierarchies.From any saliency map f , we find an approximated extinction map P such that, for any edge u of G, we have Then, we define a sequence S of minima of w ordered in increasing order for P .The watersheding of f is then defined as the saliency map of the hierarchical watershed of (G, w) for S.
In order to introduce approximated extinction maps, we first present the auxiliary notions of supremum descendant values, non-leaf orderings and dominant regions.
Definition 4 (supremum descendant value).Let f be a map from E into R + and let R be a region of B. The supremum descendant value of R for f , denoted by ∨ f (R), is defined as Definition 5 (non-leaf ordering).Let f be a map from E into R + and let R be a region of B. A non-leaf ordering by f (for B) is a total ordering on the nonleaf regions of B, denoted by ≺ (f,w) , such that, for any two non-leaf regions X and Y whose building edges are u and v, respectively, if Fig. 4: A weighted graph (G, f ), the suppremum descendant values ∨ f for f , and the approximated extinction map ξ f of f for the binary partition hierarchy by altitude ordering B of Figure 3.
Since the edge weights for w are pairwise distinct, we can conclude that, for any map f , there is a unique non-leaf ordering ≺ (f,w) by f .Definition 6 (dominant region).Let f be a map from E into R + and let ≺ (f,w) be the non-leaf ordering by f .Given any non-leaf region R different from V of B, we say that R is a dominant region for f if: For instance, consider the weighted graph (G, w) and the binary partition hierarchy B of (G, w) shown in Figure 3, and the weighted graph (G, f ) shown in Figure 4.The non-leaf ordering ≺ (f,w) by f for B is such that The dominant regions of B for f are the regions M 2 , M 4 , Y 6 .
Definition 7 (approximated extinction map).Let f be a map from E into R + .The approximated extinction map of f (for B) is the map ξ f from the set of regions of B into R + such that: , where u is the building edge of parent(R), otherwise.
In Figure 4, we show the approximated extinction map ξ f of a map f for the hierarchy B shown in Figure 3.The next property establishes that ξ f is an extinction map if and only if f is the saliency map of a hierarchical watershed.Property 8. Let f be a map from E into R + and let ξ f be the approximated extinction map of f .The map f is the saliency map of a hierarchical watershed of (G, w) if and only if the map ξ f is an extinction map.
By Property 3, we can verify that the map ξ f of Figure 4 is not an extinction map because ∨{P (M 1 ), P (M 2 )} = 2 is different from P (Y 5 ) = 1.By Property 8, we may conclude that f is not the saliency map of a hierarchical watershed.
In the next definition, we introduce estimated sequences of minima obtained through approximated extinction maps.Definition 9 (estimated sequence of minima).Let f be a map from E into R + and let ξ f be the approximated extinction map of f .The estimated sequence of minima (of w) for f is the sequence S f = (M 1 , . . ., M n ) such that for i and j in {1, . . ., n}, if i < j, then: For instance, in Figure 4, we can see that ξ f (M 2 ) < ξ f (M 1 ) < ξ f (M 3 ) < ξ f (M 4 ).Therefore, the estimated sequence of minima for f is S f = (M 2 , M 1 , M 3 , M 4 ).The next property establishes that, if f is a hierarchical watershed of (G, w), then f is the hierarchical watershed of (G, w) for S f .Property 10.Let f be a map from E into R + and let S f be the estimated sequence of minima of f .If f is a hierarchical watershed of (G, w), then f is the hierarchical watershed of (G, w) for S f .
Having defined approximated extinction maps and estimated sequences of minima, we formalize the watersheding operator in the following definition.
Definition 11 (watersheding).Given a map f from E into R + , let S f be the estimated sequence of minima of f and let P be the extinction map for S f .The watersheding of f is the map ω(f ) from E into R + such that, for any edge u: In Figure 5, we show the watersheding ω(f ) of the map f of Figure 4.After obtaining the estimated sequence of minima S f = (M 2 , M 1 , M 3 , M 4 ) for f (shown in Figure 4), we compute the extinction map P for S f .Then, the watersheding of f is obtained according to Definition 11.We can verify that ω(f ) is the saliency map of a hierarchical watershed of (G, w) of Figure 3. Indeed, the map ω(f ) is the saliency map of the hierarchical watershed H of Figure 3.
Fig. 5: The extinction map P for the estimated sequence of minima S f = (M 2 , M 1 , M 3 , M 4 ) for the map f (Figure 4) and watersheding ω(f ) of f .
In the following theorem, we establish that the watersheding of any map is the saliency map of a hierarchical watershed of (G, w).
Theorem 12. Let f be a map from E into R + .The watersheding ω(f ) of f is the saliency map of the hierarchical watershed of (G, w) for the estimated sequence of minima for f .
The following properties establish that the watersheding operator is idempotent and that the saliency maps of the hierarchical watersheds of (G, w) are the fixed points of the watersheding operator.
Property 13.Let f be a map from E into R + .The watersheding ω(ω(f )) of ω(f ) is equal to ω(f ).Property 14.Let f be the saliency map of a hierarchical watershed of (G, w).The watersheding ω(f ) of f is equal to f .From Theorem 12 and from Properties 13 and 14, we may conclude the following property, which links the watersheding operator to the problem of recognition of hierarchical watersheds discussed in [11].
Property 15.Let H be a hierarchy and let f be the saliency map of H.The hierarchy H is a hierarchical watershed of (G, w) if and only if the watersheding ω(f ) of f is equal to f .

Discussions and perspectives
In this article, we propose the watersheding operator, which converts any map into the saliency map of a hierarchical watershed.The genericity of the watersheding operator opens a range of potential applications: -Regularization of hierarchies based on non-increasing attributes.In a hierarchical watershed, the order in which catchment basins are merged are often defined by extinction values associated with increasing regional attributes, such as area and volume [14].Given an attribute A, we say that A is increasing if, given any hierarchy H, for any pair of regions R 1 and To compute a hierarchical watershed of (G, w) based on the attribute A, we first obtain extinction values based on A [22].Then, we compute the hierarchical watershed H of (G, w) for the sequence S of minima w ordered by their extinction values.The hierarchy H corresponds to a sequence of filterings of the watershed of (G, w) in which the least important regions according to A are the first regions to be suppressed.However, this property is not guaranteed when A is not increasing.When dealing with non-increasing attributes, e.g.circularity and perimeter, we can obtain extinction values by applying a regularization rule [21] on the attribute values.Alternatively, instead of computing hierarchical watersheds from regularized attribute values, we can compute the watersheding of any hierarchy based on a non-increasing attribute.This is illustrated in Figure 6.We can see that, in the watersheding H w of the circularity based hierarchy H cc , the circular regions are more highlighted when compared to the hierarchical watershed H c computed from regularized circularity values., the hierarchy H cob obtained through the method described in [12] and the watersheding H w of H cob for G .
-Refinement of coarse hierarchies.In [12], the authors propose a high-quality method (COB) to compute hierarchies.However, in some cases, fine regions are not included in the resulting hierarchies.This is the case of the hierarchy H cob of Figure 6.The watersheding H of H cob allows fine regions to appear in the hierarchy while taking into consideration coarse levels of H cob .-Learning of optimal attributes to compute hierarchical watersheds.In [9], the authors showed that combinations of hierarchical watersheds improve the performance of individual hierarchical watersheds.However, there is no guarantee that the resulting combinations are hierarchical watersheds themselves.By using the watersheding operator, we can compute hierarchical watersheds from the combinations of hierarchies.If we obtain similar results, those could be used to learn an attribute A such that the watersheding of the combinations corresponds to the hierarchical watersheds based on A.
In future work, we will extend the watersheding operator to generic graphs.We may also perform the experiments suggested here, namely testing the watersheding operator on hierarchies based on non-increasing attributes, on combinations of hierarchical watersheds and on other coarse hierarchies.

Fig. 3 :
Fig.3:A weighted graph (G, w) with four minima delimited by the dashed rectangles, a hierarchical watershed H of (G, w), and the binary partition hierarchy by altitude ordering B of (G, w).

Fig. 6 :
Fig. 6: First row: an image I, a gradient G of I [8], a hierarchical watershed H c of G based on regularized circularity attribute values, a circularity based hierarchy H cc and the watersheding H w of H cc for G. Second row: An image I , a gradient G of I [8], the hierarchy H cob obtained through the method described in[12] and the watersheding H w of H cob for G .