Image segmentation by locally specified multi-coloured polygonal Markov fields

We introduce a class of multi-coloured polygonal Markov fields driven by local activity functions. Whereas the local rather than global nature of the field specification ensures substantial additional flexibility for statistical applications in comparison to classical polygonal fields. Within the framework of this theory we develop an image segmentation algorithm based on Markovian optimisation dynamics combining the simulated annealing ideas with those of Chen-style stochastic optimisation, in which successive segmentation updates are carried out simultaneously with adaptive optimisation of the local activity functions.


Introduction
One of the fundamental problem in computer vision is image segmentation, the partitioning of an image into distinct (non-overlapping) segments which are homogeneous with respect to some characteristic (1) .For example, in an image of a street we might like to distinguish people, cars, a road, and other objects.Image segmentation can be applied to medical imaging (2) , locating pathologies (3) or to determine shapes and sizes of organs (4,5) .Neuroscientists can apply these techniques to check whether synapses exist at points of contacts, which helps to determine connections between neurons (6) .Hundreds of image segmentation algorithms have been developed, we refer the reader to (7) for a detailed discussion on such techniques.Still there is no single method which achieves satisfactory results for all images.We will mention only the most relevant classes: the morphological watershed (8) , Markov random fields (9) , and intermediate level methods which focus on the partition of the image that is the outcome of a segmentation (10)(11)(12) .The latter will be the main competitor of our algorithm.Now, we will describe an important class of probabilistic graphical models, build on the basis of undirected graphs, called Markov Network (also known as Markov Random Field (MRF)).This model has received a great deal of attention, due to its wide applications to computer vision (13) , such as: edge detection, image restoration, stereovision, image classification, and image segmentation.This type of modelling in vision was introduced in (14).In most of that vision algorithms use a subclass of MRF structured in the form of a grid, where variables correspond to pixels of the image and the edges correspond to interactions between adjacent pixels.For some tasks, like image segmentation, it is more natural to process regions of pixels with edges that define interactions between adjacent regions.
That concept was introduced in Polygonal Markov fields (PMF), originally constructed by Arak, Clifford (15)(16)(17) , and Surgailis (18) .PMF are random ensembles of non-intersecting contours in the plane, arising in a Gibbsian set-up and sharing a number of important features with the two-dimensional Ising model.Due to their purely (1)   (2) continuum nature, polygonal fields admit natural applications for instance in a digital image processing and segmentation, where they can be used for the majority of tasks traditionally reserved for lattice-based Markov fields, while being completely free of directional lattice artefacts (19,20) .In this paper we extend the algorithm (19)(20)(21) to multi-coloured images.

Formal Definition
Now we will recall the formal construction of the consistent multicoloured polygonal Markov fields (15) , adapted from (20) to better fit our needs.where  is the set of discontinuity points in .An efficient simulation technique for polygonal Markov fields has been presented in (22) .

Dynamic Representation
In this section, we recall and extend the dynamic representation presented by Matuszak & Schreiber (21) .The generalised representation provides an easy and efficient way for simulating the Arak process.Let us fix the convex field domain D and denote (  ) ∈[0,1] as a time-indexed family of compact subsets of  .It can be interpreted as a growing window which gradually reveal portions of the polygonal field.More formally (  ) ∈[0,1] satisfies:   1 coincides with  .
   is continuous in the usual Hausdorff metric on compacts.

Generalised Dynamic
Consider now growing window dynamics in time  ∈ [0,1] with rules given below:  (GE:Initialise) Begin with empty field at the time 0,  (GE:Unfold) Between critical moments listed below, during the time interval [t,t + dt] the unfolding field edges in   reaching   extend straight to  + ∖    (GE:BoundaryHit) When a field edge hits the boundary , it stops growing in this direction  (GE:Collision) When two unfolding field edges intersect in  + ∖   , then V, X or T collision occurs (see Fig. 2).With the admissible number of colours, k, fixed, the family of polygonal fields is parametrised by parameter   that controls the density of V-shaped nodes: The parameters   and   control the density of X-and T-shaped nodes, respectively  (GE:DirectionalUpdate) A field edge extending along l ∈ [[Dt]] updates its direction during [t,t + dt].Directional updates of this type are all performed independently,  (GE:LineBirth) Whenever the anchor point A(l) of a line l falls into   + ∖   , the line l is born at the time t at its anchor point with probability M(l; A(l))µ(dl),  (GE:VertexBirth) For each intersection point of lines  1 and  2 falling into  + ∖   , the pair of field lines  1 and  2 is born at  1 ∩  2 with probability M(  1 ;  1 ∩  2 )M(  2 ;  1 ∩  2 )µ(d  1 )µ(d  2 ), whereupon both lines begin unfolding in the directions away from their respective anchor points A( 1 ) and A( 2 ).

Disagreement Loop Dynamics
The concept of disagreement loop make the updates easy to simulate.Example of a disagreement loop is presented in Fig. 3.During the optimisation phase we can only add/remove vertex birth points, while keeping unchanged remaining evolution rules (GE).Detailed discussion is available in (19-20).

Adaptive Optimisation
The quality of segmentation (γ) is quantified in terms of an energy function which in our case is a positive linear combination of a pixel misclassification ratio (we are checking all possible allocations of colours and select the best one), the length element and the number of edges.To optimize the segmentation we compute the disagreement loop of two segmentation and according to Chen's Optimizations Rules (24) we optimize underlying gradient  field (21) .

Results
The programme has been implemented in language D. All test runs were executed on a machine with Intel Core 2 Q9300 2.50 GHz CPU, 4GB RAM and NVIDIA GTX 480.The application is single threaded, so it is applicable to the core of only one processor.All computations were performed with double precision arithmetic.The sequential version of that algorithm has been implemented.The parallel (GPU) version is presented in (23) .The segmentations shown in Fig. 4. have been obtained after about 50000 (accepted) updates under a linear cooling schedule, with mean execution time 0.1 sec per single update.

Fig. 1 .
Realization of a polygonal Markov field: (a) with two (k=2) available colours; (b) with four (k=4) available colours.Let D ⊆ ℝ 2 be a convex bounded domain and   be a finite and non-atomic measure on set ℒ  of all lines  in ℝ 2 which intersect D, and let  = 1, … ,  be a set of available colours,  ≥ 2 .For any collection   =  1 , … ,   of lines, where   ∈ ℒ  the set Γ    consists of all functions :  →  such that 1.    ∩  ∞ =0

2 .Γ
For any  = 1, … ,  , the intersection   ∩  consists of a single segment with a positive length and possibly some isolated points.Define the set of all realizations on D by  is the set of all planar graphs  in  ∪  with faces coloured by labels in  such that the following conditions are satisfied  all edges of  lie on the lines of ℒ  ,  all vertices of  ∈  are of degree 2, 3, or 4,  all vertices of  on D$, are of degree 1,  no adjacent regions share the same colour.In other words,  consists of a finite number of disjoint polygons, possibly nested and chopped off by the boundary -see Fig.1for a typical realization of that process.The polygonal Markov field   on  with the Hamiltonian given by total edge length is described as    ∈  =  exp (−2 ℎ ())  ∈Γ  (Λ D )∩  exp (−2 ℎ ())  ∈Γ  (Λ D ) , for all  ⊆ Γ  Borel measurable and Λ D is the restriction of the homogeneous Poisson line process Λ to .

Fig. 2 .
Possible results of a collision of two unfolding edges (GE:Collision).

Fig. 3 .
Fig. 3. Example of a disagreement loop.Original configuration is represented by the black polygon and a new one by the blue polygon.The disagreement loop is marked with red colour.