We build the Gaussian scale-space of the original image with logarithmically ordered scales, cf. Drauschke et al. 2006. Each of the convolved images decribes the image at one particular layer in the scale space. The gradients are used as an input for a watershed algorithm that returns a partitioning with regions for each layer of the image's scale-space. We define the across-scale-neighbourhood by a region mapping, where we match a region of a layer s to that region of layer s+1, which has the maximal overlap with the region of layer s. Note that this asymmetric relation between two regions of different scale-space layers does not depend on a threshold. However it not transitive, thus we are not allowed to skip one scale layer.
We therefore obtain an image intrinsic graph structure which is a forest of trees and reflects the partonomy. This structure shall be used for combining the compositional hierarchies of the classes in our ontology with the interpretation of the segmented regions at a concrete scale. Let is take the region F from the small example graph below: This region has been aggregated in scale space from the regions B, C and D of the image layer below, and in the next layer region F merges with region E to region G. Thus the extracted features and / or the classification results of the regions B,C,D and G shall be used for improving the interpretation of region F. This is planned to be modelled by e.g. a small Bayesian network with an integrated Hidden Markov Random Field.