region-based shape matching and recognition

matching segmentations

segmentation is most often used to separate the foreground regions or objects of interest from the background

Once the potentially interesting regions have been identified, the recognition module focuses on the corresponding subimages and their interrelations.


  • some approaches represents each segmented region by a collection of image descriptors and classifies based on the combined vector of descriptors.
  • some methods use recognition to aid segmentation, either via feedback or by running the two modules jointly

use segmentation in a different way

Besides separating foreground from background, a segmentation communicates the intrinsic structure of an image.

The homogeneous regions of an image can be considered as basic visual units giving a compressed image representation, and the relations between segments, and their locations and shapes, provide the global shape structure of the image.

Utilizing segmentations for region-based shape matching has several advantages:

1) By matching all regions at once, we gain robustness to the grouping failures for any single region;

2) since segmentations are computed using global image information, they can localize the true edges more accurately than local edge-detection/grouping methods;
3) segmentations can reveal global shape structures which are more distinctive than local features.

“hallucinated” boundaries

  • A typical segmentation includes both true and hallucinated boundaries.
  • To match segmentations, we need a metric that detects the true matching boundaries while tolerating the hallucinations.

Segmentation Similarity Measure

Mutual information (MI) – the basic concept is the structure entropy (SE), i.e. the entropy measuring a segmentation’s complexity.

    • A segmentation with many small segments has high SE; large segments give low SEcompute SE for individual segmentations

Structure mutual information (SMI) – between segmentations in terms of their joint segmentation obtained by superimposing the individual ones.

    • When two segmentations have matching boundaries, the joint segmentation has large regions and low SE, so the SMI is high.
    • For nonmatching segmentations, the joint segmentation has smaller regions and higher SE, so the SMI is low.

This similarity measure achieves good tolerance against small shape variations.

HoG can improve the shape representation accuracy by inserting more bins; however, since different bins are evaluated independently, employing fine-scale bins loses integrative power in describing/comparing large scale shapes.

Segmentation Unreliability – Segmentations cannot be computed reliably bottom-up from image data.

  • Soft integration of multiple regions
  • exploiting of statistical interactions between different region types
  • using multiple segmentations
  • oversegmentations

For shape matching, oversegmentation is more appropriate since it preserves many of the true shape structures
(though these may be hidden within many fake boundaries).

Since an oversegmentation includes strong curves, even if they are open, it may be exploited for matching open as well as closed curves. Oversegmentations contain enough shape information for recognition

that two pixels share the same segment label in a segmentation if and only if they are connected by pixels from the same segment.

  • distant pixels separated by pixels from different partitions do not belong to the same segment.
  • Given a segmentation s with n segments, we define the structure entropy for s as
    • sum is over segments
    • pi is the area ratio of the ith segment to the whole image



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




Just another site

Jing's Blog

Just another site

Start from here......







Just another site

Where On Earth Is Waldo?

A Project By Melanie Coles

the Serious Computer Vision Blog

A blog about computer vision and serious stuff

Cauthy's Blog

paper review...

Cornell Computer Vision Seminar Blog

Blog for CS 7670 - Special Topics in Computer Vision


Life through nerd-colored glasses

Luciana Haill

Brainwaves Augmenting Consciousness



Dr Paul Tennent

and the university of nottingham

turn off the lights, please

A bunch of random, thinned and stateless thoughts around the Web

%d bloggers like this: