Paint and Click: Unified Interactions for Image Boundaries

  1. Home
  2. Publications
We provide a novel, unified interaction for pairwise image boundaries that combines both paint and constraint-based user edits. (a) Input image. (b) Desired segmentation. (c) Using only painting edits, a user can use include (green striped) and exclude (red) annotations to select the object. These annotations can be numerous and tedious for fine features such as the legs and antennae. (d) Using only constraints (or anchors), a user can click (red) control points to form the object’s boundary. Even with automatic constraints (yellow), many clicks are required. (e) Our unified approach allows users to mix the complementary editing metaphors, leading to a more flexible and faster experience. Image courtesy of Flicker user tonrulkens.
Image boundaries are a fundamental component of many interactive digital photography techniques, enabling applications such as segmentation, panoramas, and seamless image composition. Interactions for image boundaries often rely on two complementary but separate approaches: editing via painting or clicking constraints. In this work, we provide a novel, unified approach for interactive editing of pairwise image boundaries that combines the ease of painting with the direct control of constraints. Rather than a sequential coupling, this new formulation allows full use of both interactions simultaneously, giving users unprecedented flexibility for fast boundary editing. To enable this new approach, we provide technical advancements. In particular, we detail a reformulation of image boundaries as a problem of finding cycles, expanding and correcting limitations of the previous work. Our new formulation provides boundary solutions for painted regions with performance on par with state-of-the-art specialized, paint-only techniques. In addition, we provide instantaneous exploration of the boundary solution space with user constraints. Finally, we provide examples of common graphics applications impacted by our new approach. 
	
@article {10.1111:cgf.12568, 
          journal = {Computer Graphics Forum}, 
          title = {{Paint and Click: Unified Interactions for Image Boundaries}}, 
          author = {Summa, Brian and Gooch, Amy A. and Scorzelli, Giorgio and Pascucci, Valerio}, 
          issue_date = {April 2015},
          volume = {34},
          number = {2},
          month = apr,
          year = {2015},
          pages = {385--393},
          publisher = {The Eurographics Association and John Wiley & Sons Ltd.}, 
          DOI = {10.1111/cgf.12568} 
          }