{"id":6624,"date":"2015-11-08T10:07:07","date_gmt":"2015-11-08T10:07:07","guid":{"rendered":"http:\/\/revoscience.com\/en\/?p=6624"},"modified":"2015-11-08T10:07:07","modified_gmt":"2015-11-08T10:07:07","slug":"amplifying-or-removing-visual-variation","status":"publish","type":"post","link":"https:\/\/www.revoscience.com\/en\/amplifying-or-removing-visual-variation\/","title":{"rendered":"Amplifying \u2014 or removing \u2014 visual variation"},"content":{"rendered":"

Algorithms could offer new tools for graphics software or reveal structural defects.<\/strong><\/em><\/span><\/p>\n

\"An<\/a>
An example of the Non-Local Variations algorithm that automatically detects and visualizes small deformations between repeating structures in a single image. On the left is the original image. In the right image, the variability in the shape of the corn\u2019s kernels is reduced, and the misalignment of rows is corrected.
Courtesy of Giandomenico Pozz and the researchers<\/figcaption><\/figure>\n

CAMBRIDGE, Mass.<\/strong> —\u00a0At the Siggraph Asia conference this week, MIT researchers presented a pair of papers describing techniques for either magnifying or smoothing out small variations in digital images.<\/span><\/p>\n

The techniques could be used to produce more polished images for graphic-design projects, or, applied in the opposite direction, they could disclose structural defects, camouflaged objects, or movements invisible to the naked eye that could be of scientific interest.<\/span><\/p>\n

[pullquote]The technique works not only with geometrically simple forms like corn kernels and bricks but with more complex forms as well.[\/pullquote]<\/p>\n

Conceptually, the work builds on a\u00a0long<\/span><\/a>\u00a0line<\/span><\/a>\u00a0of research from several groups in MIT\u2019s Computer Science and Artificial Intelligence Laboratory (CSAIL), which sought to amplify minute motions in digital video. \u201cIn motion magnification, the deviations are over time, and the model is deviation from being perfectly static,\u201d says Tali Dekel, a postdoc in CSAIL and a co-author on both papers. \u201cOur method takes as input only a single image, and it looks for deviation in space. We don\u2019t need to know time history to do that.\u201d<\/span><\/p>\n

One of the two papers, on which Dekel is first author, presents an algorithm that looks for repeated forms within an image, such as the kernels of an ear of corn or bricks in a wall. It can then iron out differences across the image, producing idealized but still natural-looking corn ears or brick walls, or amplify the differences, making them more evident to the naked eye.<\/span><\/p>\n

The algorithm works with color as well as shape. So, for instance, it can take an image in which a chameleon is concealed against the trunk of a tree and enhance subtle color differences so that the chameleon stands out blue against an orange background.<\/span><\/p>\n

Joining Dekel on that paper are professor of computer science and engineering William Freeman, whose group she\u2019s a member of, and colleagues from Israel\u2019s Technion and Weizmann Institute.<\/span><\/p>\n

Imperfect form<\/strong><\/span><\/p>\n

The algorithm described in the other paper amplifies deviations from ideal geometries. The roofline of a house, for instance, could look perfectly straight to the naked eye but still sag toward the middle. The algorithm can exaggerate that type of flaw, potentially drawing attention to structural problems.<\/span><\/p>\n

In other experiments, the algorithm was able to identify a rippling in Saturn\u2019s rings that could offer information about the orbital pattern of the planet\u2019s moons, and by magnifying changes to a regular pattern projected on a screen behind a candle, it revealed thermal variations caused by the candle\u2019s flame.<\/span><\/p>\n

The first author on that paper is Neal Wadhwa, another MIT graduate student in electrical engineering and computer science. Joining him are Dekel, Freeman, graduate student Donglai Wei, and Fr\u00e9do Durand, a professor of computer science and engineering.<\/span><\/p>\n

The first algorithm \u2014 the one that recognizes repeated forms \u2014 begins by comparing patches of the source image, at different scales, and identifying those that seem to be visually similar. Then it averages out all the visually similar patches and uses the averages to construct a new, highly regular version of the image. This image may look unnatural, but its purpose is just to serve as an initial target.<\/span><\/p>\n

Then the algorithm identifies a mathematical function that moves the pixels of the source image around, producing the best possible approximation of the target image. From that function, it creates a new target image. It then iterates back and forth, producing ever more natural-looking target images and ever more regular mathematical transformations, until the two converge.<\/span><\/p>\n

Once the algorithm has a function that produces a regular image, it can simply invert it to produce a more distorted image.<\/span><\/p>\n

Implications<\/strong><\/span><\/p>\n

The technique works not only with geometrically simple forms like corn kernels and bricks but with more complex forms as well. So, for instance, it can take an image of a line of dancers executing the same kick and standardize their heights and the distances between them.<\/span><\/p>\n

If it\u2019s implemented particularly aggressively, it can even cut irregularities out of an image \u2014 for instance, standardizing the size and shape of the cells of a honeycomb while deleting the bees crawling over it. As such, it could be a useful resource for image-manipulation programs like Photoshop.<\/span><\/p>\n

In materials science, a standard technique for identifying defects in a material\u2019s surface is to cover it with tiny soap bubbles and look for irregularities. The MIT researchers are also collaborating with materials scientists to use their algorithm to enhance that process.<\/span><\/p>\n

The second algorithm uses existing techniques to identify the geometric shapes indicated by color gradations in an image. Then it excises a narrow band of the image that traces the curve defining each of those shapes. It then straightens the bands out, creating a uniform representation of all the shapes in the image.<\/span><\/p>\n

At regular intervals, it considers local variations in color across the width of each band. These will typically vary, indicating deviations from the idealized geometry of the initial curve. From those deviations, the algorithm constructs a new, more erratic curve, which it can exaggerate and then reinsert into the image.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

At the Siggraph Asia conference this week, MIT researchers presented a pair of papers describing techniques for either magnifying or smoothing out small variations in digital images.<\/p>\n","protected":false},"author":6,"featured_media":6625,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[],"class_list":["post-6624","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research"],"featured_image_urls":{"full":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"thumbnail":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0-150x150.jpg",150,150,true],"medium":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0-300x199.jpg",300,199,true],"medium_large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"1536x1536":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"2048x2048":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"ultp_layout_landscape_large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"ultp_layout_landscape":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"ultp_layout_portrait":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"ultp_layout_square":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"newspaper-x-single-post":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"newspaper-x-recent-post-big":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"newspaper-x-recent-post-list-image":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",95,63,false],"web-stories-poster-portrait":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",448,298,false],"web-stories-publisher-logo":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",96,64,false],"web-stories-thumbnail":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2015\/11\/MIT-irregular-1_0.jpg",150,100,false]},"author_info":{"info":["Amrita Tuladhar"]},"category_info":"Research<\/a>","tag_info":"Research","comment_count":"0","_links":{"self":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts\/6624","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/comments?post=6624"}],"version-history":[{"count":0,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts\/6624\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/media\/6625"}],"wp:attachment":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/media?parent=6624"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/categories?post=6624"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/tags?post=6624"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}