{"id":12694,"date":"2017-07-20T07:50:40","date_gmt":"2017-07-20T07:50:40","guid":{"rendered":"http:\/\/revoscience.com\/en\/?p=12694"},"modified":"2017-07-20T07:50:40","modified_gmt":"2017-07-20T07:50:40","slug":"bringing-neural-networks-cellphones","status":"publish","type":"post","link":"https:\/\/www.revoscience.com\/en\/bringing-neural-networks-cellphones\/","title":{"rendered":"Bringing neural networks to cellphones"},"content":{"rendered":"

Method for modeling neural networks\u2019 power consumption could help make the systems portable.<\/strong><\/em><\/span><\/p>\n

\"\"
MIT researchers have designed new methods for paring down neural networks so that they\u2019ll run more efficiently on handheld devices.
Image: Jose-Luis Olivares\/MIT<\/figcaption><\/figure>\n

CAMBRIDGE, Mass. —\u00a0In recent years, the best-performing artificial-intelligence systems \u2014 in areas such as autonomous driving, speech recognition, computer vision, and automatic translation \u2014 have come courtesy of software systems known as<\/span>\u00a0neural networks<\/a>.<\/p>\n

But neural networks take up a lot of memory and consume a lot of power, so they usually run on servers in the cloud, which receive data from desktop or mobile devices and then send back their analyses.<\/span><\/p>\n

Last year, MIT associate professor of electrical engineering and computer science Vivienne Sze and colleagues unveiled a new, energy-efficient computer chip optimized for neural networks, which could enable powerful artificial-intelligence systems to run locally on mobile devices.<\/span><\/p>\n

Now, Sze and her colleagues have approached the same problem from the opposite direction, with a battery of techniques for designing more energy-efficient neural networks. First, they developed an analytic method that can determine how much power a neural network will consume when run on a particular type of hardware. Then they used the method to evaluate new techniques for paring down neural networks so that they\u2019ll run more efficiently on handheld devices.<\/span><\/p>\n

The researchers describe the work in a<\/span>\u00a0paper<\/a>\u00a0they\u2019re presenting next week at the Computer Vision and Pattern Recognition Conference. In the paper, they report that the methods offered as much as a 73 percent reduction in power consumption over the standard implementation of neural networks, and as much as a 43 percent reduction over the best previous method for paring the networks down.<\/span><\/p>\n

Energy evaluator<\/strong><\/span><\/p>\n

Loosely based on the anatomy of the brain, neural networks consist of thousands or even millions of simple but densely interconnected information-processing nodes, usually organized into layers. Different types of networks vary according to their number of layers, the number of connections between the nodes, and the number of nodes in each layer.<\/span><\/p>\n

The connections between nodes have \u201cweights\u201d associated with them, which determine how much a given node\u2019s output will contribute to the next node\u2019s computation. During training, in which the network is presented with examples of the computation it\u2019s learning to perform, those weights are continually readjusted, until the output of the network\u2019s last layer consistently corresponds with the result of the computation.<\/span><\/p>\n

\u201cThe first thing we did was develop an energy-modeling tool that accounts for data movement, transactions, and data flow,\u201d Sze says. \u201cIf you give it a network architecture and the value of its weights, it will tell you how much energy this neural network will take. One of the questions that people had is \u2018Is it more energy efficient to have a shallow network and more weights or a deeper network with fewer weights?\u2019 This tool gives us better intuition as to where the energy is going, so that an algorithm designer could have a better understanding and use this as feedback. The second thing we did is that, now that we know where the energy is actually going, we started to use this model to drive our design of energy-efficient neural networks.\u201d<\/span><\/p>\n

In the past, Sze explains, researchers attempting to reduce neural networks\u2019 power consumption used a technique called \u201cpruning.\u201d Low-weight connections between nodes contribute very little to a neural network\u2019s final output, so many of them can be safely eliminated, or pruned.<\/span><\/p>\n

Principled pruning<\/strong><\/span><\/p>\n

With the aid of their energy model, Sze and her colleagues \u2014 first author Tien-Ju Yang and Yu-Hsin Chen, both graduate students in electrical engineering and computer science \u2014 varied this approach. Although cutting even a large number of low-weight connections can have little effect on a neural net\u2019s output, cutting all of them probably would, so pruning techniques must have some mechanism for deciding when to stop.<\/span><\/p>\n

The MIT researchers thus begin pruning those layers of the network that consume the most energy. That way, the cuts translate to the greatest possible energy savings. They call this method \u201cenergy-aware pruning.\u201d<\/span><\/p>\n

Weights in a neural network can be either positive or negative, so the researchers\u2019 method also looks for cases in which connections with weights of opposite sign tend to cancel each other out. The inputs to a given node are the outputs of nodes in the layer below, multiplied by the weights of their connections. So the researchers\u2019 method looks not only at the weights but also at the way the associated nodes handle training data. Only if groups of connections with positive and negative weights consistently offset each other can they be safely cut. This leads to more efficient networks with fewer connections than earlier pruning methods did.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Method for modeling neural networks\u2019 power consumption could help make the systems portable. CAMBRIDGE, Mass. —\u00a0In recent years, the best-performing artificial-intelligence systems \u2014 in areas such as autonomous driving, speech recognition, computer vision, and automatic translation \u2014 have come courtesy of software systems known as\u00a0neural networks. But neural networks take up a lot of memory […]<\/p>\n","protected":false},"author":6,"featured_media":12695,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[],"class_list":["post-12694","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research"],"featured_image_urls":{"full":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"thumbnail":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0-150x150.jpg",150,150,true],"medium":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0-300x200.jpg",300,200,true],"medium_large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"1536x1536":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"2048x2048":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"ultp_layout_landscape_large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"ultp_layout_landscape":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"ultp_layout_portrait":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",600,401,false],"ultp_layout_square":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",600,401,false],"newspaper-x-single-post":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"newspaper-x-recent-post-big":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",539,360,false],"newspaper-x-recent-post-list-image":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",95,63,false],"web-stories-poster-portrait":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",638,426,false],"web-stories-publisher-logo":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",96,64,false],"web-stories-thumbnail":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2017\/07\/MIT-Portable-Neural_0.jpg",150,100,false]},"author_info":{"info":["Amrita Tuladhar"]},"category_info":"Research<\/a>","tag_info":"Research","comment_count":"0","_links":{"self":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts\/12694","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/comments?post=12694"}],"version-history":[{"count":0,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts\/12694\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/media\/12695"}],"wp:attachment":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/media?parent=12694"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/categories?post=12694"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/tags?post=12694"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}