{"id":17504,"date":"2020-03-25T02:50:18","date_gmt":"2020-03-25T02:50:18","guid":{"rendered":"https:\/\/www.revoscience.com\/en\/?p=17504"},"modified":"2020-06-09T12:09:49","modified_gmt":"2020-06-09T12:09:49","slug":"system-trains-driverless-cars-in-simulation-before-they-hit-the-road","status":"publish","type":"post","link":"https:\/\/www.revoscience.com\/en\/system-trains-driverless-cars-in-simulation-before-they-hit-the-road\/","title":{"rendered":"System trains driverless cars in simulation before they hit the road"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

Using a photorealistic simulation engine, vehicles learn to drive in the real world and recover from near-crash scenarios.<\/strong> <\/p>\n\n\n\n

By Rob Matheson, MIT News Office <\/strong><\/p>\n\n\n\n

CAMBRIDGE, Mass. —\u00a0A simulation system invented at MIT to train driverless cars creates a photorealistic world with infinite steering possibilities, helping the cars learn to navigate a host of worse-case scenarios before cruising down real streets. \u00a0<\/p>\n\n\n\n

Control systems, or \u201ccontrollers,\u201d for autonomous vehicles largely rely on real-world datasets of driving trajectories from human drivers. From these data, they learn how to emulate safe steering controls in a variety of situations. But real-world data from hazardous \u201cedge cases,\u201d such as nearly crashing or being forced off the road or into other lanes, are \u2014 fortunately \u2014 rare.<\/p>\n\n\n\n

Some computer programs, called \u201csimulation engines,\u201d aim to imitate these situations by rendering detailed virtual roads to help train the controllers to recover. But the learned control from simulation has never been shown to transfer to reality on a full-scale vehicle.<\/p>\n\n\n\n

The MIT researchers tackle the problem with their photorealistic simulator, called Virtual Image Synthesis and Transformation for Autonomy (VISTA<\/a>). It uses only a small dataset, captured by humans driving on a road, to synthesize a practically infinite number of new viewpoints from trajectories that the vehicle could take in the real world. The controller is rewarded for the distance it travels without crashing, so it must learn by itself how to reach a destination safely. In doing so, the vehicle learns to safely navigate any situation it encounters, including regaining control after swerving between lanes or recovering from near-crashes. \u00a0<\/p>\n\n\n\n

In tests, a controller trained within the VISTA simulator safely was able to be safely deployed onto a full-scale driverless car and to navigate through previously unseen streets. In positioning the car at off-road orientations that mimicked various near-crash situations, the controller was also able to successfully recover the car back into a safe driving trajectory within a few seconds. A\u00a0paper<\/a>\u00a0describing the system has been published in\u00a0IEEE Robotics and Automation Letters<\/em>\u00a0and will be presented at the upcoming ICRA conference in May.<\/p>\n\n\n\n

\u201cIt\u2019s tough to collect data in these edge cases that humans don\u2019t experience on the road,\u201d says first author Alexander Amini, a PhD student in the Computer Science and Artificial Intelligence Laboratory (CSAIL). \u201cIn our simulation, however, control systems can experience those situations, learn for themselves to recover from them, and remain robust when deployed onto vehicles in the real world.\u201d<\/p>\n\n\n\n

The work was done in collaboration with the Toyota Research Institute. Joining Amini on the paper are Igor Gilitschenski, a postdoc in CSAIL; Jacob Phillips, Julia Moseyko, and Rohan Banerjee, all undergraduates in CSAIL and the Department of Electrical Engineering and Computer Science; Sertac Karaman, an associate professor of aeronautics and astronautics; and Daniela Rus, director of CSAIL and the Andrew and Erna Viterbi Professor of Electrical Engineering and Computer Science.<\/p>\n\n\n\n

Data-driven simulation<\/strong><\/p>\n\n\n\n

Historically, building simulation engines for training and testing autonomous vehicles has been largely a manual task. Companies and universities often employ teams of artists and engineers to sketch virtual environments, with accurate road markings, lanes, and even detailed leaves on trees. Some engines may also incorporate the physics of a car\u2019s interaction with its environment, based on complex mathematical models.<\/p>\n\n\n\n

But since there are so many different things to consider in complex real-world environments, it\u2019s practically impossible to incorporate everything into the simulator. For that reason, there\u2019s usually a mismatch between what controllers learn in simulation and how they operate in the real world.<\/p>\n\n\n\n

Instead, the MIT researchers created what they call a \u201cdata-driven\u201d simulation engine that synthesizes, from real data, new trajectories consistent with road appearance, as well as the distance and motion of all objects in the scene.<\/p>\n\n\n\n

They first collect video data from a human driving down a few roads and feed that into the engine. For each frame, the engine projects every pixel into a type of 3D point cloud. Then, they place a virtual vehicle inside that world. When the vehicle makes a steering command, the engine synthesizes a new trajectory through the point cloud, based on the steering curve and the vehicle\u2019s orientation and velocity.<\/p>\n\n\n\n

Then, the engine uses that new trajectory to render a photorealistic scene. To do so, it uses a convolutional neural network \u2014 commonly used for image-processing tasks \u2014 to estimate a depth map, which contains information relating to the distance of objects from the controller\u2019s viewpoint. It then combines the depth map with a technique that estimates the camera\u2019s orientation within a 3D scene. That all helps pinpoint the vehicle\u2019s location and relative distance from everything within the virtual simulator.<\/p>\n\n\n\n

Based on that information, it reorients the original pixels to recreate a 3D representation of the world from the vehicle\u2019s new viewpoint. It also tracks the motion of the pixels to capture the movement of the cars and people, and other moving objects, in the scene. \u201cThis is equivalent to providing the vehicle with an infinite number of possible trajectories,\u201d Rus says. \u201cBecause when we collect physical data, we get data from the specific trajectory the car will follow. But we can modify that trajectory to cover all possible ways of and environments of driving. That\u2019s really powerful.\u201d<\/p>\n\n\n\n

Reinforcement learning from scratch<\/strong><\/p>\n\n\n\n

Traditionally, researchers have been training autonomous vehicles by either following human defined rules of driving or by trying to imitate human drivers. But the researchers make their controller learn entirely from scratch under an \u201cend-to-end\u201d framework, meaning it takes as input only raw sensor data \u2014 such as visual observations of the road \u2014\u00a0and, from that data, predicts steering commands at outputs.<\/p>\n\n\n\n

\u201cWe basically say, \u2018Here\u2019s an environment. You can do whatever you want. Just don\u2019t crash into vehicles, and stay inside the lanes,\u2019\u201d Amini says.<\/p>\n\n\n\n

This requires \u201creinforcement learning\u201d (RL), a trial-and-error machine-learning technique that provides feedback signals whenever the car makes an error. In the researchers\u2019 simulation engine, the controller begins by knowing nothing about how \u00a0to drive, what a lane marker is, or even other vehicles look like, so it starts executing random steering angles. <\/p>\n\n\n\n

It gets a feedback signal only when it crashes. At that point, it gets teleported to a new simulated location and has to execute a better set of steering angles to avoid crashing again. Over 10 to 15 hours of training, it uses these sparse feedback signals to learn to travel greater and greater distances without crashing.<\/p>\n\n\n\n

After successfully driving 10,000 kilometers in simulation, the authors apply that learned controller onto their full-scale autonomous vehicle in the real world. The researchers say this is the first time a controller trained using end-to-end reinforcement learning in simulation has successful been deployed onto a full-scale autonomous car.<\/p>\n\n\n\n

\u201cThat was surprising to us. Not only has the controller never been on a real car before, but it\u2019s also never even seen the roads before and has no prior knowledge on how humans drive,\u201d Amini says.<\/p>\n\n\n\n

Forcing the controller to run through all types of driving scenarios enabled it to regain control from disorienting positions \u2014 such as being half off the road or into another lane \u2014 and steer back into the correct lane within several seconds. \u201cAnd other state-of-the-art controllers all tragically failed at that, because they never saw any data like this in training,\u201d Amini says.<\/p>\n\n\n\n

Next, the researchers hope to simulate all types of road conditions from a single driving trajectory, such as night and day, and sunny and rainy weather. They also hope to simulate more complex interactions with other vehicles on the road. \u201cWhat if other cars start moving and jump in front of the vehicle?\u201d Rus says. \u201cThose are complex, real-world interactions we want to start testing.\u201d<\/p>\n
","protected":false},"excerpt":{"rendered":"

Using a photorealistic simulation engine, vehicles learn to drive in the real world and recover from near-crash scenarios. <\/p>\n","protected":false},"author":2,"featured_media":17505,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17,28],"tags":[],"class_list":["post-17504","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research","category-techbiz"],"featured_image_urls":{"full":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"thumbnail":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011-200x200.jpg",200,200,true],"medium":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011-300x200.jpg",300,200,true],"medium_large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"1536x1536":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"2048x2048":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"ultp_layout_landscape_large":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"ultp_layout_landscape":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"ultp_layout_portrait":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",600,400,false],"ultp_layout_square":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",600,400,false],"newspaper-x-single-post":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"newspaper-x-recent-post-big":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011-550x360.jpg",550,360,true],"newspaper-x-recent-post-list-image":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011-95x65.jpg",95,65,true],"web-stories-poster-portrait":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",639,426,false],"web-stories-publisher-logo":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",96,64,false],"web-stories-thumbnail":["https:\/\/www.revoscience.com\/en\/wp-content\/uploads\/2020\/03\/MIT-Simulation-Training-011.jpg",150,100,false]},"author_info":{"info":["RevoScience"]},"category_info":"Research<\/a> Tech<\/a>","tag_info":"Tech","comment_count":"0","_links":{"self":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts\/17504","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/comments?post=17504"}],"version-history":[{"count":0,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/posts\/17504\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/media\/17505"}],"wp:attachment":[{"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/media?parent=17504"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/categories?post=17504"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.revoscience.com\/en\/wp-json\/wp\/v2\/tags?post=17504"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}