Virtual Reality has a lot of potential uses. A CAIT researcher has developed a pair of immersive, virtual environments to prove that to transportation stakeholders.
Using VR technology, different environments such as bridges can be created where everything from cracks to the weather can be customized to meet workforce training needs. Pictured to the left is a bridge simulation, and to the right is the simulation with cracks. Photo ©Jie Gong.
When the average person thinks of Virtual Reality (VR) devices such as the Oculus Rift or HTC Vive, gaming might be the first thing that comes to mind. Innovative projects from the Rutgers Center for Advanced Infrastructure and Transportation (CAIT) aim to change that mindset as they mesh VR technology with the transportation needs of today.
Led by Dr. Jie Gong, an associate professor in the Department of Civil and Environmental Engineering, a team has developed fully-interactive virtual environments that can train roadside crews in activities such as bridge inspection and work zone setup—giving them hands-on experience from the comfort and safety of their own “home.”
“VR and AR as an immersive computing and visualization technology has seen explosive development, but there is a lack of understanding of how today’s VR and AR technologies mesh with the current transportation needs of infrastructure management,” Gong said. “How can we help transportation stakeholders achieve better understanding of the uses of cutting-edge AR and VR technologies?”
Users are able to set up easily thanks to the mobile equipment. Photo ©Yi Yu.
Using HTC Vive controllers and headset, and a mobile backpack VR computing system, users are able to set up in their office and take on a variety of simulated job tasks.
For example, construction crews can practice putting down cones to set up their work zone, he said. The cars will interact with the cones accordingly, so users can actually get an idea of what setups work and what do not in real life. Most variables are customizable too. In that same situation, a user can customize car speed, visibility, and even make vehicles drive recklessly if they want to.
“Basically, different transportation agencies or labor unions have different needs,” Gong explained. “By making our environments adaptable we are able to offer stronger training modules that are more aligned with what these workers will be facing at the job site.”
The controls make movements feel like real life too. To pick up a cone in the simulation, users can either physically walk to one a short distance away, or use the controller to aim and point to where they want to go. In order to pick up a cone, stop sign, or other realistic object, users must physically reach out to their target, press the trigger on the controller, and pull it up.
Safety is an important aspect of these projects, Gong said. And, due to the nature of these jobs, safety is a priority of the transportation workforce too. Having another driver speed by you in a tight spot on a congested road can be a nerve-wracking experience. Now imagine that standing upright without the protection of your own vehicle—a daily reality for the thousands of roadside workers who call this type of work place home.
“Safety would definitely be the first item on the list and also the cost efficiency,” Gong said about what stakeholders are interested in.
The numbers agree. According to the Federal Highway Administration (FHWA), work zone fatalities are on the rise.
Gong found that, from getting direct feedback on past projects, challenges with work zones on busy urban highways are common issues that stakeholders face. Bridges in particular are an environment of high interest due to the fact that they often represent the bottleneck in transportation systems.
So, for the past eight months he and his team have been developing these locations to be used for anything from bridge-inspection simulations to workforce training. Because of its flexibility, Gong believes VR technology will play a role in the future of the transportation industry.
“Looking into the future, we need to really think about how we can use this virtual reality environment—it’s much more immersive than traditional training, high-fidelity, and you can play a lot of what-if scenarios,” he said.
How It Works: Living in a Digital World
The Stan Musial Veterans Memorial Bridge
The first VR environment Gong developed is a bridge modeled after the Stan Musial Veterans Memorial Bridge, which crosses the Mississippi River—linking St. Clair County, Illinois, and the city of St. Louis, Missouri.
He considered a major bridge for one of the virtual environments because they are important pieces of infrastructure that are common, particularly in the Northeast region, and a bottleneck effect happens on bridges as traffic loads converge.
“Imagine if something happens at a work zone on the George Washington Bridge that closes a lane or slows down traffic,” he said. “That would create a very congested environment with different traffic loads converging into one and a lot of vehicles trying to move around the work zone.”
Traffic type, speed, and behavior can all be controlled. Photo ©Jie Gong.
Preparing workers for dealing with this type of environment can improve safety and save time and money, Gong said.
The geometry model of this bridge was developed in one of the Building Information Modeling classes that Gong teaches, and serves many roles—one being bridge-inspection simulations.
Users have the option to change the bridge settings, so they can simulate cracks in the bridge, as well as different types and appearances that can then be used for training purposes. With these inspections typically comes lane closures, he said. Users are then able to simulate setting up a work zone for their specific needs within the digital environment.
The programmable nature of the environment allows for different users to get different experiences out of the system as everything from weather to traffic types can be implemented, allowing crews to experience a variety of roadside situations.
“The virtual training environments can greatly improve the efficiency and effectiveness of training for the transportation workforce,” he said. “Construction crews can use the bridge VR environment to simulate the setup of work zones and experience the traffic flow in a fully immersive environment.”
Critical Facility Training
The second environment is modeled after an in-service pump station. It serves as a practice and training area for when a job site enters a critical state—when something goes wrong.
Gong said the station is three stories deep and represents a naturally confined space, so users can simulate how to act and evacuate a work zone if a gas leak, fire, or other accident were to occur.
“We have this air quality sensor in the simulation that beeps and sends warnings, but once you hear the warning, you have to determine what is the proper way to get out,” he said as an example.
Users can test how long it takes to get out of the unsafe environment, reactions to warning signals, and other variables. Gong said that he has been working closely with various infrastructure owners that are interested in this environment as these are problems that they have to consider.
What’s Next?
A goal of these projects was to demonstrate the value of VR to the transportation world, particularly when it comes to workforce training, Gong said.
Currently, several companies and labor unions have expressed interest in using or funding further development of these models for their own training purposes and conversations are ongoing, he said.
He hopes that through these projects and talking with stakeholders, additional work can be done to develop VR technology in this space, and to encourage wider acceptance of VR and AR technology into the life-cycle management of transportation infrastructure systems.
“The traditional way of project data analysis and training methods often fall short of helping decision makers grasp the true complexity of many construction and maintenance tasks,” he said.
One of the most effective methods of reaching stakeholders, Gong said, has been through reaching out to them, hosting demonstrations, and showing off the technology.
“What I have found to be very effective always is the visual part,” Gong said. “When you can visualize it, you naturally have more buy-in and a better understanding of the product, so I like bringing stakeholders into my lab to put the goggles on and try it out. They all really seem to love it.”
