Bridging disciplines for a safer commute
UND Civil Engineering Assistant Professor Sattar Dorafshan, pictured on the Sorlie Bridge in Grand Forks, is researching technology that can inspect infrastructure from above. Dorafshan employs unmanned aircraft to capture images of bridges and machine learning algorithms to analyze the resulting data. Photo by Mike Hess/UND Today.
UND civil engineer uses drones, machine learning to advance bridge and other infrastructure inspections
Think of the last time you got stuck in bridge traffic. There you were, cruising along and putting miles behind you with ease, when suddenly you caught sight of the traffic cones, bridge-inspection crew and flashing lights ahead that marked an upcoming slowdown.
Now imagine zipping over that same bridge with ease, blissfully unaware that overhead, a drone is silently conducting the inspection.
That’s UND Assistant Professor of Civil Engineering Sattar Dorafshan’s happy vision, too. So, at the College of Engineering and Mines, Dorafshan is embracing machine learning and unmanned aerial systems (UAS) to advance automation in civil and structural engineering.
If routine evaluations of bridges can be done from the air in a matter of minutes – if drones can use a variety of sensors to collect data on and below the surface – that could mean big savings, as well as fewer slow-downs on people’s commutes. This is especially true when it comes to the difference between fixing a bridge and replacing it. Multiply that difference by the 5,000 or so bridges in North Dakota that require regular inspection, and more than 600,000 bridges nationally, and you’ll understand: We could be storing a lot of traffic cones.
The new method of evaluating bridges from the air also means safer conditions – not only for drivers, but also for the ground crews who’d otherwise be testing the bridge’s surfaces a few feet from onrushing cars.
That risk and cost reduction has caught the eye of government agencies such as the North Dakota Department of Transportation (NDDOT), which is bringing Dorafshan’s research to bear.
In an interview with UND Today, Dorafshan said North Dakota could become one of the first states in the country to fully implement such an approach, should his most recent project prove fruitful.
Dorafshan worked with BDI to employ both UAS flyovers, as well as non-destructive evaluations during his summer-long project. Image courtesy of Sattar Dorafshan.
Improving inspection methodologies by means of artificial intelligence can save lives and keep traffic moving, he said.
“All of this could be implemented relatively quickly through the great support of the NDDOT and their amazing UAS Integration Pilot Program,” Dorafshan said. “We’re not there yet, but I think we can get there.”
Secrets discovered by air
On any given day this summer, drivers might have spotted Dorafshan along Interstate 29, north of Grand Forks. His research project involved five bridges on that highway, four of which passed over rivers in Walsh County.
The bridges were due for repairs, but the UND researcher wanted to collect data first. He knew it would be a unique opportunity to have findings from UAS flights validated by “ground truth.”
In other words, what repair crews would find by actually digging into a bridge deck could be proven to have been discoverable from the air.
So, what exactly can be discovered from the air?
The answer, Dorafshan explained, depends on whether you’re looking at the surface of a bridge, or within. When searching for surface defects – cracks – a visual camera mounted to a drone can suffice.
But below the surface, things get trickier. Dorafshan’s research looks at differences in electromagnetic radiation. He measures those differences using a thermal camera, and using what’s known as infrared thermography, he tries to find subsurface degradation of the steel rebar reinforcing the concrete material.
“If there is a defect beneath the surface of the deck, it changes the electromagnetic radiation of the object,” he said. “The heat emitted from the surface changes, depending on whether it’s coming from intact concrete or defective areas.”
Dorafshan worked with drone service providers to pilot drones over the bridges in question, programming the drone-mounted cameras to collect both visual and thermal data for analysis.
These are two examples of the data captured by UAS flyovers of bridges. On the left is a typical optical sensor by which surface conditions can be analyzed. On the right, thermal imaging can reveal potential trouble areas where emitted heat signatures differ between intact and defective regions of the deck. Images courtesy of Sattar Dorafshan.
Bridging disciplines to validate data
The imagery analysis is where machine learning steps in. Dorafshan’s dip into computer science involves training computer algorithms to recognize cracks, for instance, when looking for surface defects.
“Imagine that you took a picture of a surface of something that’s cracked,” he said. “Cracks manifest themselves as an edge within the image because of the shape and the difference in pixels in and outside of that shape.”
Of course, it’s one thing to have the machine recognize an “edge” in an image. It’s another for the machine also to be able to analyze that edge and its accompanying crack.
So, Dorafshan uses his civil engineering expertise to train the machine to recognize which cracks are “normal” and which are out of the ordinary, or potentially unsafe.
Sattar Dorafshan
“I need to be able to understand the computer model and modify it, so it’s basically a half-and-half collaboration,” he said. “I customize the model for my own purposes, to make it as efficient and accurate as possible in detecting cracks.”
The same type of process can be applied to the examination of thermal data. However, Dorafshan knows that at this point, most civil engineers wouldn’t be satisfied with such findings unless the findings could be weighed against more “grounded” methods of evaluation.
More traditionally, bridge crews perform what is known as “chain dragging,” which is the tried and true method of dragging large chains across the deck surface and listening for trouble spots, which are known to sound hollow or drum-like.
The examples Dorafshan saw in use this summer were impact echo and ground-penetrating radar, which are more generally known as forms of non-destructive evaluation. Both forms use waves (seismic and radar, respectively) to measure the consistency of the bridge material. And in both cases, defects or irregularities with the bridge result in an altered signal.
The intriguing aspect of the data collection project, according to Dorafshan, is having not only this more traditional bridge-measuring data, but also the conditions and characteristics eventually found by the bridge repair crew. Once Dorafshan learns which areas were actually found to contain defects, he can use that knowledge to improve his machine learning algorithm.
“You need to train a ‘classifier algorithm’ as you would your brain, based on the data that you already know the answer to,” he said. “Now I know if something identified in a visual or thermal image contained a defect in it or not.
“The complete dataset is unique, because the collected data were validated by the actual condition of the decks. Now I’m hoping to come up with newer, better artificial intelligence models.”
Despite the seemingly small sample size of five bridges, Dorafshan is confident in the wide-ranging impact of his work. Of the bridges in service today, around 60 percent of them are constructed in nearly identical fashion to those he examined along Interstate 29, he said. Photo by Mike Hess/UND Today.
Upper hand for asset management
Of course, Dorafshan’s models, of necessity, will be derived from a limited number of structures. Will the models also be applicable to other bridges?
“I’m optimistic that they will,” Dorafshan said. “At least 60 percent of bridges across the country are nearly identical in terms of the material and the construction method.”
If the data and resulting models – based on five bridges in rural North Dakota – could be applied to a majority of bridges, that would be a tremendous success, he remarked.
Dorafshan’s North Dakota Department of Transporation collaborator, T.J. Murphy, equated success to increasing safety, data accuracy and the overall understanding of UAS applications.
“The agency-university collaboration of this project represents a positive, symbiotic relationship, and research efforts to enhance our understanding of UAS increase our confidence behind the technology,” said Murphy, a research and materials engineer.
“UAS are great for investigating structures and pavements,” he continued. “They have the ability to collect similar and more extensive data without risking staff on the road, and any safety benefit is a worthwhile endeavor for the NDDOT.”
Moreover, another key feature is the automation of this entire process. In addition to its thousands of bridges, North Dakota contains enormous lengths of railroad tracks and pipelines stretching over remote areas. So, just as UAS research is paving the way for autonomous aircraft to regularly inspect the state’s infrastructure, keeping bridges certified via autonomous fly-overs could have wide-ranging effects.
“This technology gives us an upper hand in terms of managing our assets, finding defects and addressing them – that’s the most important thing,” Dorafshan said. “If we can easily collect data, we can better monitor deteriorating regions and develop better management plans.
“Considering bridges alone, it could mean billions saved nationally in maintenance and asset management costs.”
