Inspections provide a snapshot of a bridge’s condition, but could artificial intelligence improve understanding of deterioration rates and hold the key to revolutionising inspection regimes?
National Highways has a moonshot ambition to manage key bridge structures without human intervention – and it hopes to get there using artificial intelligence (AI), academic research and, possibly, robot dogs.
At first glance, the ambition may sound far fetched, but England’s major roads operator recognises that getting to that point requires a real change in bridge condition knowledge. Gaining this insight is the driving force behind research currently being undertaken at the University of Nottingham and jointly funded by National Highways, WSP and the Engineering & Physical Sciences Research Council.
Making the right decisions
“Essentially managing bridges is about making the right decisions,” says WSP head of civil, bridge and ground engineering Steve Denton.
“It’s about selecting how and when to intervene so that safety is maintained and the best possible outcome is achieved.
“If a structure deteriorates too much, then less invasive, simpler repair techniques are no longer technically possible. But if you intervene too early, then it’s inefficient.
“So to make the decisions well, you need to understand not just the state of a structure at a point in time, but you want to understand how that condition is changing.
“Today our inspection processes are focused more towards a snapshot in time. I think there are really exciting possibilities from digital technologies that can help us understand how condition is changing.”
National Highways wants to get a point where there are no unplanned bridge closures. For National Highways head of highway structures Peter Hill, monitoring condition and understanding gradual deterioration is key to that. He believes that the tool being developed by the research will enable that, as well as drive more consistency in decisions.
Reflecting on the organisation’s moonshot aspiration, Hill recognises that it is some way off achieving that but says it is on the journey.
“Even once the tool is developed, we expect to use it alongside the conventional approach to inspection for some time in order to build confidence and understand the reliability before moving to solely relying on them,” says Hill.
The idea for the project came from University of Nottingham PhD student Julia Bush, who has a background in bridge inspections through previous roles with Network Rail.
There are really exciting possibilities from digital technologies that can help us understand how condition is changing
“I’ve been on the receiving end of trying to make sense of bridge inspection reports and trying to make load assessments from them,” she explains. “I empathise with the people undertaking this work and my motivation is to make it easier.”
Conversations started in autumn 2020 and Bush started work on the three year project at the start of December last year.
She believes that the work is the first industry implementation of AI for bridge inspections.
One year on and the research on the Digital Technologies for Bridge Inspections project has moved at considerable pace. Bush already has 25,000 images for use as data training sets for the AI tool and is open to adding others from other infrastructure owners.
We are using deep neural networks for classification of defects – something as simple as a photograph can be used for this
“We are using deep neural networks for classification of defects – something as simple as a photograph can be used for this – and we’re also looking to take this further and seek deep neural network applications to delineate the extent of the defect. Again, this can come from a photograph or point clouds of data that increasingly come from surveying techniques.”
A neural network is an interconnected group of nodes inspired by a simplification of neurons in a brain. For this project the network is trained on datasets of images depicting various surface blemishes, with corresponding ground truth labels, created by Bush, such as “crack”, “spalling” and “exposed reinforcement”.
Bush says: “It learns to abstract visual features from the training images and to associate these with a particular class label. Once trained, the neural network can be used to make predictions for which class any previously unseen – inference – image is likely to belong to.
Trained deep neural networks
“A trained deep neural network will take an image and the output will be a prediction of what it thinks it can see on the image.”
The work has started with training on cracking and spalling defects. “We are using visual interoperability methods so once the deep neural network has identified a crack, we then ask it to identify the pixels it can see the defect in and identify the pixels that it relied on the most to identify the defect,” explains Bush.
“The result is essentially a heat map that verifies what has been used to make the decisions.”
Bush is also asking the deep neural network how certain it is about its predictions.
Once she has trained it to reliably identify defects, she can then start asking it to compare correlating images taken at different times and quantify the change between the images.
The use of images for the tool creates simplicity and, in future those images could be taken from safe points close to a structure and the tool can use data analysis to remove the effect of images taken from different angles. There is also the possibility that these images can be gathered from moving vehicles or drones.
Hill adds that robot dogs, like one developed by Boston Dynamics, creates the potential to remove the need for inspectors to enter confined spaces for inspections.
“Instead, the dog could walk through the confined space and take images that feed into the tool,” he says.
Despite the progress, Denton says that are still lots of unknowns and “there will also be blind alleys” during the research. But he adds that it is important to explore those.
Bush says that the safety critical element of the work is at the front of her mind as she carries out the research and says certainty is essential.
“My biggest challenge is to quantify how reliable this is and how to incrementally introduce it without taking risks,” she says. “There are technical challenges, of course, because this is cutting edge emerging technology and most of it is far from being commercialised.”
This is cutting edge emerging technology and most of it is far from being commercialised
Next steps for the research will be introducing National Highways bridge inspectors to the concept. This was expected to start as this issue of NCE went to press.
Over the next year, Bush will be working on the delineation of the extent of the defects being detected so that she can start designing metrics to start comparing a defect over different points of time.
“We need to be able to do this in a consistent and reliable way to feed into maintenance schedules,” she says.
As for when the tool will be ready for trials alongside conventional inspections, Bush is focusing on completing the research by May 2024, so the next stage of National Highways moonshot is not too far off.
Tackling Queensferry Bridge’s ice problem
Transport Scotland hopes that this winter a deep clean of the Queensferry Crossing cables will prevent ice forming on them and then crashing onto the road.
The four year old bridge has been closed three times in the last two years due to ice falling from the cables.
Sensors fitted to the bridge in 2020 now alert Transport Scotland’s road maintenance contractor Bear Scotland to the issue, but it is hoped that cleaning the cables will prevent further problems.
“The first cables were installed in 2015 and there are no records of any ice forming on them until 2019,” says Bear Scotland unit bridges manager for south east Scotland Chris Tracey.
“Since the Queensferry Crossing opened to traffic, dust and dirt has accumulated on the cables. These tiny particles may be helping ice to accrete as crystals form around them.”
This autumn, rope access teams have been working on the bridge to clean 24km of the high density polyethylene sheaths that encase the steel strand cables using soap and water.
“By cleaning the cables on one tower we will be able to measure the impact this has,” says Tracey.
“As part of the project, thermal cameras are being installed at the top of each tower to monitor and measure any formation of ice.”
Tracey says that work is underway to design equipment to mechanise the cleaning process.
In parallel to the work on site, tests are also being undertaken at the Scientific & Technical Centre for Building in Nantes, France.
The research facility can replicate all kinds of weather conditions and will enable the Bear Scotland team to test the impact of cleaning, as well as specialised coatings and deicing compounds, on a full-size section of Queensferry Crossing cable.
“The ultimate aim is to design measures to mitigate or prevent the problem,” adds Tracey.
Like what you’ve read? To receive New Civil Engineer’s daily and weekly newsletters click here.