Architecture is one of the last major professions yet to be swept up in the digital tide. There are some advanced design tools commonly available to civil engineers, but advanced imaging tools are rarely made use of. The latter tend to be still too expensive for many local construction companies, and they remain almost completely out of reach for the types of organizations that do inspections, renovations, or preservation work.

Following the blaze at Notre Dame de Paris, attention naturally turned toward efforts to rebuild, and that led to the realization that an enormous amount of information had already been collected on the cathedral, a volume of data so enormous it surprised even some of the experts involved in both the imaging and the civil engineering fields. Also fascinating were the range of new tools that might be volunteered for the reconstruction, including lidar, thermal imaging, holographic interferometry, and x-ray equipment. The discussion also included talk of robots and airborne drones, upon which sensors can be mounted, that can get to places that before were impractical or impossible to reach otherwise.


Recommended
Notre Dame's 3D Salvation


But Notre Dame is different. It is exceedingly well-known, situated in one of the most popular tourist destinations in the world, in a politically stable country, and treasured by people around the world. By way of contrast, the minaret of the Great Mosque of Aleppo, a World Heritage Site, was destroyed in 2013, an architectural casualty of the ongoing war in Syria. Detailed scans didn’t already exist, the destruction was far less publicized; the response far less generous. Few would argue Notre Dame doesn’t merit the attention it’s been getting all along, or that it doesn’t deserve the resources sent its way following the fire; the point is that it is subject to rare treatment.

The response to the fire at Notre Dame also distorts the view of what’s typical in civil engineering. There are few if any digital tools specifically for use in construction. There are few if any dedicated tools for inspecting structures, during construction or after. There are few if any dedicated tools for any type of non-visual evaluations.

Design

“Construction is one of the least digitized industries in the world,” said Nicolas Mangon, vice president of Architecture, Engineering and Construction (AEC) Strategy & Marketing at Autodesk. “You’re only beginning to be able to do now on an iPad or iPhone what you used to do on paper.”

When it comes to architecture, the one aspect of the job that is somewhat digitized is the design process.

Mechanical engineering was one of the first disciplines to get computerized design tools. Computer-aided design (CAD) systems started to become common in the 1970s and early ‘80s, rapidly followed by computer-aided engineering (CAE) products.

Civil engineers saw some benefits from CAD at the time, but products optimized specifically for the various categories within the profession dribbled out later. It started with tools for architects, followed by tools for engineers, and now there are tools becoming available for the construction process, Mangon said.

Modern software can not only create blueprints but also generate models that take into account the performance characteristics of common building materials. Builders can simulate a variety of additional factors such as the effects of wind on a structure, occupancy, and HVAC (heating, ventilation and cooling) systems, he noted.

It is becoming possible to scan an existing structure, and import the billions of points of scan data into these advanced tools. The aim is to use the scan to do more than just create 3-D images. Each point in the scan should be connected to information that goes beyond that point’s physical location in relation to all the other points – for example, the nature of the material represented by that point. Is it glass? Marble? Steel? Now civil engineers can begin to generate models of existing structures they can use for simulations.

As the data sets get bigger and more accessible, that knowledge base can become invaluable. “Say you want to build a new hospital in Portland,” Mangon said. “The architect ordinarily might say ‘I’ll build one just like the one I built 10 years ago.’ But now, with new software available, they can explore possibilities they might not have been able to explore before. They might be able to go through millions of options, instead of just the last two they just built.”

These tools are also able to input other types of information – seismic data, thermal imaging data, and more, Mangon said.

The next step is to get all the information available over to the construction site and use it in the construction process. Plans sometimes have to be changed in response to on-site conditions; that could be done on-site. Sensors could monitor stores of building supplies for inventory purposes; is the site low on wallboard? Order more. Are humidity levels soaring? Take extra measures to avoid mold.

Real-time 3-D modeling of the construction in progress could be used for safety or efficiency. Construction workers could be alerted when they’re in proximity to some hazard that ought to be avoided, or directed to a nearer cache of some particular supply.

Falcon UAV

Al-Balad is the historical neighborhood in Jeddah in Saudi Arabia. Saudi company Falconviz is using aerial drones mounted with laser scanners to map the densely-built area. The company employs photogrammetry to enhance the value of the visual data. (Source: Falconviz)

Autodesk has been pulling together the resources to do these sorts of things. In the closing months of 2018, the company dropped over $1 billion on a couple of companies that specialize in digital tools for construction. In November the company spent $875 million on PlanGrid, which in 2011 created software to display blueprints on iPads, and in December Autodesk bought BuildingConnected, which makes tools for managing the construction process, for $275 million.

Since we’re on the subject of money, this is a good time to note the different dollar amounts involved in various aspects of civil engineering.

A new hospital built in the US might cost $500 million to $700 million. The Burj Kalifa cost $1.5 billion to build and the Shanghai Tower cost $2.4 billion, according to Emporis. The new Tappan Zee Bridge in New York had a budget of $3.9 billion.

Maintenance gets short shrift; it always has, and it always will. It’s true of skyscrapers, and it’s especially true of infrastructure. It’s usually local governments that are charged with maintaining critical infrastructure, and local governments all operate on budgets many orders of magnitude smaller than the figures quoted in that last paragraph. Organizations that dedicate themselves to preserving heritage sites…? Essentially beggars.

Construction companies working on big projects have big budgets and can afford leading-edge digital tools. Whatever impression was created by the response to the fire at Notre Dame de Paris notwithstanding, hardly anybody else can.

Inspection

Structures have to be maintained. That includes not only cathedrals, office buildings, hospitals, schools, arenas and other buildings, but also roads, bridges, tunnels and other infrastructure

The US might be in the “new world,” but its infrastructure is notoriously aging and neglected. Take bridges for a particularly troubling example: the country has over 600,000 bridges and nearly one-tenth were considered structurally deficient in a 2017 report from the American Society of Civil Engineers (ASCE). That “only” 9.1 percent of US bridges are structurally deficient shouldn’t make anyone feel good about the other 91% – read the report.

If buildings and infrastructure are inspected, it’s usually a visual inspection, and it’s done by a human. If the structure is large, it’s likely someone is going to have to climb, dangle, or scurry someplace barely accessible to do a thorough job of it.

Of course, the answer is to conduct inspections using drones and robots mounted with sensors, but that’s not happening. Or not yet, at least – not when it comes to infrastructure. Industrial-grade drones and robots are still far too expensive for far too many inspection agencies. Mass-market drones might be affordable, but they are not yet steady enough for the job.

Jerome Hajjar, CDM Smith Professor and Chair, Department of Civil and Environmental Engineering, Northeastern, is working to develop systems for inspection specifically for bridges and towers and the like. That involves building algorithms for inspecting structures and interpreting the data that’s collected. Sometimes it’s possible to set up a camera or a laser scanner on a stable tripod and get all the data required.

vatican

Preservationists would love to be able to use advanced imaging systems, but many of these new tools remain financially out of reach. FLIR loaned Italy use of one of its advanced thermal imaging systems; the imager was used to evaluate the the structural integrity of some of the country's most well-known antiquities, including Michelangelo's David and these statues and structures at The Vatican. (Source: FLIR)

Other structures don’t lend themselves to ground-scanning – some bridges, for example, or tall towers. That’s where drones come in. The problem is that industrial drones are too expensive, and retail drones are rarely stable enough for the purpose. Hajjar is also investigating possibilities for drones that are both inexpensive and steady.

“What we’re trying to do is make everything 100 percent automatic. There can be a human in the loop, but we’re trying to make it as automated as possible,” he told EE Times. “We’re framing it as an assistant rather than a replacement. It’s a tool to be used in places impossible or dangerous to access, or places that need to be looked at repeatedly over time.

The intended results would include full 3D reconstructions of the geometry of anything exposed. Critical bolts, for example, might be behind a plate –not exposed and not subject to visual inspection. Still, a lot of useful information can be gleaned from visual data. The results can be fed into software systems for analysis. Part of the process will be gathering data and feeding it into machine learning systems, he said. Databases for this sort of thing still don’t exist.

Another goal is to develop heuristic algorithms based on knowledge that engineers have. “With a heuristic approach, we can develop methods to understand how bolts go through connections, and use what we can see to determine what might be happening with the beams behind connection plates. That’s how we’d end up gaining an understanding of concrete abutments, bridge decks, and other structures.” Northeastern is currently working with civil engineers; gleaning knowledge from human architects might come next.

Heuristics could be a great help with some of the basics. Mentioned earlier in this article was the possibility of taking multipoint scans and making each of those points intelligent, for example linking each point to the material it represents. “Heuristics can suggest what those materials are,” Hajjar said.

Preservation

Notre Dame received nearly $1 billion worth of pledges of support to rebuild in the space of a week. There’s no precedent for that.

The original architectural plans for most heritage sites, including Notre Dame, are lost. Historians may never know with any certainty how some structures were planned and designed. Short of that, it’s an enormous task just to identify all the individual stones in a structure, let alone determine how they were fitted together or mounted.

The fire at Notre Dame de Paris drew attention to creating visual scans of heritage sites. It turned out that the late Andrew Tallon, who had a fascination for gothic architecture in general and for Notre Dame specifically, had created a remarkably detailed laser scan of the cathedral’s structure about 10 years ago. That scan is accurate, he said at the time, to within 5 millimeters. He and his colleagues scanned between 45 and 50 gothic structures.

The question for preservationists is how to use that data.

Lisa Ackerman, Interim CEO of The World Monuments Fund (WMF), told EE Times, “The biggest change has been less in the hardware and more in the software. The equipment itself hasn’t changed a lot since Professor Tallon did his study. However, the ability to process data and work with it has improved vastly. There are more effective viewing tools; the ability to download point cloud data in 2D drawings is less labor intensive, and in general you can store data on more nimble equipment today. So the ability to harness the data is much greater than when Professor Tallon started.”

It’s a significant advantage to have detailed visualizations of these sites – there is no question about that. But you can scan all you want, and it’s going to help only so much unless you can accurately characterize what’s being scanned. Old structures were built with unique materials.

Autodesk's Mangon said, “In the case of Notre Dame, you have specific materials – the wood for the frames – those are 800-year-old trees.” How do you model those? “We’re open to anyone who has that information for databases,” Mangon noted.

Some of that historical knowledge exists; it just needs to be collected and codified. “There are people who can look at a piece of historic glass and understand its chemical composition,” Ackerman said. “They’d know things such as: they used arsenic in this glass in the US up until this certain date, but after that, arsenic wasn’t used.” (The composition of glass is pertinent to the fire at Notre Dame; renovators would like to look for damage in every piece of glass in the fabulously complicated stained glass windows.)

Visual scans can still help determine quite a bit about the integrity of a structure. Are materials still in place? Do they seem to still be where they’re supposed to be – in the same position relative to their surroundings? Photogrammetry is a more recent technique in which data miners extract detailed measurement data from photographic data.

CyArk is dedicated to imaging other world heritage sites. The organization was founded in response to the Taliban’s destruction of the Bamiyan Buddhas in Afghanistan in 2001. The organization uses a combination of laser scanning and lidar. Since its founding in 2003, CyArk has scanned over 200 sites around the world, including the Jefferson Memorial in the United States, the Madrasa al-Jaqmaqiya in Syria, and a notable complex of caverns and rock art sites in Somaliland.


Recommended
Blog: Did You Think of That?


All of which represents only a start. Tallon, as noted above, scanned 45 to 50 cathedrals in Europe. The Catholic Church says it has over 3,300 cathedrals worldwide. Add to that thousands of churches, abbeys, priories, and other types of sites that might be architecturally significant. Multiply that number by several other major religions, all with mosques, temples, crypts, statues and more, and amplify that by the thousands of secular structures of historic significance, and it’s clear how rare it is to be as well-surveyed as was Notre Dame de Paris.

The visual scans that organizations such as CyArk are creating are valuable, but they’re also literally just scratching at the surface. It would be useful to be able to do deeper scans.

Building materials are affected by so many possible circumstances and over different time periods. Over centuries materials can settle and shift. The ground upon which a structure is built can move over centuries or in minutes. Materials can respond to water and heat, wearing down over millennia, or expanding and contracting over months with the seasons or even daily in response to the sun traversing the sky. Catastrophic events – flooding, fire, war, earthquakes, and more – can have effects that are obvious but instead might be hidden from sight, perhaps within a sealed wall or perhaps within the internal structure of a block of stone or a wooden peg.

CyArk Rani ki Vav

Cyark has embarked on a mission to image the world's architectural treasures. This is a static screen capture of the model of the Rani ki Vav - the Queen's Stepwell - in India; the model on the organization's web site, here, can be manipulated. (Source: Cyark)

But those kinds of deep scans elude just about everybody in civil engineering. When asked about a wish list of technologies that would be helpful in inspections and maintenance, Hajjar said “We would like to be able to look inside a structure. Perhaps using thermal, or infrared, perhaps x ray, perhaps ground-penetrating radar. Also, it would be useful to have methods to measure strain or stress through contact, without necessarily needing to apply additional load.”

There could be applications for being able to detect radioactivity; that could be useful for inspecting nuclear power plants, for example. Climbing robots might be able to handle bigger payloads, Hajjar speculated.

Ackerman said the WMF has used ultrasound to see into a stone wall to understand conditions behind decorative elements, and has used lidar to see under the tree canopies around the Angkor Wat temple complex in Cambodia to understand the topography of the site.

But those were exceptional uses of high-tech tools in what Ackerman referred to as the heritage sector. Necessity leads to some clever solutions, however.

Fairly simple equipment for detecting vibration is available. An example of how that can be used is the instance of the Merchant’s House museum on East 4th Street in Manhattan. The museum isn’t being touched, but the building next it was torn down and a new hotel might go up next to it. Vibration detection was used to verify the structural stability of the exposed wall.

The WMF has set up humidity sensors in project sites all around the world, including Angkor Wat. “They’re the size of credit cards, and there’s no installation. You can hide them discretely, but they provide constant readings. You can store a year’s worth of data. You could see where the spikes were, and you can go back to the site and determine where the problems might be. That seems small, but it’s revolutionary. At Angkor Wat, a weather station sounds like a simplistic thing, but the ability to have the equipment and the data allows you to create plans you couldn’t a decade ago. …If you’re the person in Cambodia, charged with protecting it, you can say with certainty that x, y, and z has happened.”