Reliable engineering
takes many forms

This is the second part of a three part series on the history of Autonomous Supervision, Albatroz Engenharia solution for UAS inspection of power lines. The story will be told from a human and systems point of view with some basic engineering notions. This episode covers the period from 2012 to 2016.

The initial context

In early 2012, the participation in the AIRTICI project (see previous newsletter) had led to the conclusion that an opportunity existed for a robotic flying system to inspect power lines due to a) the fatigue of human pilots, b) the hazardousness for crews and c) the overall impact on the environment; the latter is more apparent for lower voltage lines that are closer to the ground, thus affecting nature and people on the countryside.

At the same time, the experience of flying Uncrewed Aerial Systems (UAS or UAV) within the visual line of sight put forward the need for a approach different than AIRTICI as each flight surveyed only about 1 to 2km maximum (0,63 to 1,26 miles) and then it was necessary to bring the aircraft back to the ground and move the whole team and gear further along the line. The speed of progress was determined by the speed of the people on the ground, not the aircraft in the air.

Flying beyond the visual line of sight added new challenges on pilot perception, communication delays, loss of communications, loss of environmental context, etc.. The opportunity for autonomous systems emerged from the expected technical progress: visual line of sight (VLOS) » beyond visual line of sight (BVLOS) » Autonomous Operation.

Autonomous operation is still under human supervision; its main difference is that in the absence of instructions, the aircraft’s auto-pilot follows an established procedure. Formally, even VLOS operations have some degree of autonomy: if there is no link from the aircraft to the pilot on ground, the auto-pilot embarked in the aircraft knows how to act: usually, it tries to return to an initial location and then hovers; if energy is low, it lands; if during the flight the auto-pilot estimates that the remaining energy is no longer sufficient to bring the aircraft to its original location, it tries to land as soon as possible.

Changing from “Remote Piloted Aircraft Systems” (RPAS) where humans are permanently in the pilot loop to UAS where a computer program must handle all occurrences shifts the emphasis of design and regulation. In RPAS, the computer handles known systems (real time control) and humans handle the ambiguity of nature; so regulation focuses on how humans perceive and react. With autonomous UAS, regulation focuses on how trustworthy computer programs are to understand and respond to ambiguity in nature in order to minimise the impact on people and the environment.

Fifteen years ago, regulation was still emerging in the European Union. A consortium including Albatroz Engenharia submitted a project for an autonomous UAS that was rejected by the EU, one of the reasons being “there was no clear path to the market”. The first Portuguese regulation dated December 2016 and EU Rules were applied from 2020. First generation regulations postponed or banned autonomous operations.

Engaging on a study of an autonomous system for power line maintenance inspections in 2012 was an R&D effort that a) could fail; and, in case feasibility is presumed, b) it remains to be shown that it operates reliably in real life, then that c) there is a business case to bring it to the market and, finally d) that the regulation will embrace it.

Engaging in a PhD program means three years to clear the first hurdle and three more hurdles to go. It was a major commitment for Albatroz Engenharia, who was barely 6 years old at the time and for Ms. Sandra Antunes, the aeronautical engineer in the team that took the banner and enrolled as a PhD student at Universidade da Beira Interior, her alma mater.

Framing the problem: the analogy with human pilots

The approach to autonomous UAS built from the experience with helicopters, where robotic systems inform the crew about emerging hazards in real time. This approach puts safety above performance which is likely to instil confidence in prospective users – also in the R&D engineers involved! -, and comfort in regulators.

As explained in April, the three variables used are DVG = Distance Vehicle to Ground (the same as AGL for helicopters), DVL = Distance Vehicle to Line (the same as DHL for helicopters) and VTG = Velocity to Ground that replaces IAS = Indicated Air Speed at typical inspection speeds:

If the robotic pilot could keep the UAS within a safe distance of the power line, the safety part of the problem would be solved. On the other hand, if the robotic pilot could keep the UAS close enough to the power line and the surrounding features, then a quality inspection could be carried. This would be represented by Figure 2 and all variables are expressed in local coordinate systems. In case of failure of global navigation systems, the UAS still performs in a safe manner using the local mode, at least as long as there is line to follow.

However, there is the problem of the initial condition: to keep the UAS within the “green volume, it has to get in there in the first place.

A second mode is necessary to supply a coarse definition of the line trajectory, where each angle is a waypoint. In principle, only angle towers are required, however, most asset management systems have global coordinates for nearly all towers (sometimes with some error) and the UAS should travel towards those coordinates as a typical waypoint flight (one of the earliest semi-autonomous behaviour implemented in most UAS).

However, just like the local-navigation mode must combine safety and performance, this global-navigation mode should do likewise; a set of boundaries in global coordinates are computed and the UAS is supposed to fly within the safe volume standing side by side the corridor formed by PT1 to PT3 representing towers of a typical 3-phase AC line.

In case the navigation in local coordinates fails, the global navigation mode keeps the UAS following the power line in a safe manner.

The need for simplifications

It is known that real world systems don’t operate under ideal conditions and yet people learn the science and equations for “ideal systems”. Galileo’s talent glowed when he devised the mechanical laws for frictionless motion while experiencing motion with friction only. The same goes for modern science and engineering. In addition to the many “frictions” common to mobile robotics, it was necessary to introduce two major simplifications to advance in R&D.

The first simplification is the orientation of the UAS relative to the cable. In terms of the local mode, DVL is a scalar and has no orientation (see Figure 1), so the system could be anywhere in a given radius around the cable. In reality this is not true: over-head lines are usually a geometric arrangement of multiple conductors (often 5 or 6 wires or bundles and up to 14 in some cases). Moreover, the guidelines for inspections change across countries, voltage levels and operators: sometimes power lines should be inspected at level with the lowest phase conductor, or at the middle of the height of conductors, or the highest phase conductor or the highest ground wire (above all) or even above that. So, it was decided that DVL would represent the horizontal distance to a vertical plane defined by the closest conductor.

The second simplification was the removal of obstacles within the flying path. It is assumed that aircraft flies within the Right-Of-Way [ROW] along the line so, it should not be an eccentric assumption; however, it is known that obstacles infringe in the ROW (that’s one of the reasons for inspection) and, more subtly, there are line crossings and branches coming from the line subject to inspection. Including these unexpected ambiguities would obfuscate the R&D involved in the demonstration of feasibility.

The issues behind these simplifications were not forgotten, they were suspended for emphasise the R&D contribution and should be restored later in the real-world implementation, in case feasibility is established.

Real-time control architecture

One of the reasons quadcopters are so popular is that they feature only four controls on four motors, which are all equal and the airframe is symmetrical or nearly so. Control engineers – who like simplifications – are very fond of this kind of aircraft since no matter the complexity of the intended movement or aircraft attitude, everything is translated as a velocity (or position) of each motor. A real-time control of an autonomous UAS of the quadcopter type requires handling input data from the motors and issuing instructions hundreds of times per second to each of these motors.

Translating a high level representation of DVL, DVG and VTG in local measurements, combined with a corridor of a “virtual fence” defined by vertexes in global coordinates into real-time control involves three main modules shown in Figure 4:

1. a “Control” module representing the dynamics of the aircraft translating sequences of incremental movements and rotations into instructions to the motors and monitoring the real-time reaction of the systems (reacting to wind, for instance);
2. a “Supervisor” to understand the scenario, the operation and issue the high level commands that will be translated into incremental commands and
3. A“surveillance” mode that connects with the human pilots on the ground, so that they can intervene if necessary.

The “Inspection System” condensates all engineering inherited from Power Line Maintenance Inspection for helicopters. All other modules were created for the UAS.

The most challenging module is the real-time “Control”. Dynamic models of commercial UAS are not available beyond their manufacturers and it becomes necessary to extract the main dynamic parameters to create an inverse model from outputs to inputs– and again to make several simplifications – to establish the equations that control the UAS efficiently and smoothly. This process must be repeated each time the aircraft changes, which means not only the aircraft itself but also the payloads and accessories (this calls for a lot of “simplifications”). In the end, even if the robustness of the controller is proved mathematically, the complexity of the analysis and the differences between the “ideal” control and real world make it a challenge to demonstrate compliance to the regulators.

The Supervision module operates at a few loops per second since distances and speeds change much slower than the motor rotations and it features a well-established procedure, also inherited from helicopter inspections.

Finally, the Surveillance mode isn’t directly involved in the control regularly: it issues notifications and shows the remote pilots how the inspection is going and only in case of instructions are sent (e.g.: suspend inspection, rotate camera, return home), it transfers them to the other modules.

An R&D success

It took longer than expected but Ms. Sandra Antunes effectively showed in November 2016 that it was feasible to perform a power line inspection with a UAS featuring her Autonomous Supervision. Congratulations and accolades were in order.

All R&D is a foraging into the unknown and a PhD thesis is a structured exploration for three years or more so Albatroz Engenhaira and Ms. Sandra Antunes knew what they signed for. However, a PhD based on real gear, real procurement and real environment adds new layers of difficulties that might diminish the scientific contribution as most of the effort is spent on issues that matter little to R&D. It was critical that each member of the team involved pulled the issues (s)he is most competent at, relieving all other from those issues so they could also concentrate on their strong points.

Conclusions (of Part II)

In a nutshell:

1. Yes, it is possible to use an Autonomous UAS to perform power line inspections.
2. This was achieved by putting safety as the cornerstone of design.
3. The solution depends on the aircraft dynamics.

After the rejoicing of Sandra Antunes and Albatroz Engenhaira- earlier that year, the company had celebrated 10 years – the team was well aware that only the first hurdle had been passed. The run towards business success was a long one but there were encouraging external signs: most people in business already believed that UAS were part of the future – CIGRÉ conferences and companies forming around the world reflected that – and regulators were envisaging a window for specific services that involved specific risks handled by trained professionals, just like an ordinary air service company. After all, these are aircraft, just like airliners, only smaller and quieter.

Internally, the team was aware of how many issues had been put aside the get the valuable R&D results. Now it was necessary to create a new path that revisits and solves each one of them and to be prepared for the unforeseen obstacles still ahead.

Note: all illustrations are authored by Ms. Sandra Antunes.

One picture per month

A look behind the curtains to discover some significant memories from Albatroz Engenharia’s archives.

The Intern

Who came to office during CoViD-19 lockdowns? The interns, naturally.

During the COVID-19 lockdowns, when the Lisbon office sat empty for weeks, only occasional visits were made to maintain plants, an aquarium, and remotely used computers. In this quiet environment, curious neighbourhood cats discovered small openings in the windows and began slipping inside.

Drawn by the stillness, the hum and warmth of machines, and the aquarium’s fish, they became unexpected and regular “interns.” Amused by their visits, the team set up cameras to capture their presence, even documenting some of the most frequent feline guests, such as this young female intern.

Even after the lockdowns ended, these “interns” still return from time to time, lingering nostalgically outside the window in nostalgia of the yester-weeks, when the office belonged entirely to them, with no humans to mess around.