Announcement

Collapse
No announcement yet.

Bad Software/Automation Design implicated in Canadian Military Crash

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Bad Software/Automation Design implicated in Canadian Military Crash

    https://www.cbc.ca/news/canada/nova-...port-1.6080877

  • #2
    "The report also recommended the military consider an engineering change "to automatically disengage the flight director under certain conditions, such as when the flight director is overridden in multiple axes, or for an extended period of time."

    i'm probably wrong but i thought that was the default. at least in commercial birds, guess not.

    death by automation.....

    Comment


    • #3
      Originally posted by TeeVee View Post
      "The report also recommended the military consider an engineering change "to automatically disengage the flight director under certain conditions, such as when the flight director is overridden in multiple axes, or for an extended period of time."

      i'm probably wrong but i thought that was the default. at least in commercial birds, guess not.

      death by automation.....
      [The following applies to airplanes but I don't see why it would be much different in helicopters]

      As far as I know, the flight director is not disengaged when overridden, the autopilot is.
      Supper brief and likely not supper accurate explanation:
      The knobs, buttons and dials that are used to select altitude, climb rate, heading, NAV mode, etc, and that we normally call "the autopilot", is not the autopilot. It is the MCP (mode control panel) where you tell the plane what you want to do. All these selections will do nothing unless the flight director is engaged.
      The flight director is a system that will tell the pilot (human or auto) what they need to do in pitch and bank to achieve what you want to do as selected in the MCP. The flight director tells the pilot what to do via a "V bar" or a set of cross bars in the attitude indicator. For example, if you select ILS and you use the yoke to adjust the attitude as to keep the V-bar or cross bars centered, you will be following the localizer and glide slope without needing to look at the localizer and glide lope indications in the ILS instrument. The flight director is looking at that for you and telling you what to do.
      If the autopilot is on, then the autopilot just follows the commands of the flight director (i.e. it keeps the v-bar or cross bars centered). If the flight director is off, then the autopilot cannot be engaged.
      If you selected the APPR mode in the MCP to follow the ILS, the flight director is on, and the AP is on, then the AP will keep the plane on the localizer and on the slope. If you want to go around and just start to apply nose-up inputs without changing the mode and without disengaging the autopilot, the flight director will keep giving cues to keep the ILS centered and the autopilot will fight back on you to try to follow the flight director commands, until the autopilot gives up and disengages, but the flight director will keep giving you cues to follow the ILS which you are free to ignore and do otherwise.
      Whenever you are hand flying and you don't want to follow the flight director commands for whatever reason, it is recommended that you turn off the flight director to avoid confusion.

      --- Judge what is said by the merits of what is said, not by the credentials of who said it. ---
      --- Defend what you say with arguments, not by imposing your credentials ---

      Comment


      • #4
        Originally posted by Gabriel View Post
        If you want to go around and just start to apply nose-up inputs without changing the mode and without disengaging the autopilot, the flight director will keep giving cues to keep the ILS centered and the autopilot will fight back on you to try to follow the flight director commands, until the autopilot gives up and disengages, but the flight director will keep giving you cues to follow the ILS which you are free to ignore and do otherwise.
        Which is why it's important to follow go-around procedure in order to engage the go-around FD modes.

        Comment


        • #5
          https://nationalpost.com/news/politi...with-experts?r

          "Mary (Missy) Cummings, an engineer and former U.S. navy pilot, reviewed the Flight Safety Investigation Report, the second of two reports by the military, after its release June 28. Cummings, director of the humans and autonomy lab at Duke University, called the pilot’s inability to regain control from automated software “a very serious problem.”
          “This needs to be addressed forthwith. It should be fixed, bottom line. Who bears the costs, that’s up to the lawyers to decide,” she said in a recent video interview from Durham, N.C.

          She said the automation on the aircraft is flawed. “There is known confusion for pilots, and instead of addressing this problem head on, people are trying to make excuses for either how the system is or was designed,” she said.

          What struck me is the system builds a weird backlog of commands in the flight director.

          According to the two reports’ findings, theautopilot was left on as the pilot executed a sharp turn, and as a result the software built up commands, preventing the pilot from resuming manual control at the end of his turn. The first military report — the Board of Inquiry report — referred to this accumulation of calculations from the automated software as “attitude command bias.”
          The Board of Inquiry report said these commands in the software “can accumulate to such a degree that it severely diminishes, or even exceeds,” the pilot’s ability to control the aircraft manually.

          “It wasn’t a hotdog manoeuvre,” said Cummings, a former director of the U.S. navy’s advanced autonomous rotorcraft program. “So something was wrong with the software code base, and if it were me, and I was in the Canadian military, I would stop everyone from using autopilot until I got this problem fixed.”


          ...

          Jamieson said he felt the publicly released “accumulation bias” description in the Flight Safety Investigation Report was unclear, and there wasn’t sufficient information provided on it.

          “It’s almost treated as this mysterious force that acts on the aircraft. It’s not an engineering description. We don’t have an engineering description of this error, and that concerns me very deeply,” he said.



          Comment


          • #6
            Originally posted by TeeVee View Post
            "The report also recommended the military consider an engineering change "to automatically disengage the flight director under certain conditions, such as when the flight director is overridden in multiple axes, or for an extended period of time."

            i'm probably wrong but i thought that was the default. at least in commercial birds, guess not.
            Not for the FD, at least not that I've ever seen.

            Comment


            • #7
              [Disclaimer: I am not particularly familiar with helicopters in general, how they are flown, their systems, etc.... I am mora an airplanes guy.
              Disclaimer 2: I did not read any of the official reports on this accident, I only go by what I read in the articles linked in this thread.

              I don't understand what these experts are talking about, and I mostly disagree with them. That is probably a sign that I am not understanding correctly the reality of the situation, but here it goes...

              What struck me is the system builds a weird backlog of commands in the flight director.
              I may be wrong, but I am interpreting this as the I in PID.
              PID stands for Proportional-Integral-Derivative, and it is the basic philosophy behind many automatic control systems. It works like this:
              Let's take for example sideslip as the parameter to be controlled, for which let's set a target of zero (no sideslip), and let's use rudder deflection as the control to be used for that.
              "Proportional" means a rudder deflection that is opposed to the sideslip value. For example, "the rudder deflection will be 2 degrees in the opposite direction for every degree of sideslip"
              The problem with that is that it acts like a spring. By the time the plane passes back through zero sideslip it will have a yaw rate that will make it overshoot the goal in the opposite direction, and the plane will keep oscillating. Here is where the D comes into play.
              "Derivative" means that the rudder input will oppose the derivative of the sideslip in time, that is the sideslip rate or yaw rate (they are almost the same). This will damp the oscillatory tendency of the "spring" alone. Systems like this are usually called "spring-damper" and they are effective ins stabilizing the motion, but the equilibrium point will not necessarily be the sideslip goal of zero.
              Say for example that there is some asymmetry in the plane (say a thrust asymmetry) and that it tends to fly with a sideslip (if left alone).
              The P-D system will be effective in keeping the sideslip steady, but it will not be zero sideslip. As you can see, if it were steady at zero sideslip the rudder deflection would be zero both for the P and D components, and because of the thrust asymmetry the plane will yaw away from the zero to the point where the combined moment of the full fin (with the rudder centered) plus the rudder deflection due to the P component at that point cancels the moment of the asymmetric thrust. The plane will be steadily flying at a non-zero sideslip. There is where the I comes into play.
              "Integral" mean the integral of the sideslip in time. The rudder input due to this component will keep increasing "forever"(of course there will be limits in place) to pose a non-zero sideslip in the same direction. So in the example above, the sustained sideslip will generate increasing rudder inputs that will eventually remove that sideslip (eventually can be 1 second).
              By correctly tuning the constants for the P, D and I components, you can make a system that keeps the goal value for sideslip even in situations where "external" factors tend to offset it. External factors like thrust asymmetry (THIS IS GOOD), or a pilot making inputs while the AP is on (THIS IS BAD)

              So if one hand you want the AP to keep zero sideslip, but on the other hand you want to keep non-zero sideslip with the rudder pedal (for simplicity imagine that the rudder pedal and the AP control 2 different rudder segments), you will find yourself that you need increases amount of rudder pedal to keep your desired sideslip while the AP inputs increasing amounts of its rudder segment to try to bring that sideslip to its goal of zero. Eventually, you run out of rudder travel and hence out of rudder authority.

              With that in mind, the problem here doesn't seem to be so much this system of accumulation of inputs (although the amount of "history" that is accumulated can be a factor, as well as how much weight is given to the history compared with the P and D components). The 2 main problems, in my view are:

              2- Why isn't the AP designed to disconnect after a certain amount of fight between the AP and the human pilot? The AP doesn't know if what is trying to offset the values is thrust asymmetry, icing, a weight imbalance, or the human pilot making inputs, so it cannot disconnect as soon as any fighting take place, But it should disconnect well short of the point where it leaves the pilot without enough authority left.
              1- WHY THE HECK ARE PILOTS INTENTIONALLY MANUALLY FLYING WITH THE AUTOPILOT ON?!?!?!??!?!? It is very well established that at least one pilot and not more than one pilot should be manipulating the controls at each time, and that includes all pilots no matter if human or automatic.

              And why aren't the experts talking about THESE 2 issues? (instead of or in addition to the commands backlog)

              --- Judge what is said by the merits of what is said, not by the credentials of who said it. ---
              --- Defend what you say with arguments, not by imposing your credentials ---

              Comment


              • #8
                I think the issue is probably WRT to the use of the tail rotor to change the heading of the helicopter. If he was giving tail rotor input to spin the bird 180degrees, then he gives an order to fly forward back towards his point of origin and the autopilot starts rotating the craft 180degrees to restore original heading while flying forward, that is going to cause a lot of problems. I'm guessing here too.

                Comment


                • #9
                  Originally posted by Gabriel View Post
                  2- Why isn't the AP designed to disconnect after a certain amount of fight between the AP and the human pilot? The AP doesn't know if what is trying to offset the values is thrust asymmetry, icing, a weight imbalance, or the human pilot making inputs, so it cannot disconnect as soon as any fighting take place, But it should disconnect well short of the point where it leaves the pilot without enough authority left.
                  I don't see any reason why the AP shouldn't be able to monitor manual control surface inputs. Even the relatively primitive CWS mode on the 737 has a breakout threshold regarding pilot commands. The most infamous example would be the Aeroflot A310 that crashed when the pilot let his son "fly" the plane while on AP and his son's aggressive inputs caused it to disconnect.


                  1- WHY THE HECK ARE PILOTS INTENTIONALLY MANUALLY FLYING WITH THE AUTOPILOT ON?!?!?!??!?!? It is very well established that at least one pilot and not more than one pilot should be manipulating the controls at each time, and that includes all pilots no matter if human or automatic.
                  In the 1960's, blended automation was seen as the future, where pilot and computer would fly the aircraft simultaneously with layered authority. The pilots could make occasional manual inputs on top of the autopilot. Control Wheel Steering Modes. It must have looked good on paper but it led to so many incidents and accidents that is was mostly phased out by the end of the 20th century. Human factors and sub-par training made blended automation one of the most dangerous regimes of autoflight.

                  Comment


                  • #10
                    Originally posted by Evan View Post
                    I see any reason why the AP shouldn't be able to monitor manual control surface inputs. Even the relatively primitive CWS mode on the 737 has a breakout threshold regarding pilot commands. The most infamous example would be the Aeroflot A310 that crashed when the pilot let his son "fly" the plane while on AP and his son's aggressive inputs caused it to disconnect.
                    I don't get your point.

                    In the 1960's, blended automation was seen as the future, where pilot and computer would fly the aircraft simultaneously with layered authority. The pilots could make occasional manual inputs on top of the autopilot. Control Wheel Steering Modes. It must have looked good on paper but it led to so many incidents and accidents that is was mostly phased out by the end of the 20th century. Human factors and sub-par training made blended automation one of the most dangerous regimes of autoflight.
                    I still don't get your point. CWS steering is not autoflight or "blended automation" but rather flight by wire (sort of). The airplane will hold the pitch established by using the yoke. It is a sort of pitch-rate-on-stick / roll-rate-on-stick manual flight mode (except that if you stop the roll inputs with the wings at 6 degrees of bank or less it will interpret that you wanted to level off and the system will level the wings and keep them centered).

                    I know that in some models of the 737, in addition to the "normal" CWS, the CWS mode can also be used with the AP on, and not only that but the CWS will be engaged if the AP is on and you make control inputs. I don't understand that, and it seems like a very bad idea to me. If you want CWS (with or without autopilot) the pilot should be required to make an action that unmistakable reflects his intent to summon CWS. A command input can be easily done by accident and as far as I know there is no conspicuous annunciation of that, since the AP remains engaged the AP disengage warning will not sound, the only sign will be some small CWS P - CWS R annunciations under the FMA.

                    --- Judge what is said by the merits of what is said, not by the credentials of who said it. ---
                    --- Defend what you say with arguments, not by imposing your credentials ---

                    Comment


                    • #11
                      Originally posted by Gabriel View Post
                      I don't get your point.
                      My point is that modern automation can definitely know whether a force is due to the human pilot making inputs.

                      I still don't get your point. CWS steering is not autoflight or "blended automation" but rather flight by wire (sort of).
                      Interesting that you say that. CWS is indeed FBW on the 737. If I remember correctly from the deep dive I did into the complexities of the elevator systems, CWS inputs are commanded from a traducer attached to the column under the cockpit floor and sent by wire to the electrical autopilot servo where the commands are executed (column force is provided via the artificial feel unit). But it is a subset of the autoflight system and was intended for the purpose of allowing pilots to smooth out janky autopilot transitions or rolicking AP behaviors during turbulence. It was provided as a means for the pilots to perfect the autoflight with manual inputs as needed. Certainly, it can be used with the AP disengaged as a sort of slacker manual piloting technique but that wasn't the vision behind it.

                      Unlike Airbus FBW, CWS lacks envelope protections, making it potentially dangerous if abused. From what I can tell, most 737 pilots rarely, if ever, use it and some operators have removed it altogether.

                      I know that in some models of the 737, in addition to the "normal" CWS, the CWS mode can also be used with the AP on, and not only that but the CWS will be engaged if the AP is on and you make control inputs. I don't understand that, and it seems like a very bad idea to me. If you want CWS (with or without autopilot) the pilot should be required to make an action that unmistakable reflects his intent to summon CWS. A command input can be easily done by accident and as far as I know there is no conspicuous annunciation of that, since the AP remains engaged the AP disengage warning will not sound, the only sign will be some small CWS P - CWS R annunciations under the FMA.
                      You will get a mode change on the FMA, also a light on the MCP, so, it you are a PM doing your job, you would see that. Remember this is Boeing: faith in the pilot. There are two CWS modes, one for pitch and one for roll, and moving controls in only one axis leaves the other in CMD. But, as I said, I think, because we can't place this much faith is all those pilots, CWS (and most blended automation regimes) are not a good idea after all.

                      The 'aircraft of the future' such as the Tristar and the MD-11 had more sophisticated CWS systems. Again, the idea was to allow pilots to add their skills to autoflight, not to be lazy in manual flight.

                      I think it's telling that Airbus FBW has no such features. If you move the sidestick beyond breakout, the AP disengages.

                      Comment


                      • #12
                        what's it doing now? oh wait! that was me!

                        Comment


                        • #13
                          I still don't get it. Explain to me what happens if you have the AP in pitch set up to climb to 10000 ft at 2000 fpm and then do a CWS input in pitch (let's say a little bit of nose-down input) and then release it after a few seconds,

                          --- Judge what is said by the merits of what is said, not by the credentials of who said it. ---
                          --- Defend what you say with arguments, not by imposing your credentials ---

                          Comment


                          • #14
                            Originally posted by Gabriel View Post
                            I still don't get it. Explain to me what happens if you have the AP in pitch set up to climb to 10000 ft at 2000 fpm and then do a CWS input in pitch (let's say a little bit of nose-down input) and then release it after a few seconds,
                            CWS systems vary by aircraft. Are we talking about the 737? How much nose down input? Firstly, the column force must exceed the breakout force to engage CWS. Simply leaning a bit on the column isn't going to do it. The force is going to be more than normal ops. Once that force is met, the pitch mode will become CWS-P. The AP will hold the commanded pitch from the time the yoke is released. The roll remains in CMD, HDG SEL, VNAV, etc. Depending on the mode, the AT should decrease thrust. The pitch mode can return to VNAV or whatever via the MCP.

                            But again, this is not what CWS is intended for. A more realistic scenario is that, as you approach 10,000, you would enter CWS in pitch to manually soften the acquisition. But, as I said before, I think it is mostly used for things like turbulence and joining a radial or LOC where the autopilot itself might cause some more abrupt manuevers.

                            Comment


                            • #15
                              BTW: I'd be interested to hear from Kent or Bobby if they had any experience with CWS. I think they both flew the 747-1xx, which, I think, had CWS (didn't Bobby also do a stint in the 737?) . I don't think anything Douglas had it until the DC-10. Probably not a feature on the Hawker 1000 or Citation X or Lear either.

                              Comment

                              Working...
                              X