Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Software News

Software Upgrade Crashes UK Air Traffic Control System 233

Posted by michael from the quality-is-priority-1 dept.
pitpe writes "Earlier today the computer system controlling most of the UK's airspace failed, after tests in preparation for an upgrade failed. The original failure occurred at the West Drayton centre, which is an old (70's) system, as opposed to the new system at Swanage, which has had its own problems. A system wide reboot to fix the system resulted in the entire system being taken down temporarily."
This discussion has been archived. No new comments can be posted.

Software Upgrade Crashes UK Air Traffic Control System More

Software Upgrade Crashes UK Air Traffic Control System

Comments Filter:
  • Maybe they can just use Prince Charles' ears for Radar?

  • Damn... (Score:2, Funny)

    by Anonymous Coward
    And I was going to put the blame on M$, but if it's a 70's system we're talking about I'll just shut up.
    • Microsoft Air-traffic Control Software? Shudder...

      • Re:Damn... (Score:4, Funny)

        by InShadows ( 103008 ) on Thursday June 03, 2004 @10:24AM (#9325277)
        Microsoft has been testing around with flight software since XP called MSFlight.. here's the article [bbspot.com] to prove it..
      • So if excel had a flight simulator as an Easter egg, would that mean the ATC software would have excel as one?

        Run away ...

        =)
      • > Microsoft Air-traffic Control Software? Shudder...

        It looks like you're trying to direct a pilot to land a plane. Would you like to:

        - Have the pilot land at Cirque du Soleil and tell him it's Denver International Airport [ntu.edu.tw]?

        - Redirect the plane to Chicago, but send the luggage on to Orlando?

        - Adjust the ground level to send the plane and all aboard to fiery doom like that scene in Die Hard 2?

      • Re:Damn... (Score:2, Funny)

        by LabRat007 ( 765435 )

        "Microsoft Air-traffic Control Software?" Shudder...

        I can see it now...


        Pilot: Air Traffic control (ATC) we've lost our nav con please advise!

        ATC: Please remain calm and click on start then control panel. Once in the control panel double-click(TM) on game controler - we need to reconfigure your yolk before we enter admin mode on the nav con system.

        Pilot: I don't see a |"start" button.

        ATC: Ok sir, I need you to find the windows key on your systems keyboard.

        Pilot: Whats a windows key?
      • Microsoft Air-traffic Control Software? Shudder...

        Would that be Macs you're talking about then?

      • Re:Damn... (Score:4, Funny)

        by identity0 ( 77976 ) on Thursday June 03, 2004 @07:07PM (#9330608) Journal
        Right, everyone knows that if you've got mission-critical, life-or-death systems that need ten nine's reliability, you should run them on iMacs.

        "I was directing air traffic on my ATC computer, giving a flight path to an KLM flight into Heathrow, when suddenly it was like *beep beep beep beep* and half of my planes were gone.

        "I was like, 'Uunh?'

        "It DEVOURED my flight path.

        "It was a really good flight path.

        "And then I had to send it again, and it wasn't as good 'cuase I had to do it fast. And the KLM crashed short of Heathrow.

        "It was... a bummer.

        "My name is Ellen Feiss, and I'm an air-traffic controller."

        Ph33r.

    • Actually, I believe they were using Microsoft Air, aka Longhorn.

    • Well, Bill Gates did develop a system to manage traffic flow [vt.edu]...
    • Windows may have been involved but I'll bet a days wages that outsourcing has something to do with this.

  • Three fingers (Score:3, Funny)

    by panda attack ( 785239 ) on Thursday June 03, 2004 @09:58AM (#9324985)
    Ah the good old reboot and hope for the best method :D

  • Software doesn't rust... (Score:5, Insightful)

    by Anonymous Coward on Thursday June 03, 2004 @09:59AM (#9325007)
    "which is an old (70's) system". As long as it's not 30-year-old hardware then the software should still be fine. Why does everyone think that simply because software was written in the past it is bad?
    • Because it IS old hardware.

    • Re:Software doesn't rust... (Score:5, Insightful)

      by GoofyBoy ( 44399 ) on Thursday June 03, 2004 @10:06AM (#9325087) Journal
      >Why does everyone think that simply because software was written in the past it is bad?

      What is implied is that its being pushed to its limits. e.g. it was designed for 100 flights a day, when today there are 1200 flights a day.

      Those small things which you could get away with before start to become factors in usability and stability.

    • Re:Software doesn't rust... (Score:4, Insightful)

      by lennart78 ( 515598 ) on Thursday June 03, 2004 @10:16AM (#9325186)
      Software written in the past also needs to be maintained. TFA states that testing for an upgrade and improvement to the West Drayton system was the cause of the problems.

      (Which I find strange, cause testing in a system as critical as this should be done in a separate environmnent.)

      I assume you've had no previous experience in maintaining a 'vintage' system like that? The code is probably written by a lot of different programmers, each with his own style, poorly documented, and thus very hard to read and understand.
      Software doesn't rust, but it clutters up and gets dirty over the years. It won't come apart by itself, but by the hands of a developer writing a necessary upgrade.
      • airport traffic control system laying around to test it on?

        • An airport traffic control system is just a computer with strange dials after all. What they tested on was likely a duplicate of the hardware, but obviously, other factors in play here threw the test off.

          If nothing else, they just proved Finagle's Law... If something can go wrong, it will...
        • Actually, as I understand it, there is a "test" system for the US FAA in NC. All changes are rolled out there first (ALL changes... down to replacing a UPS system!). The system can monitor traffic, but it is primarily a redundant system.
      • We had our boyos try out a piece of kit for Irish ATC, costing over €100 million. It didn't work. I believe they didn't test it in an operational environment though. It still made news, people are a bit suspicious of government expenditure on technology after the electronic voting débacle (Lots of equipment bought - supposed to be used June 11th. A committee pointed out they couldn't even test it to approve it as the company were still bringing out software updates! It's now sitting in a warehouse

    • Re:Software doesn't rust... (Score:4, Informative)

      by Shimbo ( 100005 ) on Thursday June 03, 2004 @10:54AM (#9325626)
      "which is an old (70's) system". As long as it's not 30-year-old hardware then the software should still be fine. Why does everyone think that simply because software was written in the past it is bad?

      Sadly, it really is running on ~30 year old hardware, at least in part. I've spoken to some of the service engineers.

    • Why does everyone think that simply because software was written in the past it is bad?

      Well, let's see. Would you like to use word processor from the 70's, "ed" perhaps? How about a nice video game, let's see we have Pong and Asteroids. Or you could go out on the Internet, I hear there are almost 100 sites hooked up now.

      Software *has* improved a lot since the 70s. Yes, I'm aware of the so-called "software crisis." My only question is where do people get the unrealistic expectations that make realit

    • Re:Software doesn't rust... (Score:4, Informative)

      by Prendeghast ( 658024 ) on Thursday June 03, 2004 @11:32AM (#9326121) Homepage

      The 'fridge size boxes are 70's vintage (I suspect bits have been replaced over the years). The CPUs are only about five years old. The system consists of two identical computers for hot failover and they they had to get two custom CPUs made by the original manufacturer (IBM, I think) to deal with Y2K.

      As for the software? Written in some weird language called Jovial, and continually repatched - never rewritten.

      BTW, where the heck is Swanage? The new NATS center is in Swanwick!

      • Re:Software doesn't rust... (Score:4, Informative)

        by general_re ( 8883 ) on Thursday June 03, 2004 @01:19PM (#9327387) Homepage
        ...Written in some weird language called Jovial...

        Muahaha. Languages from the stone-age. Jovial is an ancient semi-descendant of Algol, originally written especially for avionics systems. I'm not nearly old enough to have worked with it myself - Jovial's heyday was the mid-'70's or so - but I used to work with a couple of DoD greybeards who had done so, although even they hadn't touched the thing in years, as it's mostly been supplanted by Ada these days. The USAF can tell you a bit more about Jovial [af.mil] if you're having a slow day today ;)

  • Lucky in the US... (Score:5, Informative)

    by Kredal ( 566494 ) on Thursday June 03, 2004 @09:59AM (#9325014) Homepage Journal
    Considering that up until about 2000, all of the major Air Traffic Control centers in the US were running on vacuum tubes, we were lucky nothing like this ever happened here. Sure, there were glitches at regional centers, that required controllers to do everything by hand, but nothing that required a full reboot of the entire country's ATC system.

    Hopefully the UK will get the new system tested and online before it causes more problems!

    • Re:Lucky in the US... (Score:2, Informative)

      by gowen ( 141411 )
      Its already back running (and has been since this morning, BST) Now the only delays are caused by clearing the backlog of grounded flights.
    • Hopefully the UK will get the new system tested and online before it causes more problems!

      The only minor problem is that the new system is, if anything, more likely to cause problems than the old one. Especially if it follows the same pattern as the majority of the other big systems our useless government has thrown money at.

    • Re:Lucky in the US... (Score:5, Informative)

      by aldoman ( 670791 ) on Thursday June 03, 2004 @10:33AM (#9325373) Homepage
      There is 2 ATC centers in the UK - West Drayton which is for the 4 major London airports only (Heathrow, Standstead, Gatwick and London City). This is a 70s system and is due to be replaced by 2006. This is the one that crashed, but because a large percentage of UK air traffic is destined for London, then it caused the other one to go to a standstill.

      The other one at Swanage handles the ATC for everywhere else. This was replaced with a new system in 2002.

      But, by 2006 hopefully all ATC in the UK will be running on new systems.

      • Re:Lucky in the US... (Score:3, Informative)

        by Anonymous Coward
        This, like so many of today's reports (including the ones from the BBC), is inaccurate. England and Wales has 2 en-route ATC centres - that is centres that handle high-level traffic, outside the control of the terminal control areas round the airports. These are West Drayton and Swanick (not Swanage). The plan is that Swanick will handle all en-route traffic once flight data processing has moved from West Drayton. Swanick currently handles all en-route traffic control and routing, but if there is no flight
      • The other ATC is at Swanwick, actually not more than seventy miles from West Drayton and located in a very pleasant small town on the river hamble between Southampton and Fareham. This will replace West Drayton completely in 2006 and is already controlling a lot of the airspace. However there were some major hiccups at Swannick when the system was first introduced in 2002.

        Swanage is in Wales.

    • "Considering that up until about 2000, all of the major Air Traffic Control centers in the US were running on vacuum tubes, we were lucky nothing like this ever happened here"

      That's a statement based on a totaly false premise. Simply because it was old or used tubes did not mean it was a bad design, in fact, the simplicity of the design and the shortness of the source code made it very a very easy system to debug and program. It wasn't as pretty, but it worked fine.

      The simple fact is that there's more b

    • Re:Lucky in the US... (Score:3, Informative)

      by Anonymous Coward
      Your regional centers cover about the same airspace as the West Drayton centre, so it is hardly fair to compare a US regional TRACON to the London FIR (comparing big regions to small countries....). Also, it wasn't a reboot of the entire countrys ATC system, it was a reboot of the host computer system at West Drayton, responsible for the printing of flight strips and processing of flight information at the Drayton centre - The radar at West Drayton was still functional and airports still have their own indi

  • More problems... (Score:5, Informative)

    by Mz6 ( 741941 ) * on Thursday June 03, 2004 @10:01AM (#9325035) Journal
    I found a similar article on MSNBC [msn.com].

    It seems they have been having problems with their computer systems since 2001 when it was "privatized".

    "The air traffic service has been beset by problems since it was partially privatized in 2001. A $484 million center at Swanwick in southern England opened five years late in 2002.

    The opening was delayed by problems with computer software, and the glitches continued for months afterward, as controllers misread aircraft altitudes and destinations because of hard-to-decipher computer screens. In at least one case, controllers mistook the Scottish city of Glasgow for Cardiff in Wales.

    Now.. that seems like a pretty big mistake for me.. especially for an air traffic controller to do. However, the article later states that:

    "Transport Secretary Alistair Darling said Thursday's problem did not lie at Swanwick but at the older West Drayton center, which is due to be closed by 2007."

    Thank goodness that old one is closing, however it doesn't sound like its replacement is doing any better!

    "If you want to know what is wrong with transport in this country it is that over decades successive governments did not spend enough on the infrastructure and air traffic control is no different," Darling told BBC radio."

    Excellent quote! While terrorism is on everyone's mind, we sometimes forget that safety of transportation should also be just as high. I couldn't imagine pilots relying on themselves to fly airplanes amid the thousands of others without the aid of traffic controllers and their computers.

    • Re:More problems... (Score:5, Interesting)

      by Mr_Silver ( 213637 ) on Thursday June 03, 2004 @10:14AM (#9325166)
      "If you want to know what is wrong with transport in this country it is that over decades successive governments did not spend enough on the infrastructure and air traffic control is no different," Darling told BBC radio."

      A dutch friend of mine once remarked that she didn't understand the mentality of the British. "You" she said, "have an amazing tendency to run things into the ground and then get around to fixing them rather than spending money on continually maintaining them so they never fall apart."

      It's a very good point.

      • Re:More problems... (Score:5, Insightful)

        by Anonymous Coward on Thursday June 03, 2004 @10:23AM (#9325266)
        A dutch friend of mine once remarked that she didn't understand the mentality of the British. "You" she said, "have an amazing tendency to run things into the ground and then get around to fixing them rather than spending money on continually maintaining them so they never fall apart."
        That's a pretty fair point. The trouble is, since 1977, politics has been obsessed with who can provide the biggest tax cuts, which has meant all the state funded / supported industries have gone to hell in a handcart.
        <rant>
        I blame Margaret fucking Thatcher, who let the hospitals fall apart and flogged off the viable bits of the infrastructure to her friends (at well below market value). [We're still feeling the effects of this on the railways, which the private sector has run into the ground] Corrupt old bitch.
        • Yeah, because managerial socialism was so much better.

          Corrupt old bitch.

          Gosh, how enlightened an woman-respecting you leftists are!

        • To be fair (not that I hold any affection for Mrs T. in my heart) the rot stretches back a lot further than the 70s.

          I'd say the UK has been letting the infrastructure maintenance slide since at least WW2, maybe earlier. We inherited a fantastic installed base from the Victorians - the fact that it took 50 years of neglect to rot away is a tribute to how well they built - but the sad fact is this stuff was put together by a world-spanning Empire at the top of its game. What with paying for a couple of world

      • Re:More problems... (Score:3, Insightful)

        by drwho ( 4190 )
        A dutch friend of mine once remarked that she didn't understand the mentality of the British. "You" she said, "have an amazing tendency to run things into the ground and then get around to fixing them rather than spending money on continually maintaining them so they never fall apart."


        Funny, I noticed this about the U.S. system. But I figured it out. It has to do with the fact that civil maintenance is done by civil-service people with a union and a contract, while new equipment and construction contracts

      • Re:More problems... (Score:3, Interesting)

        by plopez ( 54068 )
        The same is true in the private sector. No money for plant maintenence until something breaks and threatens a lucrative contract.

        Or the management mentality of 'Oh, security is too expensive right now we'll ship it and fix it later'.

        Politicians only look to the next election and managers only look to the next quarter. It is a typical attempt by non technical types to ignore entropy, expressed quite nicely in the old saying 'rust never sleeps.' If you want a bridge to last, paint it today, not after it has
        • Or the management mentality of 'Oh, security is too expensive right now we'll ship it and fix it later'.

          To be fair, it's really hard to estimate the economic costs and probability of a security breech.

          The insurance industry is in this business, and they have to maintain and analyze *masses* of data to pull this off.
    • "I couldn't imagine pilots relying on themselves to fly airplanes amid the thousands of others without the aid of traffic controllers and their computers."

      Then I'm afraid you lack imagination: the idea that aircraft need to be controlled by people on the ground is a large part of the problem... not only is there no real need for such a system now that technologies like GPS can allow aircraft to communicate and ensure they're not going to collide with any other aircraft nearby, but 'air traffic control' inc
    • "The air traffic service has been beset by problems since it was partially privatized in 2001. A $484 million center at Swanwick in southern England opened five years late in 2002.

      What a nice little non sequiter. The system is having problems because it was privatised, but the Swanwick Center has nothing to do with that, since it was five years late one year after the privatization.

      Sounds like the non-privatised system was having problems all on its own....

  • What WAS the System that crashed? (Score:5, Insightful)

    by Spencerian ( 465343 ) on Thursday June 03, 2004 @10:02AM (#9325046) Homepage Journal
    Perhaps a person experienced in ATC software or hardware could enlighten us on the specific system in use, its OS and other trivial bits.

    It would help to reduce the coming surge of Microsoft jokes, which is very likely not relevant here.

    • Re:What WAS the System that crashed? (Score:5, Informative)

      by orbitalia ( 470425 ) on Thursday June 03, 2004 @12:04PM (#9326569) Homepage
      Hi, I worked on exactly this system for 4 years.

      The hardware is an IBM 9020 family mainframe, the application is written in Jovial (one of , if not THE first algebraic language), and BAL assembler (for the monitor mostly). The monitor is the operating system so it effectively is a custom written operating system for this application.

      Although MVS is also used for testing. The I/O capabilities of the mainframe are superb which means it can handle 2000+ flights with only 14 Megs of RAM (if I remember rightly).

      I believe the NAS application came as a freebee from IBM when the UK purchased the hardware and was the same NAS (national airspace system) application used all over the US. It has been continously developed since then (no mean feat when you consider that all variables are global in Jovial, It uses holleriths instead of ascii, and you are limited to 5 or 6 characters per variable name). The hardware has also been upgraded several times over its lifetime.

      It doesn't often go down, last time was 2002 sometime, and you can tell how important it is because everyone screams when it does go down. The people I worked with are extremely dedicated to their job, but one cannot test a system like this for absolutely every eventuality. No doubt some patch was applied and some special case came up that caused a FLOP (functional loss of operation). It happens, Radar is usually unaffected, so the safety implications are not large, but flow is affected.

      The UK approach to handling NAS is much different to the US, the US tends to not touch the NAS software and develop external systems that enhance the usage of airspace, where as the UK tends to delve into NAS and improve things directly in NAS. Jovial is a very interesting language it has been used heavily by the US military and exists in such applications as Cruise missiles and many other aircraft and missile systems. Read about Jovial here if you are interested.

      I can't say too much about it for various NDA reasons (OSA) I think most of the above is in the public domain.

      HTH.

      • Oops I lost my link there - Jovial Lives! [af.mil]
      • I'd worked with various Host upgrades during my years of supporting the FAA. The software is hideously old, so we avoided stirring up trouble. We were coming up with a replacement system when I left and took a position as a senior lecturer of computing in the UK. The thing about the UK approach to transportation systems engineering that has struck me is a general unwillingness to look outside the box. I can't get research funding for anything modern.
    • It's an IBM 360/370 class mainframe: not sure what model. Somebody up the line said that the software was written in Jovial, which strikes me as very likely. Jovial was an Algol variant popular in defence/high reliability circles at about the time this lot was written.

      I think the system which crashed was only responsible for admitting new flight plans to the whole complex. Any flightplan already filed could carry on; it is just that no-one could file a new plan for the next flight.

  • Links for reference (Score:4, Informative)

    by matthew.thompson ( 44814 ) <matt&actuality,co,uk> on Thursday June 03, 2004 @10:04AM (#9325064) Journal
    National Air Traffic Services http://www.nats.co.uk/services/index.html are the outfit responsible for this.

    They have a press release http://www.nats.co.uk/news/news_stories/2004_06_03 .html which explains quite nicely what they did and why.

  • So what? (Score:5, Insightful)

    by stratjakt ( 596332 ) on Thursday June 03, 2004 @10:07AM (#9325097) Journal
    There are redundant systems in place. Analog radar, humans with brains.

    At least there should be. Computers crash, break, have bugs, etc. They're a tool - a more efficient and convenient tool to be sure.

    But when they break, there are contingencies so that planes can still take off and land, and wont just fall out of the sky.

    This is also why Y2K was such a bunch of stupidity. We really aren't as reliant on computers as people think. We know they crash and are prepared to handle it when they do.

    • >There are redundant systems in place.

      The redundant systems can't replace the speed and accuracy of a computer.

      Computers are a tool. But how do you access the radar system and translate its information with out a computer?

    • Re:So what? (Score:4, Informative)

      by Scooter ( 8281 ) <owen@annicnova.f ... t minus language> on Thursday June 03, 2004 @11:28AM (#9326069)
      Hmm I don;t think there are those humans around. Cartinaly not in the quantities that would be required to manually guide the 1200 flights a day. We get dependent on the systems. We put the systems in because the load increases beyond the economic viability of an army of ATC guys, not to mention the communication overhead and possibility of error or mis communication. So we build a computer system to deal with it instead. That in turn allows us to up the load by an order of magnitude again. 30 years later, take the system away, and there's nothing.

      In scenarios like this, where load has increased whilst the computers systems were in place, we *are* reliant on them.

      Think of banks - time was when you had to almost plead on your knees to get a banck account, and they charged you for running it. This was becasue every account was written down manually in a book, and any calculations were performed by hoards of clerks. Then - computers. Now your new account is just one more record in a table somewhere, so the banks give out accounts to anyone who wants one, and do it for free. If for some reason your bank's computer system goes AWOL, there is no way they can process a month's interest calculations on the millions of balances and transactions - not to mention actually applying the transations that would now come in on bits of paper.

      I do agree that in a lot of cases, there remains a perfectly useable manual method, but where the computer system has enabled geometric increases in capacity over the manual system (which has been taken up) then, if you'll excuse the pun, it won't fly.

      You're right about the Y2k thing - I worked on a contract for a railway maintenance company in 1999 and the Y2K cordinator guy was tearing his hair out at the thousands of questions he got monthly such as "so, these nails, are they Y2K compliant?" He actually had solid steel track components called "chairs" that the rails sit on that had Y2K compliance stickers on them from the manufacturer. Presumably, they got fed up explaining it too, and decided it was easier to just stick the stickers on everything they made...

  • Same in Ireland! (Score:5, Informative)

    by pixelbeat ( 31557 ) <P@draigBrady.com> on Thursday June 03, 2004 @10:09AM (#9325120) Homepage
    Much the same thing happened last
    week in Dublin [ireland.com]

  • And the Wizard said: (Score:5, Funny)

    by the_twisted_pair ( 741815 ) on Thursday June 03, 2004 @10:09AM (#9325121)
    It appears you are trying to land a plane. Would you like to:

    [x] Allow Windows to detect new hardware ?
    [ ] Allow planes to circle in uncertainty ?

    [x] Show this window at all airports

  • http://abclocal.go.com/ktrk/news/050404_local_airp ort.html

  • Bug (Score:2)

    by l0wland ( 463243 )
    Told you so! At the year 19804 it just HAD to go wrong....
  • at least only the computers crashed

    as for the system crashing in the first place, it's unfortunate, but a good thing that they were able to cope and keep everyone safe - that's the main thing, right? (it's certainly my main concern)

    and as for the software not being up to the job, it may well not be. after all, air traffic has increased ever so slightly since the 1970's - is it reasonable to expect a program presumably designed for 70's hardware, and 70's air traffic loads to cope with heathrow in 2004?

  • Swanwick not Swanage! (Score:5, Informative)

    by perly-king-69 ( 580000 ) on Thursday June 03, 2004 @10:17AM (#9325205)
    The new centre is at Swanwick in Hampshire, not Swanage in Dorset!!

  • Swanwick, not Swanage! (Score:4, Informative)

    by Xilman ( 191715 ) on Thursday June 03, 2004 @10:22AM (#9325259) Homepage Journal
    The new system is at Swanick near Southampton, not Swanage as posted here.

    Swanage is a pleasant little seaside resort. I know it well and stayed there a few nights when on my honeymoon.

    Finding Swanwick and Swanage on a map of southern England is left as a exercise. Hint: Mapquest [mapquest.co.uk] may be a good place to start.

    Paul

  • And here we have brought up on liberal dose of reboots , preferably once every few minutes, simply to ensure everything works

    err, if you keep your fingers crossed, that is!
  • Comment removed based on user account deletion
    • It's just mobile phones that need to be off. On any plane I've been on, laptops and other portable computer equipment should be turned off during takeoff and landing, but can be used during the flight.

  • Hang on a second... (Score:3, Interesting)

    by Gordon Bennett ( 752106 ) on Thursday June 03, 2004 @10:41AM (#9325460)
    To quote from the NATS (National Air Traffic Services) press release:

    "The FDP was being tested overnight for a future upgrade. The system was successfully returned to service but at 06.03 errors were detected in the distribution of flight data between Centres. As a precaution, we decided to restart the FDP (known as a cold restart) causing an interruption to full service. The data processing system was restored at 06.42 and declared fully operational at 07.03. Flight capacity restrictions were lifted at 08.05. The system is now fully operational and we are confident that it is stable.

    Through the response team at West Drayton, we have been working with airports and airlines to clear the delayed departures, and expect the backlog to be cleared quickly.

    Our investigation into the cause of the problem is continuing."

    Let me get this straight: they ran a test on the FDP. The FDP glitched. They rebooted the FDP. They are still investigating the problem.
    Now, unless I am mistaken, I can only infer from their statement above that they are now running the FDP which is still susceptible to the problems highlighted by the test.

    • Re:Hang on a second... (Score:4, Insightful)

      by jrumney ( 197329 ) on Thursday June 03, 2004 @11:04AM (#9325738)
      Now, unless I am mistaken, I can only infer from their statement above that they are now running the FDP which is still susceptible to the problems highlighted by the test.

      That's not the way I understand it. From their report, I understand the events went something like this:

      1. Overnight while it is quiet, new FDP software is brought online for testing.
      2. Testing was successful, and they brought the old FDP system back online (probably before 5:30 in time for the first arrivals at Heathrow).
      3. At 6:03 they noticed "errors in the distribution of flight data between Centres." I don't know what exactly this means, but if I had to guess, I'd say that the other Centre was not taken offline during the tests, and some test data leaked onto their live system as a result.
      So the problem is most likely not due to a bug in either the new or current software, more likely a bug in the testing procedure they were using.
  • .. was actualy upgraded recently but they are still using the same old software.
    article [computerweekly.com]

  • Golden rules.. (Score:5, Insightful)

    by rf0 ( 159958 ) <rghf@fsck.me.uk> on Thursday June 03, 2004 @10:53AM (#9325606) Homepage
    Never just test software upgrades on Live systems

    Rus

    • Re:Golden rules.. (Score:3, Interesting)

      by hughk ( 248126 )
      A big bank did this, only they thought it was in UAT, however it registered itself with production as wanting to collect equity trades. It did, and very well too. They realised by the end of the day that the production backoffice was only seeing a fraction of the number of trades expected. Some poor bastard then had to trawl through the UAT database pulling out trades that were really intended to go to production and put them in the right place. I heard it took a couple of weeks. This is a shame because tra
  • What Operating System does it primarily use?

  • Windows Update (Score:3, Funny)

    by Anonymous Coward on Thursday June 03, 2004 @11:15AM (#9325867)
    Unconfirmed reports are stating that aparently one of the air traffic controllers accidently clicked on the "Windows update" icon. :P

  • ATC software is scary (aka, Know Your Userbase) (Score:4, Insightful)

    by Knowbuddy ( 21314 ) on Thursday June 03, 2004 @12:37PM (#9326957) Homepage Journal
    My dad is helping the FAA and the US military design and roll out the next gen ATC software here in the US. He comes home and tells stories that make my skin crawl.

    The first version of the software was built using standard current interface guidelines and widgets and the testing group that had no experience with older ATC systems were wowed at how simple and yet powerful it was. Pretty much any random person off the street could look at the screen and easily figure out what was going on and how to do various basic tasks. When that version was demoed to the ATC union the union freaked out at how different it was and thus began a cycle of making it more and more backwards.

    So, nowadays the next gen ATC software almost exactly replicates the UI of the old non-computerized and semi-computerized systems. On-screen toggle switches and dials, that sort of thing. The FAA and the ATC union have decided that retraining all of their ATCs to use modern computer interfaces would be a Bad Thing. When the computer screen doesn't exactly replicate the interface of the 50+-year-old systems, they freak out and scream bloody murder. On the flip side, kids coming into the field today that have been using computers most of their lives are finding the interface to be counterintuitive to the point of being almost unusable. Middle-aged workers who are both highly proficient ATCs and home computer users report that switching between the two types of interfaces each night when they go home requires conscious effort on their part, since they are so orthogonal.

    So who wins? Historical inertia, of course. Why fix the problem today when you can wait for your successors to fix it in 25 years?
    • Oh, the Union...

      The average ATC is a retired enlisted man or woman. They're trained to react, not to think, and any changes to the user interface make them very nervous. On the other hand, they're very good at what they do.
    • Nothing personal, and this is not intended to be an attack on your dad -- but in terms of system reliability (and who I'd want running things if I were in an airplane), those crochety old ATCers and their ad-hoc systems had an incredibly, almost impossibly good safety record. I had some free time and, for fun, ran out to start reading a history of world air disasters. ATC error is almost *never* a factor in air disasters in the United States (things change wildly when you go to other countries). (Actuall
  • The NATS London Area Control Center is of course, at Swanwick - not "Swanage" as stated in the original post.

Slashdot Top Deals

For God's sake, stop researching for a while and begin to think!

Close