Google’s foray into Facebook’s space hit an unfortunate glitch during its “field trial” … the system ran out of disk space and was down for more than an hour and those users, who were affected, received a deluge of emails … see Graham Cluley’s post “Google+ runs out of disk space, spams users with notifications” at http://nakedsecurity.sophos.com/2011/07/10/google-runs-out-of-disk-space-spams-users-with-notifications

This event was particularly troublesome. First, it doesn’t help user acceptance to have a new service fail during its initial tryout period, even though everyone is given to understand that the purpose of a trial is to flush out such problems. Secondly, for the many who see cloud computing, with its “infinite” processing power and storage, as the model of future systems, it was particularly disturbing to see that the Google+ notification system failed because it ran out of storage. Add to that the spamming of users and you have a double whammy against user confidence in the abilities of a premier company, such as Google, to manage its systems resources and the functioning of its applications under error conditions.

While it is true that the Web has substantial resiliency and that most issues, though not all, affect only a minority of customers and have relatively short durations, it is more a matter of recognizing how easily the integrity of systems can waiver and fail. And that is just the human-error component. You can be sure that potential attackers are taking notes. If major systems can fail badly without outside interference, just imagine what a concerted effort might do. Google may have more or less dodged the bullet this time, but systems are only getting more complicated and interrelated, raising concern that future glitches might have much greater consequences.

© BlogInfoSec.com, 2011. | Permalink | No comment | Add to del.icio.us
Post tags: Ben Edelman, cloud computing, FST, functional security testing, Google, Google Plus, software engineering, spotlight

There is a retrospective report on May 18, 2011 in The Wall Street Journal by Yuka Hayashi and Phred Dvorak, with the title “Fresh Tales of Chaos Emerge From Early in Nuclear Crisis,” which describes the first few minutes following the earthquake that hit Japan on 3-11-11 and how workers in error shut down a backup system, leading to the meltdown of one of the nuclear reactor cores. Here is a quote from the article:

“Soon after the quake, but before the tsunami struck, workers at one reactor actually shut down valves in a backup cooling system—one that, critically, didn’t rely on electrical power to keep functioning—thinking it wasn’t essential. That decision likely contributed to the rapid meltdown of nuclear fuel, experts say.”

Another report in InfoWorld magazine describes in detail the human error that caused a recent major outage of Amazon’s cloud computing services.

To paraphrase the cartoon character Pogo … we have seen the error and it’s human. [For those for whom Pogo was “before their time,” the actual quote is “We have met the enemy... and he is us.”]

So the solution seems simple. Take humans out of the decision process. That’s OK except for one thing … humans are the ones who design and build the automated systems that are supposed to replace the human “actors.”

Granted, it is far better to have a group of engineers consider carefully the design and functioning of the automated systems than to have operators decide on the spur of the moment what to do under fire and in the midst of a crisis. But such human-engineered systems are never perfect, so they almost always have an option for human intervention. And so, we wind up with the potential for human error under the worst of circumstances.

© BlogInfoSec.com, 2011. | Permalink | No comment | Add to del.icio.us
Post tags: Human Elements, Japan, nuclear power plant meltdown, software engineering, spotlight

As the recent reporting demonstrates, an outage in cloud computing does in fact let the sun shine through … through to the realization that the cloud is not totally reliable and resilient. The recent experiences with Amazon and Google serve to illustrate this, although there have been quite a number of other significant prior incidents.

A good overview of the cause and impact of the Amazon outage is given on page 31 of the May 16, 2011 issue of InformationWeek in the form of a sidebar by Charles Babcock with the title “When Amazon’s Cloud Turned On Itself.” The writer was particularly surprised that Amazon’s cloud services were in fact susceptible to human error, since he presumed that “clearly obvious errors has been anticipated, with defenses in place, automated checks….” Babcock suggests what should be done to avoid this particular problem in the future and so maintain faith in cloud computing. But, even if the changes were implemented, would such faith be justified?

When it comes to complex systems and networks, it is apparent that not only are some errors and bugs not readily anticipated, but their resolution will not have been fully incorporated into automated systems. This is because human beings, who design, implement and operate those automated controls, will always be fallible to the extent that they cannot predict every possible negative outcome. The question at hand is whether they should have expanded the testing of the controls to account for additional scenarios. This is the usual trade-off of the costs and time delays of such testing versus the reduction in the risk of failure, which I have discussed many times previously.

© BlogInfoSec.com, 2011. | Permalink | No comment | Add to del.icio.us
Post tags: Amazon, Availability, Charles Babcock, cloud computing, Cloud Security Forum, Google, resiliency, spotlight, survivability

Many were surprised by the Stuxnet worm that infiltrated into Iranian nuclear materials processing plants and reportedly caused the destruction of centrifuges. But they shouldn’t have been surprised, especially if they had read NIST SP 800-82 “Guide to Industrial Control Systems (ICS) Security,” written by Keith Stouffer, Joe Falco and Karen Scarfone and published in September 2008. The report is available online at the NIST website.

I happened to be looking over this particular NIST report recently and, in the Executive Summary found the following, which I have abbreviated slightly to emphasize the relevant items:

“Possible incidents an ICS may face include the following:

• Unauthorized changes to instructions, commands, or alarm thresholds, which could damage, disable, or shut down equipment …
• Inaccurate information sent to system operators, either to disguise unauthorized changes, or to cause the operators to initiate inappropriate actions, which could have various negative effects
• ICS software or configuration settings modified, or ICS software infected with malware, which could have various negative effects”

Does this sound familiar? Is this not the recipe for Stuxnet? So what was all the commotion about?

The issue here is that, just because organizations, such as NIST, point out how attacks might be structured, it doesn’t mean that the exploit will be developed. There again, it might be created, as it was with Stuxnet. Such anticipatory writings move certain risks from “unknown unknowns” to “knowns” or, at the very least to “known unknowns.” We certainly aren’t able to account for every prediction and anticipatory warning, but when the source is as authoritative as NIST, it is at least worthwhile to consider the implications and respond accordingly

© BlogInfoSec.com, 2011. | Permalink | No comment | Add to del.icio.us
Post tags: centrifuges, ICS, Industrial Control Systems, Iran, NIST, spotlight, Stuxnet

One of the most horrifying comments through the entire Japanese mega-catastrophe was that by CNBC anchor Larry Kudlow, as reported in a March 20, 2011 New York Times article by Jeff Sommer with the title: “A Crisis That Markets Can’t Grasp – As Japan’s Disaster Evolves, Wall Street Keeps Recalculating.” Kudlow said, and then apologized for the following statement: “The human toll here looks to be much worse than the economic toll, and we can be grateful for that.”

While unfeeling and unforgiveable, Kudlow’s comment reflects an issue that has generally plagued safety and security economics, namely, attaching an economic value to human injury, suffering and loss of life. One of the main reasons that it seemed so much simpler to justify expenditures on safety systems (though the Fukushima Dai-ichi nuclear power plant may be the exception that proves the rule) is that the value of a human life is usually considered to be so much greater than mere economic loss, despite the views of the Larry Kudlows of this world.

The fiasco in response to the Fukushima “accident” is perhaps best illustrated by an article on the front page of the March 19-20, 2011 WSJ Weekend section with the title “Bid to ‘Protect Assets’ Slowed Reactor Fight.” The article, by Northiko Shirouzu, Phred Dvorak, Yuka Hayashi and Andrew Morse, begins with the following:

“Crucial efforts to tame Japan’s crippled nuclear plant were delayed by concerns over damaging valuable power assets and by initial passivity on the part of the government …”

© BlogInfoSec.com, 2011. | Permalink | No comment | Add to del.icio.us
Post tags: "tragedy of the commons", BP, economics of security, Fukushima, RSA, spotlight, TEPCO

When diverse systems are integrated, systems engineers must often trade off security against safety, and vice versa. As an illustration, consider automated building-access systems. Safety engineers generally prefer building-access systems to fail open so that anyone trapped in a burning building, for example, can escape and/or emergency workers (such as firemen and EMS workers) can easily enter the building. On the other hand, security professionals would likely suggest that building-access systems should fail closed so that bad guys, such as looters, cannot enter the building.

Safety requirements should prevail. I say this because of direct personal experience and also a story that I was told a dozen years ago, neither of which I can support through published references. While there have been a number of well-documented cases where business owners locked emergency exits of factories, night clubs, etc. with resulting high numbers of deaths when fire broke out, they are not based on automated systems but on human action, such as the locking of emergency exit doors.

Sadly, I now have a documented case of an automated system locking personnel into a dangerous environment. It relates to the catastrophe at the Japanese Fukushima Dai-ichi nuclear power plant both during and immediately after the 3-11 earthquake. In a news article by Terril Jones dated March 26, 2011 and with the title “Japanese worker inside stricken reactor recalls quake,” available at http://in.reuters.com/article/2011/03/25/idINIndia-55893620110325 , you can read the following harrowing vignette:

“… the lights went out, leaving about 200 workers inside the reactor in near-darkness since the structure has no windows.

© BlogInfoSec.com, 2011. | Permalink | No comment | Add to del.icio.us
Post tags: automated building-access systems, Fukushima, safety, safety v security, spotlight

As I am writing this, a devastated Japanese nation is still struggling to recoup from the triple-whammy of earthquake, tsunami, and potential nuclear power plant meltdown. We need to wait until we know more of the facts before examining whether contingency planning was as good as it might have been in the face of catastrophes of such magnitudes. I do plan to return to the topic of catastrophe contingency planning at a later time but, in the interim, I point you to my chapter on “Responsibilities and Liabilities with Respect to Catastrophes” in the book Social and Organizational Liabilities in Information Security edited by Manish Gupta and Raj Sharman (IGI Global, 2009), which, due to it being the “Free Sample Chapter,” you can read at … http://www.igi-global.com/viewtitle.aspx?titleid=21331&sender=462d264d-7629-4504-a8c7-ef684703212c  As in the case of the BP Gulf oil spill, the Japanese government abdicated responsibility to the corporation owning the destroyed and destructive facilities, bringing to the fore the matter of what roles public and private entities should play in responding to catastrophes resulting from man-made structures.

Meanwhile, I would like to revisit supply chain integrity risks. To some degree, the topic is a reissue of outsourcing risk, which I examined in detail in my book Outsourcing Information Security (Artech House, 2004). However, focus on supply chains has increased as production is dispersed more broadly across the globe and as the complexity of software, products and services has greatly increased. The issue has again raised its ugly head as a result of the Japanese mega-catastrophe. In a mere week, we see the impact in the U.S. (General Motors), Thailand (Honda), Taiwan (Advanced Semiconductor Engineering), and Sweden (Volvo) as described in the March 18, 2011 Wall Street Journal article “Crisis Tests Supply Chain’s Weak Links” by James Hookway and Aries Poon. More details on the intentions to close GM’s Shreveport, Louisiana plant for at least a week are given in Nick Bunkley’s March 18, 2011 article “Lacking Parts, G.M. Will Close Plant” in The New York Times. The full impact of the supply chain woes are becoming increasingly apparent as each day goes by.

© BlogInfoSec.com, 2011. | Permalink | No comment | Add to del.icio.us
Post tags: catastrophe, Contingency Planning, earthquake, information security outsourcing, inventory systems, Japan, nuclear power plant meltdown, spotlight, supply chain, supply chain risk, tsunami

That might appear to be a strange question. After all, when it comes to automobiles, the focus is mostly on safety, although articles have appeared describing how the computer networks and systems built into modern automobiles can readily be hacked and the hacker can take control of various operating functions.  And stealth electric cars are known to be a “silent menace” and so they are being equipped with noisemakers to warn pedestrians of their approach.

But, no, I really mean secure in the sense that electric cars are totally dependent on the electricity grid, which is increasingly being questioned in terms of its resiliency and vulnerability to cyber attacks on the SCADA (Supervisory Control and Data Acquisition) systems, which manage the electricity grids. Modern economic life is becoming increasingly dependent on supply chains of which too little is understood and which put us at increasing risk from disruptions. We have seen how events can easily interrupt the gasoline supply chain and we have some inkling of the current impact of electricity outages. Yet we are still willing to leverage the electricity supply chain so as to be even more dependent, without taking the time to analyze the impact.

I would argue that we are going in the wrong direction when it comes to electric cars. I’m somewhat supportive of hybrid vehicles, except that I would question the environmental impact of dumping all those batteries and the supply of materials from which the batteries are made, such as lithium, being in the hands of a few countries.

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: cyberattack, electric cars, grid dependency, network resiliency, SCADA systems, spotlight, Supervisory Control and Data Acquisition Systems

The largely successful remediation of the Y2K “bug” arguably led to the worst of outcomes for the credibility of cybersecurity risk. Many believe that Y2K was all a hoax perpetrated by software consultants and vendors in order to generate income, which it certainly did. Others, who had a better understanding of the technical issues, knew that we in fact dodged a very big bullet.

It was therefore very amusing to read these opposite views in a single magazine that appeared more than a decade after the fact. The publication is a special issue of Scientific American, with a September 2010 date and the brief, but disconcerting, title “The End.”

On page 40, in a piece “Eternal Fascinations with The End,” staff editor Michael Moyer writes:

“… captains of industry shelled out billions preparing for the appearance of two zeros in the date field of computer programs too numerous to count; left alone, this tick of the clock would surely have shaken modern civilization to its foundations.”

Yet, on page 93 of the same magazine, there is an excellent article on “The Age of Digital Entanglement” by Danny Hillis, who, it is noted, “…predicted widely that the Y2K ‘problem’ would be a nonevent.”

 I mention this Y2K. paradox in a several columns, mostly notably “Cybergeddon … Ho Hum” posted on March 29, 2010. From my position in the National Information Center in Washington, DC, I observed firsthand what actually took place on that unique weekend. The attempted denial-of-service attacks, the systems that did in fact fail but were not widely publicized, and the ones that would have crashed were it not for the remediation effort and the precautions that so many businesses, government agencies and individuals took.

© BlogInfoSec.com, 2010. | Permalink | One comment | Add to del.icio.us
Post tags: "chicken little", cyber meltdown, spotlight, Y2K

On May 19, 2010, Dr. Patricia Muoio of the ODNI (Office of the Director of National Intelligence) gave a thought-provoking presentation at a symposium hosted by NITRD (Networking and Information Research and Development), which is the name of a program that “… provides a framework in which many Federal agencies come together to coordinate their networking and information technology (IT) research and development (R&D) efforts.” … see www.nitrd.gov

The name of the symposium was “Toward a Federal Cybersecurity Research Agenda: The Game-Changing Themes,” and the particular topic covered by Dr. Muoio in the “Government Overview” session had to do with “moving target” approaches. The underlying proposition is that, in order to outwit the bad guys, one should be agile and stay one step ahead of the attackers by making security systems so complex that those with evil intentions will not be able to keep up. Currently, it would appear that the shoe is on the other foot, with the attackers keeping victims on the defensive by being state-of-the-art and staying ahead of the owners’ efforts to protect information assets.

More than a decade ago, Bruce Schneier wrote a piece in his Crypto-Gram Newsletter of March 15, 2000, with the title “Software Complexity and Security,” available at http://www.schneier.com/crypto-gram-0003.html. The opening sentence of the article is: “The future of digital systems is complexity, and complexity is the worst enemy of security.” He ends the article with the words: “Secure systems should be cut to the bone and made as simple as possible. There is no substitute for simplicity. Unfortunately, simplicity goes against everything our digital future stands for.”

While Bruce was referring to the underlying software that infosec professionals are trying to protect, I believe that the same goes for security systems. If we embark on a complexity race with the bad guys, we may end up the losers. The real, though likely impossible-to-do answer, is to simplify underlying software, platforms and infrastructure and have correspondingly simple security systems. However, even if we are not able to roll back complexity, we need to do something in terms of the designs and structures of systems. The subprime mortgage fiasco, the “flash crash” of May 6, 2010, and the Gulf oil gush catastrophe are all recent examples of complexity resulting in disaster and making for much more complicated recoveries.

I bemoaned the increasing complexity in a column “The Death of K.I.S.S.” in Securities Information Magazine, in 1994. I pointed out that systems had become so complex that it was no longer possible for an individual or a group of individuals to know and understand every aspect of the systems that they are responsible for supporting.

A real-life example of the detrimental impact of complexity is the April 27, 2010 New York Times front-page article by Elisabeth Bumilller with the title “We Have Met the Enemy and He Is PowerPoint,” with due deference to Pogo. The article is accompanied by a very complicated “… PowerPoint diagram meant to portray the complexity of American strategy in Afghanistan…” According to Gen. Stanley A. McChrystal, then leader of the American and NATO forces in Afghanistan, “When we understand that slide, we’ll have won the war.” As for the war in Afghanistan, so it is for computer systems and their protection. We are continuing to build ever more complex systems, so how can we hope to protect them?

I believe that hope lies in decoupling system components … that is, introducing the computer system equivalent of circuit breakers. If a module is overheating, there should be some way of disconnecting it from the rest of the system in order to avoid a ripple effect. This is similar to isolating sections of the electricity grid so that a failure of one segment does not bring down others.

In my opinion, the answer lies in simplifying systems and minimizing their attack surfaces. Perhaps we’ll have to give up some functionality, perhaps not. Perhaps it will cost more, perhaps not, especially when you include the cost of system compromises and failures due to attacks. In any event, it is worth a try since the greater-complexity approach does not appear to be working.

© BlogInfoSec.com, 2010. | Permalink | No comment | Add to del.icio.us
Post tags: Bruce Schneier, Dr. Patricia Muoio, NITRD, ODNI, software complexity, spotlight