Disastrous Practices Part 3

By Patrick McCulley |

In the previous column, we looked at general pointers for healthy integration of best practices into the philosophy of the modern business, especially the startup. This week’s piece explores the importance of service availability, and how the Chief Information Security Officer and Chief Information Officer can work together to ensure maximum data availability and integrity.

Uptime, Downtime

In the age of information, seconds count; high speed stock trading can be negatively affected by even microsecond delays. Retail outlets are also at great risk – for every hour that their services are offline they lose potential sales; the effects cumulate over time and can either make or break a business.

When approaching the subject of uptime, the availability of your services to both your organisation and to your customer base, it is helpful to think of not only things that will happen (due to thermodynamics, like hardware failure), but also things that have a significant probability of happening in the not-too-distant future. Attempting to predict when any of these potential events might happen is an exercise in futility. A more robust approach is to examine the statistical distribution of incidents, comparing them with your organisation’s capabilities and attempting to mitigate risk accordingly. The Chief Information Officer and Chief Information Security Officer should work very closely to ensure that a granular security policy properly addresses all aspects of your data services.

Frequently, security incidents are overlooked as potential causes of major periods of downtime; the net result is that when a serious security incident does happen, the unprepared organisation is taken entirely by surprise, with no plan in place with which to approach the situation.

Outside experts must be called in, potentially interrupting services to customers while performing necessary digital forensics. An organisation which has not planned ahead finds itself paying emergency labour costs, combined with loss of revenue. Contrast this with an organisation which does plan ahead, accepting that security incidents will happen at indeterminate points in the future. The conscientious Chief Information Officer and Chief Information Security Officer work together to develop a robust incident response plan which aids digital forensics and potentially negates or reduces downtime.

Your organisation already prepares for other disasters: your offices conduct fire drills or other testing of emergency services to ensure the safety of employees. Why not apply the same rigorous philosophy to your IT and security? Schedule periods of downtime on nights or weekends to conduct stress-tests or incident-response drills. Test your policies, procedures and capabilities to recover from disasters in controlled conditions. Doing so will make the difference between an expedient resolution and catastrophic downtime during an incident.

While calculating the costs of Information Technology and Information Security best practices and your IT budget, think about them in contrast to the rest of your organisation’s overall financial health: it is more practical to pay the salary of competent security personnel than it is to pay settlements after a lawsuit.

Uptown, Downtown

As your organisation grows, uptime will consist of more than providing a backup generator to power your network and systems – it may expand to include features like Internet-based storage solutions, content distribution systems, third party application development and complex interconnectivity with other organisations. Each of these features supplies critical functionality to your organisation, and each feature represents additional opportunities for enterprising criminals to take advantage of lax security.

The more critical a system, the more important it is to isolate said system from any single point of failure. This is especially true in situations where people’s lives depend on it: modern aircraft are equipped with multiple redundant systems. The strategy of active redundancy has been widely recognised as the single most effective solution for providing high availability of critical functionality.

Implementing and distributing your organisation’s network and systems infrastructure in ways that attempt to ensure the availability of core functionality, even during an equipment failure, can mean the difference between a positive quarterly earnings statement and a negative one. Single point of failure applies to nearly every aspect of your Information Technology and Information Security infrastructure.

Employees, administrative passwords, electrical power, internet connectivity, data storage, data archival and many other factors contribute to this process, and each aspect raises security concerns. Which employees have which passwords? How much of our infrastructure is documented? Who do we have on-call in case of emergencies? Which services require a redundant Internet connection, and which provide a second entrance for the bad guys? Are all the backups stored in the building across the street which was also destroyed by the same flood? Who has access to the data-center and networking closets? Who is writing all of this down so we know what to do in the future?

As each aspect is considered in detail, their weaknesses become apparent and remedies reveal themselves to the thinker: Granular access controls, frequent documentation review, a robust VPN to allow remote troubleshooting, appropriately capable security appliances, off-site or Internet-based archival solutions, sound physical security, codified security standards which enforce company-wide best-practices for security and technology… the list is extensive and intimidating, yet the individual components are actually very simple!

A methodical approach to best-practices by examining functionality of your Information Technology infrastructure will lead to observations and improvements, both in security and efficiency.

A responsible organisation ensures its success in the Age of Information by espousing – and enforcing – sound security practices at every step, from software development to physical security: shortcuts lead only to pitfalls.

Profit, Loss

In the Age of Information, service security and availability are directly tied to the profit margin of many businesses, especially for those not directly in the technology sector: retail giants are particularly vulnerable to cascading profit loss, as was observed with Target’s plummeting revenue, scorn in the public eye and general consumer distrust.

A security breach that results in customer data spillage is bad enough – to add insult to injury, some security threats are even more insidious: the reviled malware Cryptolocker encrypts the files of users unfortunate to fall victim to its lures, drawing clicks with simple deceptions such as sending copies to itself to your contact list. Imagine how serious such a situation could be, in which a negligent administrator browsing the Web on a production server contracts malware, rendering those services compromised.

myriad of potential ways that your organisation might be compromised exist, some more likely than others. Some situations are inevitable, while other risks are existential. Businesses experiencing data breaches are near daily events: the risk of such an event is far from existential. Natural disasters highlight nature’s probabilistic behavior, which conveniently models the chaotic world of security threats. Some may be obvious, others appear seemingly from nowhere.  It’s up to your CISO and CIO to determine which risks to address, based on their potential impact to your organisation. Consider both short and long term aspects of each scenario, and plan accordingly.

Catastrophic downtime, either due to poor decision-making or malicious intrusion harms an organisation immediately, yet the lasting damage comes from public opinion. More on that next time!