Disastrous Practices – Part 1

By Patrick McCulley |

Introduction

Last week’s article emphasized the importance of ensuring that your CIO is properly vetted for their position. This week’s article expounds on the importance CIO and CISO roles play in the health of your business – especially as a startup. Your CIO and CISO are responsible for top-down enforcement of best practices in engineering and security alike.

Battles and Wars

“The general who wins a battle makes many calculations in his temple before the battle is fought. The general who loses a battle makes but few calculations beforehand. Thus do many calculations lead to victory, and few calculations to defeat; how much more no calculation at all! It is by attention to this point that I can foresee who is likely to win or lose.” – Sun Tzu

The health of an organisation’s Information Technology infrastructure is directly tied to the amount of foresight, insight and planning dedicated it. A poorly planned IT infrastructure will be riddled with haphazard fixes, unplanned downtime and general malaise. The wiring closets will be rats’ nests, the server rooms cluttered and disorganised, doors left unlocked and admin rights given leisurely. In contrast, an organisation which staunchly adheres to best practices will find itself prepared for almost any scenario, with thorough documentation, well-thought policies and codified standard operating procedures. Its wiring closets are kept neat and labeled, the server room identically maintained with professional precision. Critical infrastructure doors are secured against unauthorised personnel, and admin rights granted only to vetted IT staff.

The two most widely recognisable standards are the Information Technology Infrastructure Library Version 3 and ISO 27000 (ISO 27001 and 27002 for now). These libraries are not only comprehensive, but are widely accessible to individuals of beginning and advanced skill levels. The concepts cover from bottom to top and vice versa, enabling detailed planning, design and execution, saving your organisation (and its staff) from potential show-stopping oversights in the future. Enthusiastic adoption and rigorous adherence to codified standards and procedures is the first step in ensuring that Information Technology serves as the backbone to your business model, instead of the ball-and-chain around the company’s ankle. ISO 27001 and 27002 deal specifically with the realm of Information Security and are roundly beneficial to their practitioners.

In Information Technology, best practices win the war by fighting many small battles in a methodical and organised fashion.

Area of Responsibility

“In times like the present, men should utter nothing for which they would not willingly be responsible through time and eternity.” – Abraham Lincoln

Prior to the age of information, the chief information officer could get by on “soft skills” alone. They were the people-skills individuals who merely oversaw other workers but did not participate in the engineering process.

The time of the soft-skills CIO and the soft-skills IT Manager is over: the time of the Information Technologist is here, now. The complexity and nuances of Information Technology and Information Security demand the hard skills and intuitive thinking provided by career technologists – the very skills and foresight lacked by nontechnical personnel. 

As Defense in Depth is a strategy to implement countermeasures in every practical layer of your infrastructure, Qualification in Depth should be your strategy towards staffing your Information Technology division. From the top down and bottom up, every role should be filled with personnel who demonstrate sound understanding of their area of responsibility. In addition, the CIO and CISO must be capable of gauging technical limitations, usefulness and potential impact of emerging technologies upon the organisation and its infrastructure.

When you, as a CIO or CISO, propose an idea or technology to the board of directors, take responsibility for the preparation, planning, design, implementation, operation and optimization of said proposal. Part of the appeal of Information Technology is its ability to serve a useful purpose in many places – even within procedural philosophies: The elements of the process just described are from Cisco Systems’ Network Lifecycle model, yet their broad concepts apply quite conveniently to almost any area of Information Technology. The ITIL v3 thoroughly covers this process from a general IT standpoint as well.

As a CIO, CISO or IT manager, your area of responsibility is not only the domain you manage, but the domains of those you manage. A helpdesk manager should display a solid understanding of not only the role of a helpdesk representative, but also the experience of having held such a position. As the CISO, this means having the required technical knowledge to suggest appropriate defensive or mitigation measures to the CIO. As the CIO, this means having the skills and experience to understand when and how technologically unsound ideas will cause problems for the company in the future. Research your ideas before proposing them. Vet your requirements before being asked for them. Have the answers to the questions before you are asked them: ask yourself the questions your colleagues may ask. 

Ultimately, the responsibility for each layer of Information Technology in an organisation resides with its leadership: problems that affect only one layer may be reduced to holding that specific individual accountable, while issues which affect the entire organisation negatively (a data breach for example) go all the way to the CIO and CISO.

Document All The Things

“I basically wrote the code and the specs and documentation for how the client and server talked to each other.” – Tim Berners-Lee

For your Information Technology infrastructure to hum like a well-oiled machine, your administrators and engineers need the service manual.

In the age of information where millions are won or lost in milliseconds, time is money, and every second of a major service outage causes negative financial impact in a minimum of three areas: customers and customer-potentials negatively affected, public attention to the outage via the Internet, and the salaries of your engineers working to fix the issue. When a major service disruption happens to your organisation, do you want your engineers tracing cables, or round-tabling the issue and resolving the problem? Thorough, accurate and frequently reviewed documentation pays for itself when disaster strikes. Excellent documentation can mean the difference between a one hour outage and a one day outage.

Direct your personnel to carefully document their progress at every step, from research to execution. Excellent documentation is not very difficult to begin early, yet is a procedural nightmare years after the fact. During a major service outage, your engineers will be isolating the problem through a process of elimination: encountering undocumented configurations significantly slows the troubleshooting process, leading to more downtime. If thorough documentation is not part of your IT infrastructure model, in almost comedic irony your engineers are forced to document configurations during an outage. Outages that run into paying employee overtime can cost millions and harm public opinion of your organisation – do you want to pay your engineers to document infrastructure during normal working hours, or when something breaks in the middle of the night?

Where does the CISO fit into all of this? Almost everywhere. Your CISO is there to evaluate all layers of your Information Technology infrastructure through every step of the process, and ensure adherence to best security practices whenever technically feasible, where ever possible.

Many modern businesses treat IT as a loss-generating division. Information Technology doesn’t cost your company money: Failing to adhere to best practices does. A proactive and prescient state of mind towards Information Technology and Information Security will help you lay the foundation for a truly Information Age business.