About Me

My photo
Experienced Information Technology leader, author, system administrator, and systems architect.

Monday, January 27, 2014

Secure Application Deployment in the Cloud

The cloud provides a great way for a company to push infrastructure costs to an external vendor. But things that are minor for a locally hosted application could become a huge security hole when hosted externally.

Some key issues to look at when moving an application to the cloud include:

  • Communication channels to services and systems the software relies on.
  • Communication channels used for necessary communications to clients.
  • Encryption standards for data at rest.
  • Logging, log reviews, and monitoring.
  • Authentication and access control.
  • Privacy policies.

A lot of the security scrutiny surrounding a cloud migration focuses on the security of the cloud provider's infrastructure itself. This is important, but the weaknesses that the software platform brings along with itself are almost certainly a bigger problem.

Communication Channel Security

The key considerations here have to do with the nature of this communication. Certain types of data should not be transmitted unencrypted across an external network. This includes information protected by the privacy policy and relevant regulations, but it may also include information that would tell someone how the application works.

There is really very little incentive not to encrypt all traffic. There is a performance hit, but the only responsible way to avoid it would be do perform a close analysis of all data that would not be encrypted. Even when the analysis was complete, you can't guarantee that the program won't change in a few months (even assuming that nothing was missed in the analysis). There are a number of options for forcing encrypted traffic, including built-in capabilities in both Java and .NET to force use of SSL for web interactions.

Where programs have incorporated hard-coded IP addresses in code, there is some possibility that traffic would be delivered to entirely the wrong place in a hosted environment. This is especially the case for the standard ranges that are commonly used for internal IP addresses.

But the use of hostnames can also be problematic, since name lookup infrastructure is usually controlled by the outside vendor. (In any case, references to specific names should be contained within configuration files, not in the actual code source.)

Where possible, client-side SSL certificates can provide an extra layer of security, by providing assurance that the target side of the connection is actually the system that we are trying to contact.

Data Encryption

Data at rest can be secured using several technologies, some of which overlap. SQL Server and Oracle both provide Transparent Data Encryption (TDE), and DB2 provides similar functionality. Make sure key sizes are in line with current best practices recommendations.

Queries to databases can be encrypted during transmission by specifying SSL as the connection protocol in the JDBC driver or .NET connection.

Keep in mind that existing hard-coded encryption tokens, keys, etc may cause problems during application migration to the cloud. And if the same key is re-used in several contexts, the compromise of a single component can result in a broader compromise through the entire application or environment. It is important that encryption keys, tokens, etc be maintained outside of the code base itself, where they can be changed or updated as needed.

Logging Considerations

Logging streams usually do not use connection-oriented protocols. One concern about logging in the cloud is that logging streams are relatively easy to divert or snoop. Debug-level information might be considered an information leak about how the parts of your application communicate, which could provide the information an attacker needs. While we need to allow an adequate level of logging, we may also want to restrict access that would allow too high a level of logging to be enabled.

At the same time, it is important that logs be maintained and reviewed, just as they should be on an internal network. Given the potentially greater exposure of the data, log review procedures need a careful review as part of any application cloud deployment.

Thursday, January 16, 2014

Effective System Monitoring

In order to maintain a reliable IT environment, every enterprise needs to set up an effective monitoring regime.

A common mistake by new monitoring administrators is to alert on everything. This is an ineffective strategy for several reasons. For starters, it may result in higher telecom charges for passing large numbers of alerts. Passing tons of irrelevant alerts will impact team morale. And, no matter how dedicated your team is, you are guaranteed to reach a state where alerts will start being ignored because "they're all garbage anyway."

For example, it is common for non-technical managers to want to send alerts to the systems team when system CPU hits 100%. But, from a technical perspective, this is absurd:

  • You are paying for a certain system capacity. Some applications (especially ones with extensive calculations) will use the full capacity of the system. This is a GOOD thing, since it means the calculations will be done sooner.
  • What is it you are asking the alert recipient to do? Re-start the system? Kill the processes that are keeping the system busy? If there is nothing for a the systems staff to do in the immediate term, it should be reported in a summary report, not alerted.
  • If there is an indication (beyond a busy CPU) that there is a runaway process of some sort, the alert needs to go to the team that would make that determination and take necessary action.

In order to be effective, a monitoring strategy needs to be thought out. You may end up monitoring a lot of things just to establish baselines or to view growth over time. Some things you monitor will need to be checked out right away. It is important to know which is which.

Historical information should be logged and retained for examination on an as-needed basis. It is wise to set up automated regular reports (distributed via email or web) to keep an eye on historical system trends, but there is no reason to send alerts on this sort of information.

Availability information should be characterized and handled in an appropriate way, probably through a tiered system of notifications. Depending on the urgency, it may show up on a monitoring console, be rolled up in a daily summary report, or paged out to the on-call person. Some common types of information in this category include:

  • "Unusual" log messages. Defining what is "unusual" usually takes some time to tune whatever reporting system is being used. Some common tools include logwatch, swatch, and logcheck. Even though it takes time, your team will need to customize this list on their own systems.
  • Hardware faults. Depending on the hardware and software involved, the vendor will have provided monitoring hooks to allow you to identify when hardware is failing.
  • Availability failures. This includes things like ping monitoring or other types of connection monitoring that give a warning when a needed resource is no longer available.
  • Danger signs. Typically, this will include anything that your team has identified that indicates that the system is entering a danger zone. This may mean certain types of performance characteristics, or it may mean certain types of system behavior.

Alerting Strategy

Alerts can come in different shapes, depending on the requirements of the environment. It is very common for alerts to be configured to be sent to a paging queue, which may include escalations beyond a single on-call person.

(If possible, configure escalations into your alerting system, so that you are not dependent on a single person's cell phone for the availability of your entire enterprise. A typical escalation procedure would be for an unacknowledged alert to be sent up defined chain of escalation. For example, if the on-call person does not respond in 15 minutes, an alert may go to the entire group. If the alert is not acknowledged 15 minutes after that, the alert may go to the manager.)

In some environments, alerts are handled by a round-the-clock team that is sometimes called the Network Operations Center (NOC). The NOC will coordinate response to the issue, including an evaluation of the alert and any necessary escalations.

Before an alert is configured, the monitoring group should first make sure that the alert meets three important criteria. The alert should be:

  1. Important. If the issue being reported does not have an immediate impact, it should be included in a summary report, not alerted. Prioritize monitoring, alerting, and response by the level of risk to the organization.
  2. Urgent. If the issue does not need to have action taken right away, report it as part of a summary report.
  3. Actionable. If no action can be taken by the person who receives the alert, it should have been defined to be sent to the right person. (Or perhaps the issue should be reported in a summary report rather than sent through the alerting system.)

Tuesday, January 14, 2014

Courage in a Corporate Setting

Sometimes leaders need to leave their comfortable "safe zones" in order to be effective. The reality is that the bulk of our jobs can be done by almost anyone. Most decisions we make are of the "no brainer" variety, especially as we become more experienced and comfortable in our role as leaders. But there are a few decisions that we need to make where we really earn the money and privileges that come with a management role.

When we voluntarily step outside of our comfort zone to do what we know to be right, we demonstrate the courage that distinguishes between someone who is a leader and someone who is merely a boss.

Battlefield analogies are very common when we speak about courage. An article by Peter Voyer in Ivey Business Journal suggests some important leadership traits that translate from the battlefield to a corporate setting:

  • Don't ask subordinates to do something you would not do. Not only should you be willing to work alongside your team, you should be seen as someone who engages in the task at hand. (Of course, the way you engage the project will be somewhat different than the tasks you would assign a junior team member, but nobody on your team should feel like you are unwilling to dirty your hands to make the project succeed.)
  • Demonstrate moral fiber. You can lose years of built-up moral capital in a split second with a morally dubious decision.
  • React quickly, decisively, and fairly when presented with a moral question.
  • Maintain dignity and respect within and between groups.

I recently saw an article about a manager who was allegedly fired because he stood up for an Indian employee's right to earn the same salary as American employees with a similar job. This is the sort of courage we need if we want to be leaders and not merely bosses. Who do you want to see when you look in the mirror in the morning?

Great leaders earn the loyalty of the people who work with them. They earn loyalty by demonstrating loyalty. This doesn't mean that you cover for one of your subordinates who does something wrong; it does not help someone's development to infantilize them. But make sure that the consequences are fair and are implemented with the long-term development of your employee in mind. This may mean that you stand up for someone who has made a mistake and demand fair treatment for that person. Yes, this is uncomfortable, but it is part of how you become the manager you want to be.

Make sure you stay informed of your team's progress towards goals, and work with them to overcome obstacles. This does not mean that you do your team's work for them; it means that you provide a sounding board. Sometimes a problem is escalated to you if it is something that requires a manager's approval or advice; make sure that you do what you need to do promptly, then return the task to its rightful owner.

Maintain your integrity. Make the best decisions you can, and abide by the results of those decisions. Don't pass the blame. Instead, identify how to fix the situation and propose solutions.

Demonstrate courage by making the right decisions, even when they are hard. Anybody can be a great boss when the going is easy. Being a great leader comes from doing the right things even when they are not easy.

Monday, January 13, 2014

The Promise and Peril of Self-Driving Cars

The research by Google and others into self-driving cars has been intriguing. The vast majority of traffic accidents are the fault of drivers, and being able to eliminate human error would be a huge win for traffic safety.

But if computers are driving cars, we have to take a serious look at information security in the context of a self-driving automobile. Unfortunately, most current automation does not have adequate safeguards to protect from malicious inputs.

In particular, components do not do checking or validation to make sure that commands are being issued from an appropriate source. Security researchers have demonstrated that they are able to issue commands to a Prius to control steering, braking, acceleration, and dashboard displays. They were also able to disable an Escape's brakes at slow speed.

Ford and Toyota both point out that the researchers were connecting directly to the car's CAN (Controller Area Network), which limits the impact of some of their demonstrations. But keep in mind that wireless controllers on on-board systems such as Bluetooth controllers on sound systems and telematics units on satellite roadside assistance services may provide an entry point into the automobile. Anywhere a wireless connection allows access to a component connected to a CAN is a possible entry point for malicious code.

The sorts of security measures we use for other network-connected items would still work inside a car. Provide air gaps between components that don't need to be connected. And provide for validation and authentication of commands from components that do need to be connected.

I remember discussions about PC security in the early days of the Internet, when most computer viruses were still spread by injudicious insertion of floppy disks. Way back when, we were told that PCs didn't need to have security programmed in from the ground up. I'm hoping we learn from the history of those poor decisions. A Blue Screen of Death is one thing, but a traffic fatality is another.

Monday, June 24, 2013

The Risk from Insiders

There has been a lot of coverage of the role of a system administrator in the recent release of information about the National Security Administration's intelligence gathering methods. Regardless what you think about the methods that were revealed, information security professionals need to take a hard look at the sorts of exposures that exist due to organizational insiders.

Snowden's position as a system administrator is just the most recent high-profile insider who betrayed his employer's trust. His removal of documents on a thumb drive was viewed as unsuspicious precisely because of his job function.

As long as we have an IT infrastructure, the people who manage it will be in a privileged position. IT professionals recognize the risk; four of five professionals in a recent survey list insiders as the greatest source of risk to the environment.

The same methods that are used elsewhere in the security landscape will help to control and mitigate the risk from insiders. At a high level, there are three steps that need to be taken:

  1. Data Classification: Identify the types of data in your environment, and what the confidentiality, integrity and availability requirements are for each type of data. NIST 800-60 can provide some guidance here.
  2. Establish Control Standards: For the different types of data, we need to describe the measures that are taken to protect the data.
  3. Audit: The controls need to be evaluated for effectiveness, and the organization's compliance with the the controls must be verified on a regular basis.

Controls

There are several publicly available documents outlining control best practices and standards. Here are a few:

Some common controls include:

  • Physical access controls: Things like security guards, mantraps, proximity card systems, and combination locks on doors control physical access to sensitive areas and systems.
  • Logical access controls: In general, people should only have the level of access required for their jobs. Access controls should be as granular as possible, and high-level access should require extra levels of approval and scrutiny. Two-factor authentication should be in place for access to sensitive facilities.
  • Personnel management: Some common measures include criminal background checks, periodic security awareness training, contractual attestations, and organizational communications.
  • Separation of duties: Where possible, access should be limited to particular functions, and functions should be defined to limit access to sensitive data. In general, developers should not have access to production, system administrators don't need database access, application administrators don't need system-level access, and only the people who manage the hardware and network need physical access to the systems.
  • Network security: The network should be segmented appropriately, and firewall rules should be in place to restrict traffic between different security zones.
  • Workstations and laptops: Hard drives should have robust encryption and strong password policies should be in place. The types of data that are permitted for local storage should be established and monitored. The ability of end users to install applications needs to be restricted. Patches, anti-virus updates, and security workarounds need to be applied regularly.
  • Backups and continuity: Data needs to be protected by a combination of archival backups, long-distance replication, and local disk mirroring/RAID-ing.
  • Logging and auditing: Logs need to be collected to measure the effectiveness of these controls, and the logs need to be reviewed on a regular basis.

Some controls should get particular attention as directly addressing the issue of insider-led breaches.

It is bad enough that the security training level of such government employees is not monitored by the DHS. (Most parts of the private sector also don't track administrator security training, for that matter). Beyond carelessness or incompetence, employers need to consider the direct risks posed by their most trusted employees.