Deliberate practice in information security

Deliberate practice is the act of performing a set of tasks that are just slightly more difficult than what you’re used to, so you can get better at a specific activity and move from a novice to an experienced practitioner. Throughout my career, I’ve considered how information security professionals can use deliberate practice to get better at their craft, and how mentors can help to improve the skills of novices. In the context of our security team at Fastly, working on the fundamentals by deliberate practice helps us fulfill our mission of defending the modern web; we leverage our ever-improving expertise in order to better protect our customers.

Musicians and athletes are quite familiar with deliberate practice; for example, musicians improve their skills by playing musical scales, which allows them to train their fingers to play common patterns that are seen in many musical compositions. Because of this, they’re able to quickly learn new music, and become better musicians overall. In sports, professional tennis players practice using the speed ladder to improve their footwork, so they can serve and volley more effectively. Deliberate practice increases an individual’s ability to perform well when it counts. So, if we can break down the fundamental skills that are necessary to becoming an effective information security professional, then we can apply the idea of deliberate practice to information security. However, before we can think about the fundamental skills, we first need to determine what separates effective experienced security professionals from novices.

Developing mental models

A key difference between novice and experienced practitioners is their use of mental models, which are extremely powerful tools that help sort out what’s relevant in a given situation. Experienced practitioners have much better developed mental models of systems with which they have previously worked. Their mental models tend to be concise and have just the right amount of detail, while novices tend to include extraneous details, which slows down their ability to use their mental model to analyze a given situation.

In addition, experienced practitioners decrease their learning curve when they use previously developed mental models, and compare aspects that are similar to the current system they are learning about. Most importantly, these well-developed mental models help experienced practitioners predict what will happen to the system, given some change, and to determine what’s out of place when something unexpected happens to that system — such as when, in a previous role, I found a security vulnerability in an in-house developed web server by noticing an unexpected TCP socket state on the server side, while the web server was processing a client request.

A framework for deliberate practice

Given that mental models are useful tools, how can novices quickly improve their own mental models? We need a framework, for which two key things are required: using the scientific method as a process and becoming well versed in using tools of observation.

The scientific method, as a process, can be used as a methodical way for improving our existing mental models. The first step is to examine your current mental model of a system and ask whether some aspect of it is accurate. Then, form a hypothesis about your current mental model, and use a tool of observation to observe and gather data. Last, reflect on whether your current mental model of that system is accurate, given the data that you’ve gathered so far. If it’s not, update your mental model accordingly.

For instance, before I joined Fastly, I was asked to quickly learn a large codebase for a web server developed in-house, that was written in C++. In order to learn an efficient method for quickly understanding new systems, I did some research and came upon the concept of threat modeling. I created a threat model of this web server, and after examining the parts of the code that handled input, I created a hypothesis that I would be able to overwrite the instruction pointer of one of the worker threads, if I gave the web server some unexpectedly long input. Next, I tested my hypothesis by writing a mock client in Python and sending over the unexpectedly long input. I noticed that I was not able to overwrite the instruction pointer as I had thought, however, I noticed that both the client and server were not responding as I’d expected. So, I used the netstat tool to gather information about the web server’s TCP socket state. After using the Visual Studio debugging tool to gather even more information, I was able to confirm that the web server was susceptible to resource exhaustion attacks. In short, I used the scientific method to methodically understand a large codebase, with the bonus that I was able to find a security vulnerability.

Being efficient at using tools of observation is key to quickly updating mental models, and experienced practitioners excel at it — in order for novices to improve efficiently, they need to work on deliberately practicing using observation tools.

If you’re a novice, one way to deliberately practice is to start off thinking about a current mental model of a system. Next, get feedback from experienced practitioners about the accuracy of that mental model and ask what kinds of tools of observation are needed in order to confirm or modify the mental model. Out of the set of observation tools, pick one and study how to use it. Set up practice exercises that stretch your abilities of using that tool. For example, learn how to use tcpdump in various situations, and become familiar with creating filters.

Note: if you’re not mentally drained after a practice session, you weren’t practicing deliberately!

Mentorship

Once you’ve mastered deliberate practice, be sure to pass your skills along — you can help guide novices towards deliberate practice by encouraging the use of the scientific method to develop their mental models. Ask them to describe and draw (if possible) the mental model, and ask about all the assumptions they have made. Then, for each assumption, ask questions to determine whether those assumptions are reasonable. Next, encourage and guide them in learning the most important features of each of the tools of observation related to the mental model that you’re both examining.

Without a framework for improving, it can be very difficult to enhance our information security skills. However, if we keep in mind that experienced practitioners have well-developed mental models of systems with which they’ve previously worked, then we have a strong foundation to start from. Improving our mental models means using tools of observation to gather feedback information, and consciously modifying those mental models from that feedback using the scientific method. Mentors can support novices’ growth by encouraging them to become familiar with the scientific method and the appropriate tools of observation. Most importantly, novice practitioners can quickly become better by deliberately practicing using those tools.