This piece is part of our ‘Faculty Focus' series, an interview-style article where a member of the IANS Faculty shares firsthand, practitioner-based insights on an infosec topic. In this feature, Josh More discusses common vendor assessment challenges and provides best practices to make the vendor assessment process much more efficient and effective.
Five questions with IANS Faculty member, Josh More
Josh More is the Owner and President of Eyra Security, an information security and business improvement consulting firm that specializes in helping startups and organizations
in transition take advantage of lean and agile methods, open source technology, and varied frameworks used for security, risk management, and compliance. He also serves on the GIAC Advisory Board. Additionally, as an active member in the information
security community, Josh is a member of ISSA, Agile Iowa, OWASP, DC612, Central Iowa Area Linux Users Group and Infragard.
1. How would you describe the current state of vendor assessments in the security landscape? What have your clients experienced?
Josh: I mostly get involved when the vendor management program breaks down. Often, the problem is that there are just too many vendors to process in a timely manner. However, there are also concerns about the validity of the process, how to get better
information, how to make better decisions, and how to actually drive change. Industry-wide, the biggest commonality is that companies are spending a lot of money and time on third-party assessment and aren't getting anywhere near that much value out
of it.
A lot of companies are also still tiering vendors, using the theory that assessing a "high risk" vendor annually and a "low risk" vendor every three years is effective. This leaves aside the entire question of why anyone would do business with a vendor
they describe as "high risk." Vendors that engage in business practices that significantly raise your risk need to be continuously monitored, not checked annually. Moreover, vendors that only engage in “low risk” activities might not need
any review past verification that they still only engage in “low risk” activities.
2. What are the main challenges in performing vendor assessments? What are some of the results?
Josh: Vendor responsiveness is certainly an issue, as is how well you can trust the responses they provide. The internal cost of the program is another big issue. The most common result is that someone knowledgeable and skilled starts up a program and
either leaves or gets moved to another program, and companies try to run the program with less experienced, lower cost resources.
Another common issue is the belief that collecting more data will allow for better analysis and result in a more effective program. To be completely frank - it does not. For the vast majority of assessments, the decision you can make after 30 minutes
of assessment is the same decision you'd make after 30 hours.
Unfortunately, these two challenges tend to combine in truly awful ways, where you have one team of unskilled people sending 200 question questionnaires to a vendor, who employs their own team of unskilled people to copy/paste answers and send the questionnaire
back, where it's reviewed and sent back with more questions. And back and forth it goes, like a tennis ball, doing nothing to improve security and just raising the cost of the entire effort.
3. How do security teams currently address these issues?
Josh: Security teams often try to address perceived problems with their programs through adding more questions, ostensibly to provide "better coverage", and through automation. Unfortunately, the companies that provide automation software attempt to show
their value by adding even more questions, effectively automating the wrong part of the problem.
To deal with the deluge of the data, they then try to wrap a scoring paradigm into the model. I’m all for decent metrics, but if you are only measuring a company against an arbitrary set of questions, you are not actually measuring anything useful.
Security is contextual, and understanding the context is not something that works at scale - which is why the software vendors, service vendors, and even scoring vendors tend to not provide the value you might think.
4. What is a better vendor assessment solution?
Josh: The number one thing is to focus on contextualizing the vendors. Sometimes I call this "tagging" or "identifying vendor attributes" - it doesn't matter what you call it. The point is that you need a standardized way to contextualize the vendors,
effectively making a profile for every single vendor in your environment, so when it comes to assessing them, you can do so contextually. This also allows you to consider internal and external actions that can be taken by your organization and the
vendor to reduce the total risk in each contextual area.
As you do so, you will find that some contexts have very little security impact and others are significant. With a limited staff, just focusing on profiling/re-profiling every vendor on a regular basis can allow you to identify those critical security
contexts. If you then focus on assessing just those higher-risk contexts, you might only assess 5% of your vendors - but you're focusing where your risk is. This is lot better than building a big queue of vendors and getting through them as best you
can, based on sporadic availability.
5. What are the benefits of improving vendor assessments?
Josh: By taking a contextualization/profiling approach and then targeting the assessments, the organization moves much faster. With security aligned to the organization's actual needs, via the profiling process, vendor assessment is more efficient and
more effective, as are other aspects of the vendor relationship - such as how the vendor interacts with disaster management and incident management activities with your organization.
Streamline Your Vendor Selection Decisions
IANS Vendor Assessment Community provides an unbiased perspective from both IANS Faculty and IANS customer peers on vendor solutions. Save time with research and tools developed to compare vendors with market overviews, selection criteria, case studies
and implementation advice. Get in touch to learn how IANS can help make vendor selection more efficient
and maximize the ROI of your vendor spend.
About the IANS Faculty
Our Faculty are comprised of over 100 renowned security practitioners with deep, domain-based knowledge who understand - firsthand - the challenges faced by CISOs and their teams.
IANS connects clients with Faculty to help them make better decisions, grow professionally, save time & stay compliant. Get in touch to
learn more about how we can help move your security program forward.
Although reasonable efforts will be made to ensure the completeness and accuracy of the information contained in our blog posts, no liability can be accepted by IANS or our Faculty members for the results of any actions taken by individuals or firms in
connection with such information, opinions, or advice.