How to Review and Improve Your Cybersecurity Training Processes

Continuously improving your IT security training program is a very noble cause. However, most training leaders do not have sufficient processes in place (beyond what's in their own heads) to drive meaningful and coherent change.

First Published 12th April 2021  |  Latest Refresh 17th January 2023

Janet, Head of Learning for the Tech Department, reflects before taking her next step. Be like Janet.

6 min read  |  Reflare Research Team

Rear-view Mirror

Once you have delivered your cyber security training across the organisation, it is of critical importance (if you're looking to do it right) to spend some time assessing both the effectiveness of your approach, and the process of how you design your training program.

This will involve gathering input from senior stakeholders and carefully reflecting on their thoughts and experiences throughout working with you, your team, and the training initiative. This research article lists out the process steps you will take to craft a meaningful and actionable review of your entire training program. 

One, and-a Two, and-a Threeeee

There are three primary steps to conducting a thorough training program review. Firstly, you must Undertake a stakeholder feedback survey. Secondly, you will consolidate the survey feedback into a workable format that you can extract valuable analysis and insight from. Thirdly, from your analysis, you will produce a process improvement plan which will define the next steps for your next cycle of cyber security training.

Let's look at each of these steps individually.

Stakeholder Feedback Survey 

Senior stakeholders and leaders generally hate receiving feedback survey requests, as it is often viewed as a time-consuming, low ROI activity. However, cyber security is on the list of every organisation's top priority list. With such a business-critical issue, their input on your work will be seen as time well spent (and if they don't see it that way, maybe you should remind them what would happen to the jobs if there was a massive data leak). Regardless, the main point here is that when framed up correctly, a well-designed survey will invoke stakeholder contributions.

Your task is to build a low effort – high impact survey that statistically quantifies the four elements of your training program's creation and execution process.

Ask participants to indicate the level of agreement with each statement on a scale of 1–5 where: 1= Strongly Disagree, 2= Somewhat Disagree, 3= Neutral, 4= Somewhat Agree, and 5= Strongly Agree.

Modify the feedback survey as per your requirements by adding or deleting statements to the statements below. Ultimately, you should at least be aiming for no less than 10 survey participants to complete the survey. However, if your stakeholder account is greater than this, then you would include all your stakeholders to exceed the minimum participant count.

1) Determining goals for the cyber security training initiative 

-   Get executive buy-in and sponsorship to develop the required cyber security capabilities.

-   Identify cyber security training objectives and outcomes.

-   Identify cyber security capability development areas that align with strategy.

-   Create a timeline for the cyber security training initiative.

2) Building the cyber security training program 

-   Outline training curriculum and content.

-   Identify and select sponsors, trainers and vendors for cyber security training.

-   Create cyber security training content and materials.

3) Preparing participants and contributors for the cyber security training initiative 

-   Prepare cyber security training sponsors and trainers.

-   Prepare managers of trainees.

4) Launching and managing the cyber security training delivery 

-   Establish a baseline profile of trainees.

-   Hold training kick-off meeting and additional support meetings.

-   Organise final presentation with executive sponsor.

-   Debrief and evaluate the cyber security training initiative.

Download: This easy-to-use survey framework will help your stakeholders quickly evaluate each step of your process. (pdf)

Please note: You will notice that in this stakeholder survey there is little information about the actual training experience for trainees, and deliberately so. There is a separate framework for garnishing training feedback, which you can read about in our research article 'How to Ask for Honest User Feedback on your IT Security Training Program'. 

Survey Analysis and Insight 

Once all of your stakeholders have completed your survey, you can now begin to consolidate responses to show an aggregate of the survey results, therefore identifying specific strengths and weaknesses that exist within your training creation and delivery process.

You'll see in the graphic below an example of your survey analysis and insight framework. Your goal is to consolidate the scoring for each survey statement to gain a holistic view of your creation and delivery process across the stakeholder group.

With the aggregate score totals in hand, you can now begin to identify the specific areas of your process for evaluation and possible improvement and reflect on why you believe this is the experience of your organisation. Commit at least a few days to sit with this information, and document all of your thoughts on why you believe the feedback is the way it is.

Process Improvement Plan 

Now with your personal reflections documented, you can now begin to facilitate discussion amongst your core team, going line-by-line to identify specific actions in your process that could be improved, redesigned, or abandoned completely, based on the context of your stakeholder's experience.

Use these discussions to gain input from your team on possible process improvement, consider evaluating other IT security training vendors, and brainstorm entirely new ideas for making the process more fit-for-purpose.

Similarly, facilitate these same discussions with the stakeholders themselves. Share with them your aggregate scoring sheet and personal reflections, and listen to what they have to say. Be sure to capture all of the thoughts and suggestions, as this intelligence, along with your team's intelligence, is what will contribute to your process improvement journey.

Consolidate and populate your intelligence into four primary columns (Areas for Improvement, Identified Problems, Lessons Learnt, and Actionable Next Steps) with this coherent and simple framework below. Be sure to state specific actions you plan to undertake to close the gaps your stakeholders have identified.

It is the Actionable Next Steps column that becomes your process improvement plan. 

Design how you will implement your action plan, and resource it accordingly. It is also wise to check in with your stakeholder group periodically, to ensure that the requirements of the organisation have not altered to a point where your improvements are deemed no longer relevant.

Cybersecurity is a fast-moving industry. Even though your stakeholders may be able to identify trends that challenge the direction of your improvement plan, there is a high probability that they do not know everything that's happening in the space.

Subscribe to Reflare's Research Newsletters when the latest in IT security trends, threats, and training best practices will periodically arrive in your inbox. This information will help you stay on top of your work as you lead into your next cybersecurity training cycle.