Validating and Proving Skills ROI – LMS Showcase Recap

I'm welcoming you from Australia, where we have just a whole bunch of deadly animals and deadly creatures that nobody wants interact with. Well, that's what we told tourists anyway, so they don't come here. So it's all to ourselves. Thanks everyone. I will now share my screen. And shall have a look at what we're going to cover off today. So just to let you know the validating improving the ROI of skills is really one of five pillars of what we call our learning effectiveness index. So if we think about learning it impacts business fundamentally. It's the one lever you can pull to consistently improve business performance. And we think about the other sort of five pillars there's learning access and engagement that looks at access to learning and how quickly and how best we engage people. There's learning performance outcomes, so how do we actually improve the performance of learning and the outcomes that are directly driven through the learning process. We then look at pillar number three, and that's where we're going to focus today, which is the workforce capability and readiness piece. And if we think about skills, mean that's what skills is all about, workforce capability and readiness for the organisation. But that does feed into the next pillar, which is our operational efficiency. So how well are we actually managing our learning and how well are we driving the development of our learning and the investment in our skills etc to then apply that to the fifth pillar which is around business impact. So we think about business impact and value recognised recognising value. That's really the objective of what we're trying to do. So when we think about today's session it's really pillar number three in a five pillar strategy. All right and we think about it your job as a learning development professional is really around curating the delivery of learning as it relates to supporting a skills demand with skill supply gap. And so you want to balance skills demand with skill supply and your tools that you're using are skills curation. And we're going to talk about that, some of the tools you can use to curate your skills more effectively, and then how do you value that? How do you actually determine the ROI of that? Alright, so let's move on. So, first thing I'm going look at this in three lenses. The first one is the challenge. The second one is the tools that you can use. And the third one is around how you can start to value the investment in those tools and the outcome from those tools as it relates to skills development. So the first one is the challenge and it's not easy, a bit like picking up this barbell, and that's really because things have changed. So back in the day we used to have a very standard sort of jobs framework which all jobs were little boxes and those boxes were very clearly defined, had a very clear sort of begin and end state, they were very clearly joined with other boxes that were other jobs. And it was clear in terms of the progression from job to job. People also came in little boxes. They were pretty much predefined in terms of skills and experience, and sure there was some nuance to that, but ultimately it was relatively easy to place people into jobs because there was sort of this linear career progression. And the ROI from that was actually quite significant because it wasn't that complex and placing people in the jobs was a lot easier. And it didn't require a whole bunch of systemisation to facilitate that outcome. So I always think about it as likening you to fitting pieces of a puzzle together where you start from the edges and it's pretty easy to work out which the edge pieces are. All the puzzles have a very clearly sort of defined shape and images and colours and things, so pulling it together is relatively straightforward. However, that's not the case now. Now it's like this right, so it's harder to try and match the right people to the right jobs because the difficulties on both sides are the people side and on the job side. They're no longer very clearly defined. And more importantly, it keeps changing. So imagine if you're trying to work in a spherical jigsaw puzzle where all the colours were the same. It's pretty tough, I don't know if you've ever tried it, I have. I gave up halfway through. But it's something that we're faced with day by day as learning development professionals is how do we facilitate getting people into jobs jobs through the lens of skills and skills application. And if we think about the jobs, the problem is with the jobs on one side of the equation which is they're no longer boxes, they're now I call them blobs where those job blobs are not well defined, they could be changing size and shape constantly, they overlap other jobs, there's not that sort of consistency which we saw in the past in terms of the way that the jobs operated or even the jobs interface with each other. At the same time people have changed they no longer come in boxes they're all shapes and sizes and they also come with different requirements, different experiences, as well as different expectations around what they're going to do for their career, how they're going to add value for you and your organisation, and what value are they going to extract from that. And so matching all that up is a bit like spaghetti, right? It's really hard to do and certainly without some form of systemised framework it's almost impossible to do at scale. So the only way to do scale is pattern matching, where effectively we're matching people to jobs based on getting best fit for the pattern. And the pattern will consist of what skills and experiences and education etc do we need for those jobs versus what do we have and what gaps do we need to fill and how quickly do we need to fill those so we can minimise the impact or maximise the value of getting that right person in the right job and keeping them there as long as we possibly can. At the moment, without systemisation, doing that has extremely variable ROI from two perspectives. One is it costs you a lot of money to try and do it. The next one is it actually has a high failure rate. And that failure rate results in people leaving the organisation, also results in missed opportunities, missed KPIs, etcetera, which all have a business impact. And so pattern matching the right people to the right jobs and doing it at scale is absolutely critical in order for you to positively impact return on investment of the effort of getting the people into the jobs. So if you think about that, what does that need? What does that require you to have? Well, if you've a patent match, you really need two things. You need a model of which you're going to then define a patent, and then you need the data in order to be able to overlay that with the pattern to then match the right people to the right jobs with the right skills. And so to do that you need to systemise. But your job in that process is really about, as I mentioned before, it's matching skills demand with skill supply and you do that through the process of skills curation. And skills curation is absolutely critical for this seesaw because curating skills is not a one off thing. You can't just do it once. You've got to constantly do it because the demand is going to change. Those job blobs are going to constantly get bigger, smaller, change, interface with other jobs. New ones are going to be created because things happen and then existing ones are going to disappear because there's no longer a requirement for that job. So that skill supply piece and being able to do that dynamically is critical. Now skills curation is possible and most organisations do it, maybe not necessarily effectively, but at least they do it. The challenge is that the dynamic pace of change of demand means that doing that manually is really, really hard. And trying to scale that is almost impossible. So this is where being able to use tools, I. Artificial intelligence, in the right sense, right, will help with that skills pattern matching. Because ultimately, that's what AI really is at its heart, is it's a pattern matching machine which is far more efficient than your eye. And so being able to define what is it that I need with what I've got and join those two together is what AI excels at. And so your job is to facilitate that relationship between supply and demand by using the right tools to help get that outcome. So let's look at the tools you need. The good news is it doesn't have to be as big as what you're seeing on this workbench, right? You don't need as many tools, fortunately. You just need the right tools for the right job. So when I look at the tools you need, you first have to look at the model, right, and the ecosystem that underpins it. And I'm going look at what we do, so how we do it with our existing customers. And so we call it our skills curation ecosystem because it is an ecosystem. And it starts at the very bottom with the skills. So skills come in all shapes and sizes, a bit like our people and our jobs. But they come from various different sources. So you've got LinkedIn, SFIA, Workday, etcetera. You know, these are all providers of a skills taxonomy. They also provide the content that's tagged to the skills taxonomy as well, certainly in the case of LinkedIn. On the right hand side you've got skills technology providers who are already inferring skills relationships and are providing that to organisations and tools like ours, so that becomes a source of the skills taxonomy. But every single organisation has, that we've dealt with at least today, has more than one skill taxonomy that they operate with. And a lot of those are custom skills. So being able to define custom skills and align custom skills and acquire custom skills is really important. If we think about the skills, they sort of fit into two buckets, durable or human skills, I got told off the other day of calling them soft skills, but you might know them as soft skills or power skills. And then there's technical skills and I've always liked to think of durable skills sort of shaping how technical skills are used. And durable and technical skills are also got their own seesaw in terms of at what stage of my career am I focusing more on using and developing technical skills versus durable skills. But they're both important. To get those into your model, you need to be able to ingest those. So our skills ingestion model, we actually call our skills funnel, is really ingesting the skills from multiple sources, but more importantly, normalising those skills. And when I talk about normalising, I'm really talking about smoothing those skills or joining those skills together so that a particular skill in LinkedIn can be equated with a skill in Workday's taxonomy. And that way when you are developing a skill, you're really looking at the skill, not who is providing the skill, and then it's linked back to the taxonomy, Workday's taxonomy, LinkedIn's taxonomy, etcetera, so that you don't have to make that a problem of the individual trying to find which taxonomy and which skill. They're just interested in the skill. So giving them the tools to develop the skill is more important and that normalisation piece is a critical missing piece of most skills models functions because if you can't normalise the skills across multiple skills taxonomies, you really can't use it at scale and it becomes an inhibiting factor. Same time you want to categorise skills so people know which skills to develop and you can align them with things like your roles, functions and tasks. And then assigning those skills to roles, functions and tasks or to people or to job areas or to parts of your organisation, automating that is the essential ingredient to being able to manage a robust skills model. If you can't do that, then you're not going to get value from your investment in a skills taxonomy for example, or taxonomies. You also have then the skill supply part, so that's how do I get the skills. And you'll notice I've called out two skill types in particular. So endorse skills where I might endorse myself or my manager or my peers may endorse me for a skill, they don't necessarily have to have evidence, they're just saying, hey, I think you've got that skill so I'm going give it to you through endorsement. Most systems provide that. It has value for sure, but it also is limited, particularly when it comes to highly regulated processes, critical roles, or where highly developed technical skills are required to be not just acquired, but they're also required to be validated. And so a validation, which we do through several tools I'll take you through in a minute, that allows us to say, okay, not only are we going to give you the skill, but we're going to consistently not just give you the skill, but then validate that you have that skill through a process of subject matter expert validation and then sign off. So that it is something that has value, it also has validity, and behind it it has evidence. And AI helps drive the assignment of those skills as well as to a large degree now actually validating those skills as well. And then the last one on the left hand side is performance skills, and that's a new area for us where we're actually looking, we're actually acquiring information from multiple different external sources that are driven from operational processes. And we're using the volume and quality metrics for that to then determine, okay, can we infer a skill based on how many times you've done this action and then at what level of quality. And I'll give you an example, so you're a software developer and you're checking in your software code through GitHub. So we know how many check ins you're doing, we also know the quality of those check ins, so how many bugs are being found? How many check ins are failing? How many check ins have been accepted? And aligning an AI driven matrix to that then allows us to infer the skills from what you're doing as part of your job. That's really powerful and a growing part of curating this model. And then finally you've got the agency piece which is really looking at you know what skills are we acquiring individually, so your skills backpack or your portfolio. Not only that, but giving you recommendations on how to close those gaps based on your aligned job roles or skills that you're developing or need to develop for your job role, or things you're interested in. So through things like persona surveys which are being automatically surfaced and then you're creating or completing persona surveys to then help drive what skills I'm trying to acquire. Plus also just what we're looking at in terms of tenure, what type of roles have you had in the past, what's your browsing history, all of those things are being used to say we think these skills are going to be important for you. On the job role skills side that's really about managing skills in the job role lens where you might have a job role or roles that have been assigned to you. So being able to say these are the skills we need versus these are the skills we have, joining those two things together, absolutely critical for determining do I have the capacity to meet my production or business goals? But also do I have enough coverage to be agile to take advantage of opportunities when they come up? And that agility piece is a really important part of building return on investment from your skills investment. And the last one is around doing that at scale where you're looking at org, skills maps, etcetera, where you're trying to map out, okay, what does our organisation look like in terms of capacity, in terms of gaps, in terms of gaps over time? Can we predict the future in terms of the skills we need to invest in? And then overlay that into our productivity goals or our business goals. If we can do that through our org skills mapping, that then gives us much, much greater direct line from skills investment to skills realisation or skills value realisation, which gives us our ROI metrics. So just quickly, some of these I'm going to demo today, some of these I won't, so there are a couple like skills funnel. A lot of getting the skills into the system it's actually back end so you don't really see it. But we did build our own AI agent to ingest, normalize and categorize the skills. The other one around CE skills is really about continuing education and that you know I could spend an entire half an hour on that one. But really the CE skills piece is looking at regulatory skills that are used to help manage licensing or qualifications and then helping to develop automatically your plan to close those skills gaps on an ongoing basis so you can maintain your license or credential. And these all generally work well together to build up that skills portfolio, that job role analysis and then that org skills mapping piece. All right, I'm going to jump out now and we're going to have a look at the system, and we're going look at it from the perspective of job roles. And so job roles in our system are really based around, you might have one or more job roles, and by the way job roles don't get hung up on the name, it's just a term. But in this case, I've got sales representative and I'm a solutions architect. And if I click through to have a look at my job role details, I can see that it's made up of some training, some knowledge and things that I need to know, some compliance programs. I've also got skills that I need to have and in this case, I can see that I actually have already achieved the level of skill that I need for the job. So in this case, I've got my proficiency level of level three that I need my required status is level three. So those match up and I already have that skill for critical thinking. But in other cases, I've got some gaps. You know, I've got some red gaps so I need to close those gaps and I can actually go and click through to find courses to close those gaps. So I'm going to look at the other job role now, which is looking at my solutions architect level four. So Priyanka, she's a solutions architect level four, as well as a sales representative. Probably unusual and unlikely, but it can happen. And in this case, I've got just skills that I'm looking at and I need to develop. So I can see that I have my collaboration skills here, I exceed expectations for this particular skill, but problem solving I'm below expectations. So I can actually click through here on the find courses and immediately it's already looking at the courses that have been tagged to that skill to help me close that skill. So the important thing is that it's already sort of linked together through that ingestion model and through those job role creations where I can actually see which skills I need at what level do I need for the jobs that I'm doing and then it gives me direct access to find the appropriate programs to close the skills gap. So in this case I've got my problem solving. I can see that I've got some courses here. I can actually click through or just launch the courses. And all of these courses that you're looking at here were all ingested from in this case LinkedIn Learning. So automatically ingesting and curating this behind the scenes is part of that model of being able to not just acquire the skills but maintain those skills through maintenance of the programs and maintenance of the skills taxonomies behind the scenes in an automated sense. And I can see that this has got a critical thinking alignment, artificial intelligence for business. Notice that when I was searching for it, it was actually not looking for critical thinking or artificial intelligence for business. But there is an inferred match back to that skills alignment that I went through as part of the job role selection. So, this is really powerful, used a lot in terms of helping drive people to the right courses at the right times. That job role mapping as well, when we start to look at it from the perspective of the learner and providing automated recommendations for skills, again this is all about building out return on investment by not having to manually curate this, but provide it through an automated means. So our system is using AI to then drive alignment and prioritization of skills that you need to close based on interests, also based on that job role alignment with skills opportunities. And these things are automatically being surfaced into recommended training based around those skills gaps. And I can then just interface with these or I can choose to get rid of it if I think it's not interesting or if I know other people are interested I could possibly share that with somebody and I could recommend it to colleague or colleagues. Going back to skills, if we look at my skills themselves, I also have a skills portfolio that I can look at which looks at skills which have been mapped to things like mandatory skills, so these are things that I have aligned with my job role or job role requirements, skills I must have in order to do my job, versus if I scroll all the way down to the bottom, I've got skills that I'm also adding in as additional skills or these skills may have been endorsed or submitted by me for endorsement by a manager and then manager or supervisors then approved it. So again this is the part of that left hand side of the equation where I'm getting those skills in. I'm also managing my skills acquisition based around aligning it with what I need to do for my jobs and then tracking how I'm tracking the skills gaps by looking at my proficiency level in those skills versus what I actually have. Pretty powerful. On the other side of the equation, we can look at from a managerial perspective, I'm managing a team and I actually want to manage my team's, development. So I can actually manage my team's development. Now I'm looking at, Priyanka's manager and I can see that I've got personal development plans that I'm either automatically being curated or I'm curating for them in order for me to develop skills for these people. And so in this case I can look at a a gap analysis which looks at the various different skills and you'll notice that these align back with those job roles. I can look at the skills, can look at the behavioural indicators, I can go into the full description of the themes, I can look at the level that they're I'm trying to get them to, or I can actually look at all the levels that they may be part of. And I can then use that to determine based on either evidence or based on my review of their performance whether they've met expectations or they're below expectations and I can complete these evaluations here. You can also see I can add comments as to where I think there's some additional work that needs to be done and specifically, you know, calling out actions or performance behaviours. I can then ask additional questions, in this case, you know, what role aspirations, etcetera, do you have? And then I'm going into my development discussion. Now, is the most important part because now I'm looking at my gap analysis. And in this case it's ordered from fit to gap. You can also order it from gap to fit. But now I'm actually looking at where am I going to focus and then how am I going to prioritize these gaps. So, I've already pre done that. By going down to the bottom and then going out into my review, this is arguably the most important part of it, which is now I'm developing an action plan. And I'm curating that and completing that with my team members. Now, the action plan is not just helping me to execute on gaps which we've already prioritized, but I can also add programs that are available that match the skills that the programs offer to the skills and the skill gaps at the proficiency levels that are gaps to the assessment of the individual. So, this is directly closing a gap. But if I select any one of these, for example, I can also choose when do I want this to be made available over. Now what this does is it doesn't just make available a program, which you can make available immediately by just putting quarter not required. But if you choose a particular quarter and let's just say a program isn't available yet, this this is used for demand generation because it goes directly through to the owners of the program who can then schedule the programs based on demand from this process. So you get return on investment not just in terms of closing the skills gaps, but you also get it from being able to more effectively manage your programme production. And that's really, really critical. On the other side of the coin, when we look at technical skills development, we're looking at sort of aligning that more to productivity. In this case with our OJT framework, OJT stands for on the job training. It's very much primarily focused on technical skills, but it doesn't have to be. This actually aligns do I have enough people with the right skills for the right tasks or tools that they need to use by shift, department, by crew, by production area. In this case, we're just looking at by shift. And I'm looking at do I have enough people who are certified to perform these various different tasks for this shift, for these various different tools or processes. And I can then do an immediate analysis because the system's already automating the prioritization. And by saying, for this one I need three, I've got zero. So now I need to look at, okay, what do I have to do to close that gap? Well, I can look at my expired people and I can recertify them. So this gives me the ability to see this information to then drill in and then complete the training plan or complete the skills evaluation or just sign them off so that they're actually ready to continue to provide that production service, So whether it's in this case using a particular assembly station. And the great thing is it's all consistent because whenever I look at the training plans or the skills evaluations, they're all tied directly to stand operating procedures. So when I'm validating the skill, I'm validating against a known set of constraints. And that's really important from an audit perspective. So, being able to do this and then manage it to that view that we saw before of the dashboard of do I have enough people or not? That's critical for operationally managing skills in an operational setting. And in fact, we actually even tie that into whether or not, you know, if you're certified or not certified on a particular tool requiring certain skills, if you don't have it, that tool does not turn on. And that's because we're managing the integration of the certification data based around that skills with the tools themselves. Also from a system perspective, we have the ability to tie skills to badges. So, I'm choosing one of our customers here, Red Hat. They have a very extensive skills framework where when you acquire skill that's then tied into a badge which effectively acts as a credential which gives you gigs or jobs. And those skills have been assigned back to the badge. So when you're earning a badge, you're not just earning a badge, it's great, you're actually earning skills, which are represented by the badge. And when we think about how they're acquiring those, they're doing it through our channels framework, which effectively provides what we call this, you know, academies. So in this case they've got an AI skills academy where you're building up AI skills by going through a very highly curated set of programmes which are curated by subject matter experts in AI, not by LMD professionals, so they're able to scale it. But also you're earning skills through accreditation as you progress through your development within the channel, which ultimately then earns you a badge. So being able to do this at scale is really important. And also being able to order things and even looking at what's most popular, etcetera. So this allows you to engage with content really quickly. Really powerful and is being used extensively at scale by Red Hat and other organisations. Alright. I do want to spend a little bit of time on analysis now, and this is really where the rubber hits the road and it's about Jerry Maguire, you've to show me the money. And a lot of what I do is about proving that the value of your investment in what we just showed is actually paying off. And I'll give it to you in a couple of examples. So in this example here, this is ROI validation of an organisation where they picked particular roles that were critical within their organisation, that they were struggling to maintain, and then invested in building out the skills framework for those. So, was clearly defined and mapped skills that were then acquired through that ingestion process, and then normalised and mapped to the roles. And then the various different tools we just showed around helping with that skills curation piece, then facilitated return on investment over time. The key thing to note here is it's not immediate, right? And if you think you're going to get immediate payoff from your skills investment, you may in localised areas, but as an organisation it takes time to build. But once it does, it's significant. So the build phase in this case took twelve months. The payoff period where we look at the initial build cost plus sustainment, so it doesn't happen for free forever, you still need to sustain model and that's where everything from curation of additional content, curation of your skills, investment in change, etc. All of that is a sustainment cost. But you do reach typically a payoff period, it's roughly about two and a half years we're seeing sort of on average. But at that point, you'll see the exponential growth. And this is looking at several different criteria that are really driving where is that return on investment actually coming from. We put together a whole document, I'll put together a checklist which I think we're leaking out to after this event, which really looks at what does that actually mean in reality? How do you value the components of return on investment? And then how do you look at valuing that over time? But you can see here in this case, the delta between the cost and the red line, which is the return, you know, 7x over sixty months, when you look at that in dollar terms, I mean, that's a significant figure. And it's also not taking into account organisational agility, which is another benefit that you get from your investment in skills. We look at the cost breakdown. If you look at the cost, most of the cost actually as a percentage was really in the consulting and design, and to agree the change management. The platform and tools part, funnily enough, was actually one of the lowest costs. But that change management piece is an important investment to get the results. What we're measuring, and these were the sort of key four measures, retention, mobility or you know, the velocity of mobility, reduction in hiring costs, but, you know, most importantly productivity gains. Interestingly enough, the retention savings were the biggest savings of the lot because we're able to keep the team together for longer, develop their skills and also then part of that was being sort of applied into productivity gains. But significantly this was actually resulting in motivation increases and the motivation increases not reflected here, were actually resulting in a higher velocity of skills acquisition. So being able to join those things together gives your organisation much more agility, which is really effectively what you're selling. And then we looked at this. This is just over a two month period and looking at breaking it down by various different roles and looking at that in terms of the benefits versus the dollar benefits versus the return benefits. You can see that, you know, obviously, it's not huge in this view. But when you break that out over like a sixty month time period, and you're looking at analyzing that by role, that gives you two things. One is it can tell you, particularly for critical roles, that this role is really moving ahead faster. And that means that from that role, we're expecting greater agility and greater increase on returns. On other roles, if they're not performing at the same level, you can then give you the information to drill down within that to further improve the skills investment for those specific roles. So you get better bang for buck effectively. And then tracking that over time and looking at you know key metrics like skills gap closures or you know competency acceleration indexes overlaid with productivity gains. Those things are really critical, but looking at that over time, tracking those metric trends is really important because it also gives you ammunition as, okay, is this sustainable? Are we sustaining the pace that we thought we would? And then if we're not, it gives me a clear line into which areas are underperforming so I can go further invest. And again, because this is only pillar three of a five pillar framework, when you start overlaying that into, you know, business impact, for example, that's when you start really seeing some true benefits of being able to align these models together. And then, you know, dashboards being able to help you with determining do we have agility within our business? Have we got enough coverage? Do we have coverage by key capability areas and what does that mean? Down the bottom, do we have a lot of people in the major development opportunity? It means that they've got a big gap between current skills versus required skills. Being able to hone in on that allows you to better invest dollars on closing those skill gaps versus other areas. And that's an important bar of creating trust from your senior stakeholders in terms of the impact you're having. But also from even a CFO level, you know, are we managing this business well, investing at the right people at the right time and do we have evidence to back it up? This is part of providing that evidence. And then if we think about looking at it from a coverage by skill area, then you can start to look at that particularly when you start breaking that up across your organisation, you can start to look at do we have the right skills in the right organisation or part of the organisation to be able to continue to operate or to take advantage of a growth opportunity or to mitigate risk. And being able to see that and quickly make decisions based on that information, that's the output of the model and the data working effectively with a scaled framework that is supporting all of that. And then finally, if we look at you know reducing operational risk, being able to provide sort of heat map based reporting which doesn't just tell by shift department and crew and tools and procedures that we're using, do we have enough people certified or current? Do I have a gap that's looming? Can I see that there's reassessments, for example, that are coming due that I need to get on top of because if I don't then all of a sudden I'm not going to have enough people to meet our productivity goals? So having that information at your fingertips and being able to use that in an operational sense is also creating increasing the velocity of you being able to make good business decisions, which results in you know increased performance at business impact level. Alright, that's a lot of stuff to throw you away very quickly. We do have a QR code you can scan, which then will take you to a document I put together around building the future of work through skills, and it is very much focused on that ROI driven case and it goes into a little bit more detail on the how including some checklists. This does form part of what we call our learning effectiveness index series That has a much, much more detail, it's about two forty pages of analysis, etc. It can make your eyes sort of glaze over if you're not into that. I geek out on it, so I do. But it is actually the bible of how to create an effective learning business, sustain it, and then grow it.