3 Priority Questions for your LMS Feasibility Study
When clients talk to me about their Learning Management System (LMS) project, above all, they typically ask three priority questions as part of their LMS Feasibility Study:
Is what we want available?
How much does it cost?
Will we make that money back – i.e. what is the ROI?
In this article I will dive into these questions and how your LMS Feasibility Study will answer them.
1. Is what we want available?
How to identify if solutions available in the market are covering your needs
Let us go back to your LMS Needs Assessment, which I covered in a previous article. Remember, that we identified all needs for your Learning Management System under a number of categories:
Reporting Needs
Business Goals
Learner Needs
Content Needs
Communication Needs
Technical Requirements
Specialty Requirements
Prioritization Needs
I recommend that you list your Needs Assessment in a spreadsheet which will form the basis for your Feasibility Study.
Asking for information
This spreadsheet also serves an important role in asking for information from suppliers. As a result of a written process, you will receive information that is based on the same input, making the responses more easily comparable. I have seen clients in the past who conduct interviews with suppliers only. This can be very confusing and carries the risk that you make the wrong decision.
A structured process for conducting your research is absolutely necessary. Typically, this is done through a “Request-for-Information” (RFI) process or through a “Request-for-Proposal” (RFP).
Depending on the type of organization (e.g. crown corporation or government versus private enterprise), you might post your RFI to a Bid Platform or send it to system providers, which you pre-selected, inviting them for their response.
Analysis of the information
The next step is to analyze the information you receive against the Needs Assessment and this is where your spreadsheet will be particularly helpful.
Simplified Analysis Spreadsheet for your Feasibility Study
Providing the suppliers with the spreadsheet and instructions of how to present the information increases the likelihood that the information received will become easier to compare.
The above image is a simplified example spreadsheet for your analysis. In the "real world" you want the functional requirements to be very specific. You can read up on these requirements in my article "8 Top Priorities For Your LMS Needs Assessment".
Gather the responses in one master document to make it easily comparable.
At this stage you will likely come across some gaps and open questions. Now it is time to find the answer to those gaps so that you have a full understanding of how each of the potential solutions will match your needs.
I have actually found that sometimes you get some really good ideas for additional requirements when you see the responses from a supplier. These new requirements will need to be checked out with all the other supplier options too. A note of warning - keep this in check and beware of scope creep!
The role of demos, walks-throughs and product tests
The responses to your RFI will likely be detailed and revealing. However, nothing will replace seeing the system in action. Many LMS providers will offer a product demo. This can be helpful as part of your Feasibility Study. You will be able to see the system in use and usually you’ll be able to ask questions. Suppliers might offer a demo as a recorded overview or a private walk-through session. The more you can get this to be customized to your needs, the more applicable the information will be.
Ensure you have some clear evaluation criteria you want to test each system for and prepare a way to document this effectively while you are going through the demo.
To get even more familiar with the system and the limitation and how it meets your needs, you will likely want to test the system. Again, most LMS providers will allow for a free test period.
However, in my experience the test periods tend to be too limited in time. Usually you will want to have multiple people test the system in various roles. This includes your trainers or curriculum designers or course authors as well as your other users. This will make timing difficult within your Feasibility Study. Ask about the flexibility of the LMS supplier before you start your test and plan the timing with attention to availability of your test participants, completeness of your test cases and access to content and resources to accomplish that test properly.
Compromise
It is likely that not all your needs will be met by each of the system options that you are reviewing.
This is where the prioritization you documented in your Needs Assessment will become really important. Identify the gaps in each of the systems and assess them against your requirements. Did you identify if each component you are looking for is in any of these categories:
Must have
Highly desirable
Desirable
Now it is time to assess the gaps against these categories and eliminate systems that don't fulfill all “must have” features.
If none of the systems fulfill all “must have” components, you have two choices:
Reach out to more suppliers and specifically look for options that are able to fulfill those needs.
Review if those needs are presently not available in the market. If so, you may need to compromise on this need or look for customization options.
Assessing your options using a point system
A point system could be helpful in the analysis of the system options. By assigning points as to how accurately they meet your needs, you can sequence the options by assessed match value.
For example, you could use a 10-point scale and on each component provide points from 0 (does not meet this criterion at all) to 10 (meets this criterion fully). The total sum of all points across all criteria divided by the number of criteria measured multiplied by 10 will give you the percentage of match against your needs as long as all needs are weighted the same.
If you are weighting your criteria, you will need to take that into account for your evaluation.
At this stage of your Feasibility Study you should have a good idea if what you wanted and identified in your Needs Assessment is available in the market or what compromise you may need to make. You also know at this stage to what extend each solution fits your needs and which ones you need to eliminate because it does not fulfill your mandatory requirements.
2. How much does it cost?
What needs to be considered?
This question is so simple and yet so hard to answer.
There are many factors that impact the cost of your Learning Management System:
System licensing costs
Maintenance costs
Implementation costs by the vendor or related third party
System integration costs by third party vendors with other systems
Licensing costs for additional systems acquired related to the project
Costs for consultants hired to support the project
Internal system integration costs such as
IT department time and resources
Training department time
Stakeholders and Subject Matter Experts time
Marketing time and assets for branding purposes
Cost due to impact on training programs
Re-work of existing training programs
Cost of development of new programs not previously offered
Price models
In many instances, while conducting a system feasibility study, I found that the system providers offer different pricing models that can make the price comparison more complex. Overall, costs may be quoted across a number of factors:
Licensing (either a system access fee or a license fee by number of learners)
Consulting (additional cost for consulting and implementation – may be included in licensing fee)
Customization (for any changes required to upgrade the default settings – may be included in licensing fee)
Maintenance (annual fee that includes product updates and maintenance support)
Service and tech support (for learner or administrator support)
Not all system providers will charge the same types of fees and as a result it can be complex to calculate overall costs that are comparable. Consider calculating across multiple scenarios, for example based on last year’s number of learners and your anticipated number of learners for the next year or two.
eLearning Industry, an online publication specializing in the Learning and Development sector, has published an eBook article about the different LMS Pricing Models that differentiates between:
Cloud-Based LMS Pricing Models (metered per learner, use or active learner)
Self-Hosted Pricing Models (all-inclusive)
Free/Freemium (open source)
They also list a number of potential hidden fees to look out for:
Initial set-up fees
Undisclosed licensing fees
Design customization fees
Upgrade fees
Excessive use fees
Ongoing maintenance
According to the author of the Ebook article, the key is to determine a realistic budget for your LMS.
Where do you draw the line?
You can already see that the assessment of the cost of your LMS can be a big part of your Feasibility Study and there is no clear definition of what should be included versus what may be considered as a cost under a different line item.
Hence, before you conduct the cost analysis, it is important to define what should be included and what not.
Option 1:
You might choose to ONLY consider system costs when comparing the different options. This would include the purchase of the license of using the LMS and the ongoing maintenance costs.
Option 2:
In addition to the system costs for Option 1, include the cost for implementation of the LMS. This will give you a good baseline: the investment it takes to have a new course delivery platform.
Option 3:
As a further more comprehensive option, you may also add the cost of moving your existing programs to the new platform. Now you have the complete cost for replacing your existing course delivery platform. In this option, you may need to include transition costs from your old system as well.
Which option to choose?
Clearly, with each step to the next Option the cost analysis becomes more complex and, as a result, less accurate and also more costly and time-consuming to produce.
Option 3, for example, deals with a lot of uncertainties. Probably, by the time you transition your programs to the new platform you may eliminate some or choose to produce new content. You might decide to run two platforms while you transition out of some of your existing programs.
You will need to make a decision where the benefit of more complete information outweighs the efforts of calculating a trustworthy number.
I believe that Option 2 provides you with a good basis to assess the costs of various options and, consequently, a realistic figure of what the effort is going to be to set up a system for your content move.
Furthermore, a rough analysis of the efforts of moving your existing programs will still be helpful to guide you in the decision to purchase the new system.
3. Will we make that money back – i.e. what is the ROI?
If you were thinking this is getting rather complicated, the ROI calculation will take this to another level, hence, you won’t want to do this across all the possible LMS options. Choose your top selection LMS to conduct an ROI analysis.
In the ROI, you need to compare the costs of the new system with the system you presently have in place. Furthermore, you need to take into account the benefit that each of these systems provide.
Benefits can be:
The ROI calculation compares the benefits with the costs associated to obtain those benefits. Divide the benefit in monetary terms with the cost and multiply by 100 to get the percentage.
For example, if your cost for the LMS is $50,000 and you have a benefit of $100,000 then your project will yield 200% ROI. Reality is that often your initial costs are high, and you might have a much lower ROI. Finally, you might be interested to calculate the time it will take for you to re-coop your investment.
eLearning Industry published an article identifying six steps for calculating the Learning Management System ROI:
Calculate time-savings versus traditional training
Review previous L&D expense sheets
Factor-in employee retention rates
Consider compatibility and data integration costs
Determine the LMS learning curve
Assess the costs of running an LMS
One aspect that can impact your ROI is also if you are implementing a new LMS or if you are switching from one system to another.
A thorough ROI analysis takes time and care to complete. At the beginning of your project be clear on what you need to measure. As a result you will be able to support your decision-making process in your Feasibility Study.
To sum it up...
A Feasibility Study for your LMS project is an important decision-making tool. The quality of this process depends on a well formulated Needs Assessment for your Learning Management System and clear direction on the cost calculation and ROI calculation needs. They are needed to support your organization’s decision regarding which of the LMS systems you identified are suitable and affordable for you.
This process is time consuming and requires focus and attention to detail. Like the Needs Assessment, this is not something that can be completed as a side-of-the-desk task, which is why, working with an independent consultant can be a life-saver. And independent consultant can provide the focus required to get the correct data for your internal decision making and budgeting processes.
Contact me for a no-obligation consultation to learn more how I can help you by conducting a Needs Assessment and Feasibility Study for your Learning Management Project.