SUMMARY OF VALIDATION ISSUES

1. Acknowledge that the LSA Framework is a large-scale and long-term project. Many stakeholders will be involved, much time will be spent on building relationships with data collectors, and many data points will need to be collected. A large pool of resources will be needed.
2. Define the population to which the results must generalize. The answer to this question drives all the other decisions. It has to be made first.
3. Determine the questions about the pathways that are to be answered such as
   
   • What are the Essential Skills needed by learners to effectively operate in the pathway?
   • Are the pathways identified in the pilot work viable?
   • What are the Essential Skills needed by learners to effectively complete the pathway
4. Determine the improvement in Essential Skills by students as they engage in formal course training. This means that
   
   • the course in which the students are enrolled must be profiled for the level of complexity they use in their curriculum
   • the degree to which the course engages them in Essential Skills development directly should be collected
   • variables such as the length of the course, number of scheduled student contact hours outlined in the course, number of hours of student contact actually occurring for each student, number of hours outside the course the student spends on course-related activities all have to be collected so they can be controlled for in any pre-post gain design
   • the degree of match between the level of Essential Skills of the student and the level of Essential Skills used in course delivery should be examined
   • the degree of match between the goals of the student and goals of the course should be collected
   • the degree of match between the goals of the student and goals of the course should be collected
5. Determine the relationship between the Essential Skills of individuals and individual differences such as gender, age, number of years of formal schooling, ethnic background and native language. This means these data points must be collected.
6. Determine outcomes or markers that will be used to validate the predictive utility of the Essential Skills. These may be course grades, student retention, goal destinations, instructor ratings of student progress, student satisfaction with the program, etc. Many possible markers exist and the ones used should be standard across the participating institutions.
7. Conduct sampling instead of trying to capture all the data across all institutions and students in Ontario. Use a stratified sampling process to ensure enough “courses” with content representing the transition paths and the students within those courses are assessed and tracked. About 50 courses in each pathway should suffice.