Data Explained
This Section provides further technical detail including definitions, methodology, data sources and dates that data are updated relating to the Performance Indicators used in FE Choices
The latest Performance Indicator data on the FE Choices comparison site were updated on the following dates:
Learner Satisfaction – 01 March 2013
Employer Satisfaction – 14 December 2012
Learner Destinations – 24 October 2012
Success Rates – 26 July 2012
FE Choices is concerned with four key aspects of an organisation’s performance: (a) Success Rates, (b) Learner Destinations (including Learning Rate and Employment Rate), (c) Learner Satisfaction and (d) Employer Satisfaction.
Each aspect of performance has an associated performance indicator (PI).
Each PI uses an agreed data collection method and an agreed set of rules to calculate a percentage score or a score out of 10.
The following paragraphs contain information that is relevant to all PIs and explains some of the statistical concepts used. Further details of each PI are described in later sections.
We publish a score for each PI for all organisations where we are able to collect enough data to give a reliable indication of performance.
All PI’s except Success Rates, in most cases, are based on a sample of people rather than everybody who used an organisation. For these PIs we do not publish scores for organisations where:
- The number of people who responded to the survey is low, or
- The number of responses from people with certain characteristics is very different from the number of people with those characteristics at the organisation.
We determine whether these factors make a score unreliable using two statistical principles.
- Confidence Intervals – If we conducted the survey again an organisation could get a very different score. We would not publish a score if we believe that re-running the same survey would give a score that is more than or lower than the current score. If more people complete the survey there is less chance of this happening.
- Representation – This is best explained with an example. Imagine an organisation has 70 people doing A-levels and 30 people doing basic skills courses. Their survey is completed by 65 people who do A-levels but only 5 people who do basic skills. Even if the organisation passes the confidence interval test we would not publish the score because people who want to do basic skills at the organisation could not be certain that the findings of the survey are relevant to the course they want to do.
When looking at the score for an organisation of interest it is useful to see how it compares with other similar organisations. To help with this comparison we publish some key summary figures for all performance indicators.
For Success rates we use:
- 5th Percentile - if an organisation has the same score at the 5th percentile then at most 5% of organisations have lower scores and at most 95% of organisations have higher scores.
- Median – This is a type of average and represents the middle score. If an organisation has the median score then at most 50% of organisations have lower scores and at most 50% of organisations have higher scores.
- 95th Percentile - if an organisation has the same score at the 95th percentile then at most 95% of organisations have lower scores and at most 5% have higher scores.
For Learner Destinations, Learner Satisfaction and Employer Satisfaction we use:
- Lowest Score – This is the lowest score across all organisations.
- Middle Score – This is a type of average and represents the middle score. The middle score represents the value where at most half (50%) of providers are above this value and at most half (50%) are below this value.
- Highest score - This is the highest score across all organisations.
FE Choices information will be of benefit to organisations themselves, their learners, employers and other users. Existing data collections which underpin FE Choices form part of the funding agreement with organisations. The agreement makes clear what action will be taken where problems are encountered with the supply of data.
Further information on data quality can be found at: http://www.thedataservice.org.uk/data_quality/
We continue to explore ways in which data returns for each performance indicator can be maximised.
For further, more detailed information on the four performance indicators, including information on their broad description, scope as well as methodology and data sources can be found below and by clicking on the following links:
Return to top
Broad description
The FE Choices Success Rates performance indicator uses Qualification Success Rates for individual colleges and other training organisations across a broad range of learning provided. The information also shows comparison data that will enable learners and employers to make an informed judgement about the performance of an organisation relative to its peers.
Success Rates are broadly defined as the number of learning aims achieved divided by the number of learning aims expected to complete, excluding any learners who transferred onto another learning aim within the same organisation. The figure is provided as a percentage.
We define learning aims achieved as all those learners who, as a minimum, passed the qualification they started.
A full description of the success rate calculation can be found under the (see the Methodology and data sources section below).
Scope
The Success Rates performance indicator is applicable to all organisations within FE Choices that deliver learning aims that are categorised as falling into the qualification types listed below.
There are no exemptions from this performance indicator – every organisation with relevant learning aims will have their Success Rates displayed, subject to consideration of statistical robustness and/or data protection (see the Disclosure and statistical robustness section below).
Qualification types
The Success Rates performance indicator is comprised of Qualification Success Rates for eight qualification types. These are:
- FE Long qualifications, excluding A-levels, AS and A2 courses – These qualifications last for 24 weeks or more and include qualifications such as GCSEs, NVQs, BTECs, certain Certificates in Adult Numeracy/Literacy, International Baccalaureate Diplomas and other Certificates and Diplomas (for example ‘Certificate in Basic Plumbing Studies’ and ‘Diploma in Childcare and Education’).
- FE Long qualifications - A-levels, AS and A2 courses only – These qualifications last for 24 weeks or more.
- FE Short qualifications – These qualifications last for between 5 and 24 weeks and include qualifications such as some ESOL courses, certain Certificates in Adult Numeracy /Literacy, Skills for Life and other learning aims (for example ‘Award in Community Sports Leadership’ and ‘Diploma in Book-keeping’).
- FE Very Short qualifications – These qualifications last less than 5 weeks and include qualifications such as ‘Award in Health and Safety at Work’, ‘Certificate in Outdoor First Aid’ and ‘BTEC Award in Door Supervision’.
- Intermediate Level Apprenticeships – These combine on and off the job training and are nationally recognised Full Level 2 qualifications. Apprentices work towards work-based learning qualifications such as a Level 2 Competence Qualification, Functional Skills and, in most cases, a relevant knowledge-based qualification.
- Advanced Level Apprenticeships - These combine on and off the job training and are nationally recognised Full Level 3 qualifications. Apprentices work towards work-based learning such as a Level 3 Competence Qualification, Functional Skills and, in most cases, a relevant knowledge-based qualification
- Higher Apprenticeships - These combine on and off the job training and are nationally recognised Full Level 4 qualifications. Apprentices work towards work-based learning qualifications such as a Level 4 Competence Qualification, Functional Skills and, in some cases, a knowledge-based qualification such as a Foundation Degree
- Workplace Learning (Full Level 2 and 3 previously funded as part of Train to Gain). This qualification type does not include the entire programme that was Train to Gain. It shows Full Level 2 and Full Level 3 qualifications only. The Train to Gain programme no longer exists and so this now includes training mainly delivered through the workplace (excluding Apprenticeships).
In addition to displaying the Success Rate for each of these qualification types, each type is further broken down by Sector Subject Area (SSA) and Age Group.
Sector Subject Area and Age Group
There are 15 categories of Sector Subject Area (SSA) in tier 1 that group qualifications by broad subject area. For example ‘Arts, Media and Publishing’ or ‘Business, Administration and Law’.
Qualifications on FE Choices which do not correspond to one of the 15 SSA tier 1 categories are grouped into the ‘other’ category. They are included in the total.
FE and Apprenticeship Success Rates are broken down into age groups- either 16-18 year olds or Adults (learners aged 19 and over). Learners with an unknown age are classified into the adult age grouping. Age is calculated as at 31 August of the academic year the learner started their qualification or framework.
Workplace Learning Success Rates are not broken down by age as the vast majority of these learners are aged 19 and over.
Methodology and data sources
All Success Rates published in FE Choices are calculated using data returned in the Individual Learner Record (ILR). They adhere to the methodologies used by the Data Service( http://www.thedataservice.org.uk/statistics/success_rates/) throughout
Success Rates for all qualification types published as part of FE Choices are calculated according to the Overall Success Rates methodology.
For qualification types (1) to (4) this is defined as the number of achieved learning aims with a planned end date in the Qualification Success Rate (QSR) reporting year expressed as a percentage of the number of starts with a planned end date in the QSR reporting year.
As Success Rates are calculated at a qualification aim level, learners with more than one qualification aim will be included more than once. Learners who leave their programmes within six weeks of starting are excluded from the calculation.
Success Rates for qualification types (5) to (8) are calculated as the proportion of learning aims that are completed at any time, irrespective of when the aim was due to be completed. This measure is based on the hybrid end year of a learning aim, which is the latter of the planned or actual end date.
Further information on the Overall Success Rate methodology can be found at
http://www.thedataservice.org.uk/statistics/success_rates/success_rates_2011-12/
Disclosure and statistical robustness
If the number of starts/leavers is less than 5 then the corresponding success rate is suppressed. In these cases, the success rate will be displayed as a hyphen (‘-‘).
Success rates are not suppressed based on the number of achievements - 0% and 100% are included as long as the rule on starts/leavers is met.
Suppression rules for success rates are summarised in Figure 1 below.
| Starts |
Achievers |
Starts |
Achievers |
SR |
|---|
| 4 |
0 |
X |
X |
X |
| 4 |
4 |
X |
X |
X |
| 5 |
0 |
X |
X |
o |
| 5 |
5 |
X |
X |
o |
| 30 |
0 |
o |
X |
o |
| 30 |
5 |
o |
o |
o |
| 30 |
30 |
o |
o |
o |
Above values are rounded to nearest 10 in reports
o = Display
x = Suppress
Figure 1 - Success rate suppression rules
Comparative Data
When looking at the score for an organisation of interest it is useful to see how it compares with similar organisations to itself and against all other organisations in-scope for FE Choices. To help with the Success Rates comparison we publish comparative data as follows:
- 5th Percentile - if an organisation has the same score as the 5th percentile then at most 5% of organisations have lower scores and at most 95% of organisations have higher scores.
- Median – This is a type of average and represents the middle score. If an organisation has the median score then at most 50% of organisations have lower scores and at most 50% of organisations have higher scores.
- 95th Percentile - if an organisation has the same score as the 95th percentile then at most 95% of organisations have lower scores and at most 5% have higher scores.
A percentile is the numerical value of a variable (in this case Success Rate) at which a certain percent of values are at or below. For example if the 5th percentile is 30% this means that 5% of organisations had a Success Rate of less than or equal to 30% .The median is the numerical value of a variable (in this case Success Rate) at which half the values are at or below and half of the values are at or above. For example if the median of 65%, this means that half of organisations had a success rate of less than or equal to 65% and half had a success rate of 65% or over. For example if the 95th percentile is 70% this means that 95% of organisations had a Success Rate of less than or equal to 70%.
Comparative data (5th percentile, median and 95th percentile) are not shown where the numbers of organisations is too low to provide a statistically robust value. The following rules are applied:
- Percentiles will be shown for 5th, median, and 95th percentiles if there are 20 or more providers.
- Where there are between 11 and 19 providers the median only is shown.
- Where there are 10 or fewer providers no percentiles will be shown.
Return to top
Broad description
This performance indicator (PI) provides a statistically reliable provider-level measure of learner destinations. The PI is based on the proportion of learners on priority programmes who completed a Skills Funding Agency*/ YPLA** funded programme and progressed in the next academic year to a defined destination.
Learner Destinations are established through a combination of data matching and telephone surveys.
* Formerly the Learning and Skills Council
** Now the Education Funding Agency (EFA)
Timetable
The table below shows the Learner Destinations survey year and the corresponding learning and destination years.
| Survey Year |
Learning Year
(Study Completion Year) |
Destination Year
(approx. 1 year after Study Completion Year) |
Publication Date |
|---|
| 2012 |
2009/10 |
2010/11 |
Survey ended May 2012. Results to be published in 2013 on FE Choices comparison site. |
| 2011 |
2008/09 |
2009/10 |
24 October 2012. |
| 2009 |
2007/08 |
2008/09 |
15 December 2010 |
Scope
Organisations that delivered provision funded by the learner responsive or employer responsive funding stream are in scope except:
- Schools
- Central government departments and/or organisations reporting to them
- Non-departmental public bodies
- Organisations co-funded to deliver European Social Fund programmes only
- Organisations funded to deliver Adult and Community Learning only
- Employers and local authorities funded only to train their own employees
Organisations are included if they have eligible learners in the learning year and are still in receipt of funding in the destination year. Learners are eligible if they completed at least one priority learning aim in the learning year and were not continuing any aim into the destination year. For example, learners on the first year of a two year programme are excluded (including AS progression to A2 programmes). There are exceptions for learners with specific combinations of learning difficulties and/or disabilities.
Priority learning is defined:
- Aged 16-18 inclusive at the start of the learning year or were on one of the following programmes of study
- Apprenticeships/Advanced Apprenticeships
- Workplace Learning programmes (previously Train to Gain)
- Skills for Life programmes which were target bearing
- Programmes contributing towards a full Level 2 or Level 3, as flagged on the Learning Aim Reference Application (LARA) formerly Learning Aims Database (LAD)
Defined Destinations
A defined destination in the next academic year (destination year) is:
- Enrolled in priority learning with the same highest level of learning,
- Progressed to learning with a higher level of highest learning aim,
- Remained in employment or self-employment with improved job security or enhanced careers prospects,
- Entered employment/self-employment or training having previously been in learning or outside the labour market
- Not in employment, education or training but the destination activity is considered neutral for the purposes of this measure
- Learner not tracked into further learning and ILR field L27 (which indicates restrictions on the use of the learner record) prevented further contact
- Learner not tracked into further learning but no contact made for other reasons
- Learner interviewed but did not meet any of the criteria to record a positive outcome or neutral destination.
Methodology and data sources
The Learner Destinations indicator is calculated through a combination of information established through the Learner Destinations survey and data matching.
Detailed information on the descriptions, analysis and explanations of all phases of the Learner Destination Performance Indicator score are available in the 2010/11 Learner Destinations Study – Technical Report.
The Learner Destinations indicator is calculated through a combination of information established through the Learner Destinations survey and data matching.
Information on learners who completed priority learning is obtained for the FE learners in FE institutions from the Learner Responsive FO5 and period 13 Employer Responsive ILR Learner and aims data returns for the learning year.
The destination is firstly determined by data matching. The data set for the FE learners in FE institutions is derived from the Learner Responsive FO5 and period 13 Employer Responsive ILR data returns plus the National Pupil Database and HESA student records for the destination year. Providers, including constituent parts of subsequently merged providers, are identified directly from Skills Funding Agency data.
For the FE learners in HE institutions (HEIs) a range of data sets are used for the first stage of identifying in scope learners. For the HEIs which return ILRs the Learner Responsive ILR F05 and the Employer Responsive ILR P13 Learner and Aims files for the learning year are used to identify in-scope learners. The HESA return records for the learning year are also used to identify the base cohort. Destination data sets used are the ILR Learner Responsive F05 and Employer Responsive P13, the National Pupil Database and the HESA student records for the destination year.
Use is also made of the Analytical Learning Aims Database. This provides detailed information about learning aims and is used to identify priority aims and levels of learning.
Algorithms to determine completers and eligible learning are run to enable the flagging of individual learners. They are then matched against the destination year datasets using a process known as ‘fuzzy matching’.
A learner is deemed to have achieved a positive destination if they have:
- Continued in any learning, if aged 16-18 at the start of the destination year
- Progressed to any learning in destination year with a higher level of highest learning aim
- Enrolled in priority learning in destination year with the same highest level of learning
Some learners who remained in learning might not have been identified through the data matching process, for example if they did not have a Unique Learner Number (ULN) and had changed name or address between the completion and destination years so fuzzy matching was not possible. Their judgement was used to help establish if they had progressed or not during the Learner Destinations Survey.
The destination of the learner is determined using all of the above. Once this has been done it is possible to calculate a Learner Destinations score – i.e. the proportion of an organisation’s learners who went into a positive destination.
An additional component to the Learner Destinations score is the calculation of an overall provider Learning Rate and Employment Rate where there was sufficient data to allow the calculation of these two additional measures. Learning Rate and Employment Rate are fully described in later sections of this document.
Comparative Data
When looking at the score for an organisation of interest it is useful to see how it compares with similar organisations to itself and against all other organisations in-scope for FE Choices. To help with the Learner Destinations comparison we publish comparative data as follows:
- Lowest Score – This is the lowest score across all organisations.
- Middle Score – This is a type of average and represents the middle score. The middle score represents the value where at most half (50%) of providers are above this value and at most half (50%) are below this value.
- Highest score - This is the highest score across all organisations.
Disclosure and statistical robustness
Four quality thresholds are used before a Learner Destination score is published:
- At least 30 learners’ destinations must be established;
- At least 10% of the provider’s unmatched learners must have been interviewed.
- Standard measures of the robustness of the sample must be such that we can be 95% confident that the true value of the score lies within +/- 10 percentage points of the recorded value.
- The sample profile is within a predetermined level of bias, that is, achieve a “skew score” of 40% or less. The “skew score” compares the distribution of the interviews achieved, compared to the distribution of the total learners at the organisation, using eight different learner categories based on combinations of age, gender and level of learning. The distribution of learners at the organisation is applied to the number of interviews completed, to give the expected number of responses in each of the eight categories. The difference between the expected number and actual number is calculated for each category. The positive values, that is those which indicate that a particular group has been under-sampled are added up across the eight categories. This figure (the sum of all the positive values) is divided by the sample size to obtain the final “skew score”.
Return to top
Broad description
A positive Learning Rate is the proportion of eligible learners (minus the projected number of neutrals***) completing a Skills Funding Agency/YPLA-funded programme in the learning year that progressed to one of the following outcomes in the destination year:
- remained in learning with the same highest level of learning aim, where at least one aim was a priority learning aim.
- remained in publicly-funded education or training with a highest learning aim below Level 4 that was higher than the highest learning aim in the completion year or
- remained in publicly-funded learning with a highest learning aim of Level 4 or above.
Statistical corrections are used to ensure that the estimated figure is free from sampling bias.
Also shown are the two following figures which are calculated using the entire eligible population of the organisation minus the projected number of neutrals:
- Further learning – Learners who went on to study at the same or higher level up to and including Level 3
- Higher education – Learners who went on to study at Level 4 and above
*** Neutrals are defined as all those learners who indicated that they could not be counted as having a positive destination, but neither should they count against a provider’s score.
A learner was considered neutral in the destination year if they were, for example:
- In a caring role and had no acceptable care services available, or
- Were In ill health.
Return to top
Broad description
A positive Employment Rate is the proportion of eligible learners (minus the projected number of neutrals***) completing a Skills Funding Agency/YPLA funded programme in the learning year that are projected to have progressed to one of the following outcomes in the destination year:
- remained in employment or self-employment, with improved job security, income, skills or enhanced career prospects;
- entered employment or self-employment, having been in learning prior to the learning year, where the learning carried out in the learning year had a positive impact; or
- entered employment, self-employment or training, having previously been outside the labour market prior to the learning year.
Also shown are the two following figures which are calculated using the entire eligible population of the organisation (minus the projected number of neutrals):
- Found a job – learners who found a job which lasted more than six months
- Reported career benefits – learners who reported receiving a higher salary, more job security, having better skills etc.
*** Neutrals are defined as all those learners who indicated that they could not be counted as having a positive destination, but neither should they count against a provider’s score.
A learner was considered neutral in the destination year if they were, for example:
- In a caring role and had no acceptable care services available, or
- Were In ill health.
Return to top
Broad description
This is a performance measure derived from a questionnaire which asks learners about:
- the information, advice and guidance received from the learning organisation;
- the quality of teaching and learning on the programme;
- their overall satisfaction with their learning experience;
- their satisfaction with the level of support available to them from their learning provider;
- whether they are treated with respect; and
- whether their learning organisation responds to their views.
Scope
The following learning organisations are in-scope for the Learner Satisfaction survey:
- General FE and Tertiary Colleges
- Specialist Colleges (including Art and Design, Land based i.e. Agriculture, Horticulture, Animal Care)
- Specialist Designated Institutions
- Higher Education Institutions (HEI) for (FE learners in HE institutions)
- Private sector publicly funded organisations
- Other Publicly Funded Institutions &
- Employers training their own staff
This survey applies to the following learners on programmes with in-scope organisations:
- 19+ programmes funded by the Adult Skills Budget (excluding Community Learning & Discretionary Support)
- Classroom based learning
- Apprenticeships
- Work-place learning
- 16-18 learners (except those in sixth-form colleges)
The survey applies to learners, as defined above, including those on programmes delivered by sub-contractors, where the lead organisation is in-scope. Learners with learning difficulties and/or disabilities are in-scope for the survey (at the discretion of learning providers). All eligible learners enrolled at an in-organisation during the survey period are entitled to submit a response.
The only exceptions are:
- learners under 16; and
- learners on Offenders’ Learning and Skills Service (OLASS) provision
Whilst all in-scope learners are entitled to take part in the Learner Satisfaction survey, learning organisations are free to decide whether to attempt a census of all such learners or to attempt to achieve a representative sample.
In order to achieve a representative sample, the Skills Funding Agency made available a Minimum Sample Size Calculator to all in-scope learning organisations so that an estimate could be made of the minimum number of responses needed to achieve a valid Learner Satisfaction score. All learning organisations are encouraged to achieve more than the minimum sample size to correct for any skew or invalid responses received during the survey phase.
Methodology and data sources
Full details of the methodology and data sources used to calculate the Learner Satisfaction Performance Indicator (PI) score can be found here.
However, an overview of the process is provided in this section.
The data used to derive the score for this PI is obtained from the Learner Satisfaction survey. The survey is primarily an online survey but in exceptional cases learning organisations are able to use paper questionnaires.
The base size and minimum sample size targets are calculated for each organisation. Responses to the survey are then verified and validated against the Individual Learner Record (ILR) returns made by each organisation to make sure that the respondents are representative of all learners enrolled with the organisation. Once a learner has been positively identified, their responses are matched into their ILR records and corrections made (if necessary) for sample skew and survey methodology.
For paper based responses a slight downward correction is made as there is evidence that people give different answers to questions depending on the response method used. An adjustment factor of 0.9755 is therefore applied to the paper responses to ensure that learning organisations using online methods are not disadvantaged. After applying tests for sample validity, a score is calculated.
Higher Education Institutions (HEIs)
HEI learning organisations delivering FE (please refer to Scope) are in-scope for the Learner Satisfaction survey. Survey results are validated against the FE ILR and the HE Individual Student Record (ISR) return. The ISR return is received by the Agency at a later point in the year.
For this reason, the Agency treats HEIs separately for publication purposes.
This will mean that there is an interim publication of Learner Satisfaction data which excludes HEIs. The final data, including comparative statistics, will be published once the results for HEIs have been included.
This methodology will ensure that there is maximum data validity for both HEI and all other learning organisations.
Comparative Data
When looking at the score for a learning organisation of interest it is useful to see how it compares with similar organisations and all other organisations in-scope for FE Choices.
To help with the Learner Satisfaction comparison we publish comparative data as follows:
- Lowest Score – This is the lowest score across all organisations.
- Middle Score – This is a type of average and represents the median score. The middle score represents the value where at most half (50%) of providers are above this value and at most half (50%) are below this value.
- Highest score - This is the highest score across all organisations.
Disclosure and statistical robustness
For a learning organisation’s overall score to be reported:
- they must have received at least 10 valid responses to the survey (a valid response is at least one score given in answer to any of the nine scoring questions);
and either:
- the Agency must have 95% confidence that the confidence interval was less than or equal to 5% (the confidence interval gives an indication of how accurately the score from the valid responses reflects the result the learning organisation would have achieved had all learners responded to the survey);
or
- the learning organisation must have achieved a valid response from at least 70% of learners.
In addition, learning organisations have to plan the composition of their sample based on age/gender/highest level of study combinations in order to avoid a skewed or biased sample. A skewed sample would result in less accurate estimate. These combinations are used to test whether responses coming into the survey are broadly representative. Samples for each organisation are assessed in terms of the 16 learner sub-categories (Entry Level, Level 1, Level 2 and Level 3 or higher for males and females in the 16-18 and 19+ categories) and the balance of responses is compared to the balance for eligible learners in the most recent ILR. Where a sub-group is under-represented in the sample, the group’s responses are weighted to give a fair representation.
Some results are not shown (suppressed) in order to ensure that individual learners cannot be identified from their responses to the Learner Satisfaction survey questions and that learner confidentiality is maintained. Suppression occurs when:
- all learners responding to a particular survey question respond with the same rating and the number of responders is equal to the number of learners in-scope for that particular learning organisation e.g. 20 identical responses from 20 in-scope learners;
- where the number of learners in-scope for a particular learning organisation is fewer than 10;
and
- where the number of responders is fewer than 3.
Course level data
The Learner Satisfaction survey also asks learners three additional questions which apply to their specific course(s). The main survey and course level questions can be found here.
Courses are identified by the learning aim reference which is supplied by the learner. If the learner doesn’t have the reference they can identify the course using text which is then validated against the ILR return.
Course level data does not contribute to the overall score for learner satisfaction. Results are published for information only.
Wherever possible we will publish data at course level. Courses that are the same but offered by different awarding bodies are grouped together. Further information regarding this can be found here.
Return to top
Broad description
The Employer Satisfaction survey captures employers’ perceptions on the quality of the training provided and the responsiveness of their training organisation to the employer’s needs. The scores calculated for each training organisation enable comparisons about their performance to be made against other training organisations of the same type and against all training organisations in scope for FE Choices who complete the survey and pass the necessary quality thresholds.
Scope
In Scope Training Organisations:
The Employer Satisfaction Survey applies to all General FE and Tertiary Colleges, Specialist Colleges, Private Sector Public Funded Institutions and Other Public Funded Institutions deliver learning to employers funded through the Adult Skills Budget or Apprenticeship funding. The exceptions are:
- Sixth-form colleges; and
- Organisations funded to deliver training exclusively to their own employees.
In Scope Employers:
An employer is in scope for the Employer Satisfaction survey if they received publicly funded training from any of the in scope training organisations.
Methodology and data sources
Detailed information on the descriptions, analysis and explanations of all phases of the Employer Satisfaction Performance Indicator (PI) score are available here.
An overview of the process is detailed below.
The Employer Satisfaction PI is a score derived from a questionnaire that captures employers’ perceptions of the quality of the learning provided and the responsiveness of the training organisation to the employers’ needs.
There are three versions of the questionnaire – one for each of the survey modes: postal, online and telephone.
The questionnaires used in each of the survey modes are broadly the same and comprise nine rating questions. The online survey however, includes additional questions on courses and subjects and the online and postal survey allow for open ended comments on the overall quality which is valuable to the training organisation. These questions are not included in the telephone survey as it would make the survey unreasonably time consuming for the employer and their inclusion would likely negatively impact the telephone survey response rate.
The response to each of the nine rating questions is a score from 0 to 10, for example, from 0 ‘poor’ to 10 ‘excellent’. For each training organisation, the scores from all the answered questions, covering all employers surveyed, are aggregated and averaged to give the overall score for the training organisation which is a mean score on a scale of 0 to 10. Each answer carries equal weighting with all other answers.
The source of the information for each training organisation’s eligible employers is the Individual Learner Record (ILR).
Comparative Data
When looking at the score for a training organisation of interest it is useful to see how it compares with similar training organisations to itself and against all other training organisations in-scope for FE Choices. To help with the Employer Satisfaction comparison we publish comparative data as follows:
- Lowest Score – This is the lowest score across all training organisations.
- Median Score – This is a type of average and represents the middle score. The middle score represents the value where at most half (50%) of training organisations are above this value and at most half (50%) are below this value.
- Highest score - This is the highest score across all training organisations.
Disclosure and statistical robustness
For a training organisation’s overall score to be reported:
- they must have received at least 10 valid responses to the survey (a valid response is at least one score given in answer to any of the nine scoring questions);
and either
- we must have 95% confidence that the confidence interval was less than or equal to 10% (the confidence interval gives an indication of how accurately the score from the valid responses reflects the result the training organisation would have achieved had all employers responded to the survey);
or
- they must have achieved a valid response from at least 70% of the employers they worked with
Return to top
