In order to ensure that Aimhigher Nottinghamshire activities are effective and are reaching those students most in need, it is vital that we adopt a robust monitoring and evaluation policy. Below is a summary of Aimhigher Nottinghamshire’s monitoring and evaluation strategy for the funding phase 2008-11.

Aimhigher Nottinghamshire monitoring and evaluation

Key aspects of our work that we will be evaluating during this phase include:

• the effectiveness of our learner targeting process
• the impact of Aimhigher activities on attainment, aspirations, awareness and decision-making capability
• the extent to which Aimhigher is embedded in the culture and practice of schools, colleges and training providers
• the extent to which the Higher Education Progression Framework has contributed to a change in learner identities.

Cohort data

We collect Aimhigher cohort data through parental/carer consent forms and from activity registers. We aim to collect 100% of consent forms from the Aimhigher cohort. Registers will be collected from 100% of activities and cross-referenced against the cohort. This will allow us to follow the cohort’s learner journey in order to identify the impact of a range of activities on the individuals and in particular on their attainment and progression.

Data protection considerations

To ensure data protection requirements are met, cohort data will only be input into the monitoring and evaluation database if the appropriate parental consent has been granted. In the case of older learners where parental/carer consent is not required, consent will be obtained from the cohorts themselves.

Monitoring the effectiveness of targeting procedures

The collection of participant data allows us to monitor the effectiveness of our targeting procedures through a comparison with the cohort data. This will highlight the degree to which each activity has focused on disadvantaged groups of learners.

Cohort Outcomes

Attainment

Working closely with Nottingham and Nottinghamshire local authorities, we will be able to match cohort data from the Aimhigher monitoring and evaluation database with that from the schools and colleges attainment data. We will compare Aimhigher cohorts’ projected attainment with their actual performance in order to make professional judgements with regards to the likely influence of Aimhigher interventions. We are particularly interested in outcomes of Key Stage 4 learners who have engaged with Aimhigher activity (and been recorded on the monitoring and evaluation database) in the two to three years prior to their final examinations.

Post-16 progression

Procedures are also being put in place to identify post-compulsory education progression routes of Aimhigher cohorts. We will liaise with the two local authorities and/or Connexions, again to match data from the Aimhigher monitoring and evaluation database to external records, enabling comparisons and impact analysis.

Progression to higher education

Aimhigher is planning to provide name, address and date-of-birth cohort data from the Aimhigher monitoring and evaluation database to the Universities and Colleges Admissions Service (UCAS) who will return a number of fields including flags indicating if cohorts have applied and been accepted to higher education. We will then undertake exploratory analysis to help establish any potential association between participation in Aimhigher activities and progression to higher education.

In addition to higher education progression analysis of a sample of identified individual cohorts, Aimhigher Nottinghamshire will continue to undertake analysis of UCAS applicant trends. Increases in the number and proportion of applicants from Aimhigher target groups can provide rich evidence of any progress made, in terms of increasing higher education opportunities for disadvantaged (Aimhigher target) groups.

Aimhigher evaluation methods

In order to explain the links between cohort data and learner outcomes we will employ various evaluation methodologies. We recognise that it is difficult to isolate the influence of the Aimhigher effect on learner outcomes. However, qualified researchers will carry out our evaluation work and make evidence-based judgements with regards to the likelihood that Aimhigher interventions, and the programme as a whole, are having the desired impact.

Questionnaires

Questionnaires provide useful contextual value to our overall evidence strategy. We evaluate around 20% of activities in this manner. Questionnaires allow us to cost-effectively involve a large sample of learners and are useful for quality control purposes, particularly for new and innovative activities.

Core (evaluation) partner schools and colleges

To help us understand the full effects of interventions, some in-depth evaluation is required. For this reason, we have identified a group of ‘core’ schools and colleges. These schools and colleges engage in a whole range of activities, enabling us to capture the breadth of Aimhigher, as well as the context of the Higher Education Progression Framework. We undertake research with our core partner schools to ascertain the professional views of teaching staff with regards to the Aimhigher impact on individual cohorts, and any change in the learning culture and ethos of the schools.

Learner follow up

As part of the process of collecting data, cohorts (or their parents/carers, as appropriate) will be asked for their consent to be contacted for further follow-up. Without this consent, cohorts will not be followed up. The follow up will take the form of focus groups, interviews, and/or learner diaries. We will focus on students within core partner schools.

It may be appropriate to follow up learners at the start, mid-way through, and towards the end of the funding phase. This will enable us to gauge the impact on the ways cohorts subsequently engaged with learning, and the role played by Aimhigher in terms of developing their learner identities.

Research will also be conducted with a range of stakeholders, which may include:

• cohorts
• teachers/Aimhigher co-ordinators, particularly senior management (see above)
• Aimhigher practitioners / project co-ordinators
• student mentors
• parents/carers
• higher education institution staff
• further education college staff
• School Improvement Partners (SIPs)
• local authority staff.

Evaluation of specific interventions

We may, on occasion, evaluate specific interventions, particularly pilot projects, to help us establish their perceived quality and success, identify best practice, and inform the planning of subsequent projects.

Results consolidation

In year 3 of the funding period, we will pull together all evaluations over the funding phase (and previously, if appropriate) to draw conclusions on the success of the Aimhigher programme in Nottinghamshire as a whole.

Finally, we will undertake ongoing evaluation of our Higher Education Progression Framework programme, particularly in terms of incorporation into school and college improvement plans, and training provider equivalent, to demonstrate where Aimhigher has impact. We will draw on this evidence base to provide further context to our overall evaluation of school, college and training provider activity.

Evaluation reports

After the end of each funding year of the 2008 to 2011 funding period, we will publish an evaluation report which will be distributed to the Higher Education Funding Council for England (HEFCE) and other interested partners. This report will summarise the main findings of the various evaluations commissioned by the partnership throughout the year. In particular, the report will focus on the extent to which the Aimhigher programme has:

• raised awareness of and aspiration to higher education
• improved attainment
• facilitated learner progression amongst the target group.

The report will not merely focus on apparent ‘successes’, as we believe that important lessons can be learnt from the evaluation methodologies we will implement (in terms of what works and what doesn’t work), and will enable us to continuously improve the Aimhigher programme. Additionally, we will widely disseminate more detailed specific evaluation reports throughout the funding period, as deemed appropriate.

If you have any questions about monitoring and evaluation procedures, please contact Marie Louise Aldridge, Aimhigher Data and Evidence Manager, on 0115 848 2575, email marie-louise.aldridge@ntu.ac.uk.