Clinical validity of the Me and My School questionnaire: a self-report mental health measure for children and adolescents

There is increasing interest in how best to get children and young peoples’ own views on their psychological state and sense of wellbeing. Whilst there are increasing numbers of child report measures for a range of psychological problems [1, 2] most of these are for one particular type of problem, do not go below the age of 11 and charge to use (ibid). Measurement of mental health in children to date has typically been achieved by measures completed by other reporters. With the increasing focus on children’s perspective being important and necessary [3, 4] which is reflected in policy focus on shared decision making in health services and the concept of self-defined recovery [5] there is a real need for measures that are valid and reliable for younger children. Moreover, research indicates that children as young as 7–8 years old are able reporters of their own mental health [6, 7]. In community settings, particularly schools, self-report measurement supports screening for problems and early intervention [8]. A recent review of self-report general mental health measures [1] highlights the lack of self-report measures of general mental health for young people aged less than 11 years old. Additionally existing measures developed for wide-spread use in both community and clinic settings cost to use and can be impractical for large scale population based studies and routine outcome monitoring due to the financial burden associated with large-scale and/or long-term use. Hence, the development of the Me and My School questionnaire M&MS; [9] filled a necessary gap for a free-to-use, short, self-report screening measure of child mental health that was suitable to use with a wider age range of young people and covers both emotional and behavioural difficulties.

The M&MS questionnaire has been validated with children as young as eight years old, which makes it (as far as the authors are aware), the only free to use, validated, self-report screening measure of general mental health for children of that age. The measure was developed with short items and simple language to especially facilitate use with younger children [9]. The measure also translates to clinical settings as clinical thresholds of risk have been established to aid school based staff and practitioners in identifying high-risk children. Initial validation and analysis of psychometric properties revealed it to be a measure with good content validity, internal reliability, construct validity and minimal item-bias [9]. As per criteria outlined to validate questionnaires [10–12] these analyses indicated that the measure had good psychometric properties for the criteria that have been looked at. However, Deighton et al. [9] also recognised that assessing the properties of the measure in a clinical population would be essential towards establishing the screening capacity and clinical usefulness of the measure. Particularly, assessing the ability of the test to discriminate between community and clinic populations and the utility of the established cut-off scores are necessary steps in determining its utility as a screening tool [12].

The present study aims to test the ability of the measure to discriminate between a clinic and community sample (discriminant validity), assess the internal consistency of the scales in a clinic sample (internal reliability), compare it to another self-report measure (construct validity), examine cross-informant agreement with parent completed questionnaires (inter-rater reliability) and explore the correspondence between scale scores and clinical assessment.

The current study uses a one-one matched two group design with a clinical sample and a community based sample to examine the discriminant validity of the M&MS questionnaire. Additionally, further analyses are carried out in the clinical sample to establish the internal and inter-rater reliability and construct validity of the measure in a clinic setting.

The M&MS measure is a recently developed measure that has been widely used as part of a national evaluation of school-based mental health support in England (40,000 plus young people [19]). Although Deighton et al. [9] established the measure’s properties in a community sample, additional analyses of the measure’s capacity to discriminate between high-risk and low-risk samples was a necessary step to justify its use as a self-report screening tool for mental health problems in community settings. This is especially relevant as the measure is currently being introduced as part of a UK national initiative to improve quality of psychological therapies and a programme promoting use of outcome monitoring in child and adolescent mental health services in the UK [20]. There is increasing emphasis on user perspective and patient-reported outcomes measures [21]. However, to date this has been more problematic for younger populations, with almost all data being provided by adult proxies on their behalf [22]. With further development, this measure has the potential to fill the gap for a free-to-use, brief, self-report measure that extends into this pre-adolescent age range in both community and clinic settings. The main benefit of examining the transferability of the measure from community to clinic settings is that it provides practitioners a simple self-report tool to help assess clinical need.

Analyses indicate that both the scales, emotional difficulties and behavioural difficulties, of the M&MS questionnaire sufficiently discriminate between a clinic (high-risk) and community (low-risk) sample. The amount of discrimination as represented by the AUC statistics (emotional difficulties = .79, behavioural difficulties = .77) are comparable to the AUC of the emotional symptoms scale (.75) and the conduct problems scale (.77) of the self-report SDQ [16]. In mental health in particular, very high scores for discrimination (e.g., >.8) are rare, partly because of the overlap in characteristics of community and clinical populations. Specifically, for mental health problems, being in a community sample does not indicate the absence of clinical problems and, correspondingly, a substantial proportion of young people who attend mental health services have no impairment or diagnosis [23]. Mean differences between the clinic and community sample were statistically significant with large effect sizes (d > 1.1) with individuals in the clinic sample being 4.5 times more likely to be above the threshold indicating problems on the emotional difficulties and 6 times more likely to be above the threshold indicating problems on the behavioural difficulties scale. Sensitivity of the individual scales of M&MS was 40-41%, The overall sensitivity of the scales’ thresholds was 58%, which is comparable to the 59% found for the SDQ [16]. This suggests that even though the M&MS is a brief measure with a general mental health focus it captures clinical need to a similar level as other brief measures such as the SDQ. In terms of the community sample 12% had scores above the threshold, which is lower than the 23% found by Goodman et al. [16] for the SDQ but is more in accordance with the 9-12% expected from a normal representative population in this age range [13, 24].

The measure has good internal reliability as indicated by the Cronbach’s alphas of the two sub-scales (emotional difficulties, α = .84; behavioural difficulties, α = .82). The correlations between the corresponding scales of the M&MS and self-rated SDQ were high (emotional difficulties r = .85; behavioural difficulties, r = .56) and compared favourably to the correlations found in community samples (emotional difficulties r = .67; behavioural difficulties r = .7 [9]) which supports the measure’s construct validity in the clinic setting. Correlations between the corresponding scales of the self-reported measures and the parent SDQ were similar for M&MS with the parent SDQ (.3) and the self-report SDQ with the parent SDQ (.4). Overall the inter-rater correlations for the M&MS were in line with expected correlations between parent and child report which are generally significant but not high and were comparable to results from other measures (average r = .25) found in a meta-analysis [25].

Cross-referencing clinical assessment with the scale scores provides some evidence that both scales are responsive to clinical diagnoses as indicated by the mean scores and proportion above the clinical threshold in each diagnostic group. Given the numbers are small this could be a chance observation but the finding suggests that children with only a behavioural assessment might have more difficulties perceiving problems with their own behaviour. Alternatively the measure might not be effective at capturing self-reported difficulties in this particular sub-population. Additional research is required specifically exploring this discrepancy in self-reporting behavioural problems.

While clinical assessments provide early indication of the scales’ sensitivity to case type, the small numbers identified within each diagnostic category mean that formal statistical testing could not be carried out. This is something that could be explored in further studies when the measure is used more widely with clinic samples. Of particular interest is the consideration of an amendment to the clinical thresholds of the emotional difficulties scale. In the initial validation [9], thresholds were computed using an equi-percentile approach with the SDQ in the community sample. The results of this study indicate that a lower threshold might capture clinical levels of emotional problems better. This could be compounded by the use of thresholds developed on the computer-based survey, as children have been shown to report less problems on the paper based survey of the questionnaire [15], suggesting the need for different norms and thresholds for the paper survey. Future research should explore this possibility to ensure the measure has optimum screening capability.

The primary aim of the current paper was to establish the credentials of M&MS as a screening tool for use in community settings. As such, results indicate that the measure discriminates sufficiently between clinic and community samples. However, the measure requires further research regarding responsiveness to change over time.

In conclusion, the findings of this study indicate that this measure sufficiently discriminates between high-risk (clinic) and low-risk (community) samples, has good internal reliability, compares favourably with existing self-report measures of mental health and has comparable levels of agreement between parent-report and self-report to other measures. Alongside existing validation of the M&MS [9, 15], these findings justify the measures use as a self-report screening tool for mental health problems in community settings. Hence, the measure fills the gap for a free, brief self-report measure for under 11 year olds that can be used in community settings to identify children with mental health difficulties to facilitate them receiving the right support and intervention.