The current study uses a one-one matched two group design with a clinical sample and a community based sample to examine the discriminant validity of the M&MS questionnaire. Additionally, further analyses are carried out in the clinical sample to establish the internal and inter-rater reliability and construct validity of the measure in a clinic setting.
Sample
Clinic sample
Data were collected from n = 91 (46.2% female, N = 42) children and adolescents (mean age = 12.34 years, SD = 2.03) attending two community out-patient teams from child and adolescent mental health services in an urban location in England (67% from one team and 33% from the other team). In order to allow for comparisons with existing data from a community sample service-users were excluded if they were younger than 8 years, older than 15 years or in circumstances where cases were deemed to be highly sensitive. A large proportion of the sample belonged to the White ethnic group (69.2%, N = 63) and the remaining participants were Asian (N = 8), mixed race (N = 6) or did not have a recorded ethnicity on file (N = 14).
Participants completed the questionnaire either before or after their session with the clinician in the mental health service. Parents and young people were given information about the study and asked for their consent. Participants were informed of the confidentiality of their responses and their right to decline to participate. Ethical approval for collecting these data was received from the National Health Services Research Ethics Committee in England.
Community sample
To allow for comparative analysis with a community sample, matched controls were selected from a sample of young people who had completed the questionnaire in the same year as part of a school based study (N = 863, aged 8–15 years, mean age = 11.97, SD = 1.65; female 48.9%; ethnicity 63.6% White) from 7 schools (4 primary and 3 secondary schools) in urban locations. The community sample was matched to the clinic sample to control for demographic differences between samples biasing the results. This was done because risk of mental health problems has shown to be varied based on gender, ethnicity and age [13]. A one-one matched community sample was created using propensity score matching psmatch2; [14], which allows finding exact or closely matched individuals based on selected criteria. Matching was done based on gender, ethnicity and age and resulted in a matched community sample of 91 participants (49.5% female, 68.6% White, mean age = 12.29, SD = 1.87).
Questionnaires were completed in classroom-based sessions facilitated by researchers. Consent was sought from parents via mail beforehand. All individuals received information about the study, including explanation of the confidentiality of their responses and their right to decline to participate and drop out at any time. Ethical approval for collecting these data was given by the university ethics board at University College London.
Measures
Me and My School (M&MS)
The M&MS questionnaire [9] is a 16-item measure comprising of a 10-item emotional difficulties scale and a 6-item behavioural difficulties scale. Items in the emotional difficulties scale include ‘I feel lonely’ and ‘I worry a lot’; items in the behavioural difficulties scale include ‘I lose my temper’ and ‘I break things on purpose’. Participants respond to each item by selecting one of three options: Never, Sometimes, Always. Total scale scores are created by summing the item scores which results in a possible range of scores of 0–20 for the emotional and 0–12 for the behavioural difficulties scales, a higher score indicating more problems. In case of missing items person-mean (prorated) imputation was conducted for up to a third of items in the scales. During the validation of the measure cut-off scores with clinical significance were established resulting in a score of 10 and above indicating problems on the emotional difficulties scale (10–11 borderline, 12 + clinical) and 6 and above indicating behavioural problems on the behavioural difficulties scale (6 borderline, 7+ clinical). The original measure was developed as an online questionnaire but a paper-based version has since been developed and validated [15] which was used in the present study.
Strengths and Difficulties Questionnaire (SDQ) self-report
The SDQ self-report [16] is a self-report measure of mental health suitable for children older than 11 years. The measure consists of five five-item scales: emotional symptoms, conduct problems, hyperactivity, peer problems and prosocial. The first four scales also sum to give a total difficulties score. Items in this measure are generally longer and more complex than the items in the M&MS (e.gs I am nervous in new situations. I easily lose confidence [or] I fight a lot. I can make other people do what I want) to which participants respond on a 3-point scale (not true, somewhat true, certainly true). This questionnaire was completed by the 56 participants (57% female) in the clinic sample who were old enough (N = 56, 11+ years; mean age = 13.46, SD = 1.29).
Parent SDQ
Accompanying parents or carers were also asked to complete the parent version of the SDQ which like the self-report version is a 25 item measure with five scales [17]. The items in the parent version correspond closely to the items in the self-report version except being in third person form (e.g. Has at least one good friend). 92% (N = 84) of accompanying parents/carers completed the questionnaire (58.3% mothers, 10.7% fathers, 3.6% other and 28% not known).
Clinical assessment
Clinical assessments were made according to ICD-10 diagnosis or ICD-10 Z-code which represent factors influencing health status and service use (e.g. removal from home, emotional neglect, disability). For individuals with no diagnoses, under assessment or a Z-code, presenting problems were recorded. 54% (N = 49) had a clinical diagnosis, 35% (N = 32) had presenting problems, 33% (N = 30) had a Z-code and 7.7% (N = 7) had no recorded diagnosis, z-code or presenting problems.
Two child clinical psychologists then independently classified the diagnoses and presenting problems into groupings based on their clinical expertise and experience. The groupings used were emotional, behavioural, emotional and behavioural and other. This was then collated which resulted in a complete agreement in coding for 82% of the items and any disagreements between the two coding clinicians were resolved in a discussion to ensure there was a clear classifying system. Based on this classifying system, for example, depression and anxiety were classified as emotional and learning disorders, hyperactivity, autism, and tourette’s were in the Other category. These groupings were then applied to assign participants’ diagnoses (and in the absence of a diagnosis, their presenting problems) to these groups. This resulted in 34 individuals with emotional, 7 individuals with behavioural, 13 individuals with co-morbid emotional and behavioural and 25 individuals in the other clinical assessment grouping.
Analysis
Analyses were carried out in four stages to specifically look at different psychometric properties of this measure. In the first stage, internal consistencies were computed to assess reliability of the scale in the clinic setting. In stage two the ability of the M&MS to discriminate between clinical and community samples was assessed using mean comparisons, receiver operating curves (ROC) and comparing proportions above the scales’ clinical thresholds. In the third stage correlations between the M&MS and Parent SDQ and SDQ self-report were explored to assess inter-rater reliability and construct validity. Lastly, the predictive validity of the emotional difficulties and behavioural difficulties scales was examined using clinical assessment.