Parent-adolescent informant discrepancy on the Strengths and Difficulties Questionnaire in the UK Millennium Cohort Study

Abstract

Background

Developmental researchers often use a multi-informant approach to measure adolescent behaviour and adjustment, but informant discrepancies are common. In general population samples, it is often found that parents report more positive and less negative outcomes than adolescents themselves. This study aimed to investigate factors associated with informant discrepancy, including adolescent sex, and parental level of psychological distress and education.

Methods

Informant discrepancy on the Strengths and Difficulties Questionnaire was investigated using a Latent Difference Score (LDS) approach, which estimates the true difference between parent and adolescent reports in a structural equation model. The sample were parent-adolescent dyads from the seventh wave of the UK Millennium Cohort Study (N = 6947, 49.3% female, aged 17 years).

Results

Parents reported lower levels of difficulties (emotion symptoms, peer problems, conduct problems), and higher levels of pro-social behaviour than adolescents themselves. Conditional effects were found, as discrepancy was greater amongst parent-daughter dyads for emotion and peer problems, and greater amongst parent-son dyads for conduct problems and pro-social behaviour. Parent-adolescent discrepancy was also greater generally if parents had a lower level of psychological distress or a higher level of education.

Conclusions

In a large general population sample from the UK, it was found that adolescents tended to report more negative and less positive outcomes than parents reported about them. Conditional effects were found at the parent and adolescent level suggesting that specific informant biases are likely to impact the measurement of adolescent behaviour and adjustment across reporters.

Developmental researchers are generally in agreement about the benefits of using a multi-informant approach, such as from parents, teachers, and self-report, to measure child and adolescent adjustment, which can provide better precision than single reports alone [1, 18, 49]. However, studies consistently report large discrepancies and only low to moderate levels of agreement between informants, resulting in problems of interpretation and leading researchers to question the measurement properties of certain instruments [2, 35].

The Strengths and Difficulties Questionnaire (SDQ) is a widely used, relatively short, multi-informant measure of child and adolescent adjustment, capturing five domains including emotion symptoms, peer problems, conduct problems, hyperactivity-inattention, and pro-social behaviour, in youth aged 3–17 years [16]. A recent meta-analysis found cross-informant agreement on the SDQ to be in the low to moderate range in general population samples between teacher–child reports (r = 0.19–0.33), parent-teacher reports (r = 0.26–0.46), and parent–child reports (r = 0.30–0.38) [2]. Concordance rates on the SDQ have been shown to be higher than other multi-informant measures, such as the Child Behaviour Checklist [17], and tend to exceed levels considered acceptable for cross-informant agreement [2], however large informant discrepancies persist.

Cross-informant agreement on the SDQ tends to be highest for externalising difficulties, such as hyperactivity-inattention, and lowest for internalising difficulties, such as emotion symptoms [2]. This finding is often attributed to the fact that externalising difficulties are more easily observed, compared to internalising difficulties, which may be hidden or unobserved [10, 44]. However, this does not always apply, as a wide array of factors can explain differences in rates of informant discrepancies across domains. For example, conduct problems are considered more observable, yet may be underreported due to social desirability [23], or non-disclosure of certain behaviours to parents, such as antisocial or illicit adolescent behaviour [5]. And, in some cases, internalising problems may be well reported by parents, such as when the parent–child relationship is strong, characterised by more time spent together, including open communication [47].

Studies in clinical samples, of children presenting to services with psychopathology, tend to find higher levels of cross-informant agreement between parents and children, than those found in general population samples [2]. The direction of informant discrepancies also differs between clinical and general population samples, as parents tend to report higher symptoms than children in clinical samples [7, 20], whereas children tend to report higher symptoms than parents in general population samples [12, 24, 44]. This suggests that parents of children in clinical settings may be more aware of their children’s difficulties, which could be attributed to the fact that parents usually facilitate access to clinical services [25].

Reasons for informant discrepancies are likely to be wide ranging, and have been summarised in theoretical models, such as the Attribution Bias Context (ABC) model [10]. This model posits that informant discrepancies may arise due to (a) differences in attribution of the cause of children’s behaviour (e.g., situational vs dispositional), (b) individual informants’ own cognitive biases (e.g., depression-distortion hypothesis), and (c) the reporting context of the informant (e.g., at school or at home) [10]. The latter has often been used to describe parent-teacher discrepancies, as informants report from either the home or the school context, providing valuable information across settings [13].

Mental health, in particular that of mothers, has been identified as a potential source of informant discrepancy. Early research in this area, which found that mothers with depression reported higher symptoms in their children than mothers with no depression, concluded that mothers with depression tended to overreport their children’s symptoms due to their own mental health problems [4, 39]. However, these reports were strongly criticised for lack of empirical evidence and misinterpretation of results [40]. Firstly, higher mental health difficulties would be expected in households where either the parent or child suffers with mental health difficulties, due to bi-directional effects and heightened psychological distress [28]. Secondly, parents with a history of mental health difficulties may be more aware of their children’s distress and sensitive to signs of maladjustment, resulting in more accurate reporting of children’s symptoms compared to parents with no history of mental health difficulties [8].

In summary, while parent reports of adolescent mental health and adjustment are widely used, they are not directly comparable to adolescent self-report. Both types of report are prone to error due to subjective bias and differences in reporting context, and neither can be considered as the gold standard [10]. It is important to understand the factors associated with informant discrepancies, in order to inform models of reporter bias and improve the interpretation of adolescent adjustment measures.

The current study

In the current study, parent-adolescent informant discrepancy was investigated using data from the seventh wave of the Millennium Cohort Study (MCS), collected in 2018–2019 when cohort members were aged 17, reflecting the only timepoint when both parent and self-reported SDQ were collected. In line with previous research, it was hypothesised that parents would report lower difficulties and higher pro-social behaviour than adolescents [12, 24]. Conditional effects were investigated, and it was hypothesised that discrepancy would be greater in parent-daughter dyads, due to the large increase in self-reported difficulties observed in older adolescent girls [36, 37, 42]. In addition, it was expected that informant discrepancies would be lower if parents reported a higher level of psychological distress themselves, due to increased understanding of mental health difficulties [8]. Parent’s highest level of education was also explored as a potential conditioning factor, as research tends to find that higher levels of parental education are protective for children’s mental health [43, 46], although to our knowledge this has not been explored as a moderator of discrepancy.

Latent Difference Score (LDS) modelling was used, which is considered a flexible and robust way to investigate informant discrepancies [9]. LDS models are a specific type of structural equation model, which infer unobserved (latent) constructs from observed (manifest) variables [27]. A true difference score can be estimated from the self and parent reported latent factors, which is considered free from measurement error [14]. The mean difference score, interindividual differences in change, and the covariance between the self-report and the difference score can then be included as model parameters for further investigation [14]. In addition, due to the multiple manifest variables observed within a latent model, measurement invariance across reporters can be tested and accounted for, to ensure that any differences observed are the result of a true difference and not variation in how parents and adolescents interpret and respond to items in a questionnaire [21].

Method

Participants and design

MCS data are freely available and were downloaded from the UK Data Service website. The sample used for the current study was smaller than the total number of participants at wave 7 (N = 10,757). Participants were 6947 parent-adolescent dyads who had at least partial SDQ data and matched demographic information at wave 7. Missing data was observed due to either failure to complete and return questionnaires (14.4%), or in some cases failure to match completed questionnaires to respective parent information (18.2%). To account for missing data, we created an Inverse Probability Weight (IPW), as the propensity to have complete data based on the observed characteristics of the child. The MCS data team already provide survey weights to account for non-response and attrition over time [11]. Therefore, we created a product of the new IPW and the existing MCS wave 7 survey weight, to account for both types of missing data. The sex distribution was equal among participants (49.3% female); although, parents were more likely to be female (85.6%). Most participants were ethnically White (86.2% vs 13.8% ethnic minority), which is representative of the population at baseline.

Measures

The Strengths and Difficulties Questionnaire (SDQ) [19] was collected by self- and parent-report using the respective versions at wave 7, which have previously shown good psychometric properties [17]. The SDQ has five subscales in total, four measuring difficulties (conduct problems, hyperactivity-inattention, emotion symptoms, peer problems), and one subscale measuring pro-social behaviour. Each subscale contains five items, which are rated on a 3-point scale to indicate frequency (0 = ‘Never’, 1 = ‘Sometimes’, 2 = ‘Always’). Confirmatory Factor Analyses (CFA) were run to establish the most suitable factor structure for the data. The five-factor structure was found to be a poor fit for both the parent (CFI = 0.854, RMSEA = 0.049, SRMR = 0.047) and adolescent data (CFI = 0.831, RMSEA = 0.051, SRMR = 0.050). Therefore, univariate CFA were run for each subscale and reporter, which were found to be in the acceptable-good range for conduct problems, emotion symptoms, peer problems, and pro-social behaviour (Additional file 1: Table S1). Model fit was poor for hyperactivity-inattention across reporters and was not examined further, perhaps because this scale was tapping into multiple facets of impulsivity, which are not unidimensional. Internal consistencies are also reported in Additional file 1, which were all in the acceptable range.

The Kessler-6 [26] assessed parent’s level of psychological distress at wave 7. Parents were asked to indicate how often in the last 30 days they had been feeling (“down”, “depressed”, “hopeless”, etc.), using a 5-point scale (0 = ‘None of the time’, 1 = ‘A little of the time’, 2 = ‘Some of the time’, 3 = ‘Most of the time’, 4 = ‘All of the time’). The six items were summed to create a total score (range 0–24). Validated cut-off scores were applied to indicate greater than moderate (≥ 5) levels of psychological distress [26]. One third of parents exceeded the moderate level of distress (34.6%).

Parent’s highest level of education was collected at the baseline of the study. We used data collected at wave 2 (2004–2005), where a sampling boost was carried out, thus reflecting the first wave for some MCS families. If no information was available at wave 2, then information was augmented from wave 1. A binary variable was created to indicate whether parents had a university degree or equivalent vocational diploma (equal to a National Vocational Qualification (NVQ) Level 4 and above: 56.3%), versus a lower level of education (NVQ Level 3 and below: 43.7%).

Statistics

Measurement invariance was tested initially using a series of nested multi-group confirmatory factor analyses [31]. Three models were compared: the baseline model tested for configural invariance (i.e., whether the same factor structure held across groups). If this level of invariance was found to hold based on the model fit [22], then metric invariance was tested, where item loadings were constrained to be equal across groups. Since LDS models concern comparisons of the factor means, scalar invariance (where loadings and intercepts of the items are constrained to be equal across groups) was deemed necessary to ensure comparisons to be free of measurement bias. The loss of fit after introducing any of these constraints was compared against usual criteria, i.e., loss in fit smaller than 0.010 and 0.015 in CFI and RMSEA, respectively [6].

If models were not deemed to hold based on the outlined criteria, a partial measurement invariance approach was used. This approach was implemented using modification indices, which inform about which parameters need to be freed to improve model fit. Since each of the latent variables under study was measured by five indicators, we allowed a maximum of two out of five of the measurement parameters to vary across groups, so the majority of items were invariant before establishing partial measurement non-invariance. Although there is no consensus on the thresholds for partial measurement invariance [38], ensuring invariance in the majority of indicators goes in line with recommendations by Vandenberg and Lance (48), and is slightly more conservative than other approaches where only two indicators are constrained to be invariant [45].

LDS models were then conducted on the best fitting model for each of the remaining constructs using the methods outlined in DeHaan et al. [9]. Using an LDS model specification, a latent factor for the self-report was measured by the self-reported indicators. Similarly, a parent-reported factor was measured with the parent-reported indicators, but the latent mean of that factor was constrained to be zero. By regressing this latent factor on the self-reported latent factor and adding a second-order latent variable measured by the parent-reported latent factor, this latter variable can be understood as the latent discrepancy (or difference) between the two informants’ latent factors. Positive scores on this factor represent higher parent-reported ratings and negative scores represent lower parent-reported ratings relative to self-report. A graphical representation of the unconditional LDS model is presented in Fig. 1A.

Fig. 1

Graphical representation of unconditional (A) and conditional (B) LDS model

Conditional LDS models were then estimated to evaluate whether discrepancy varied as a function of adolescent sex (male/female), parental level of psychological distress (low/high), or parental level of education (low/high). To evaluate conditional effects, the latent self-report and the latent discrepancy factors were regressed on adolescent sex, parental psychological distress, and parental education, in separate models for each exogenous variable. This way, the relationship between the exogenous variables and the discrepancy factors (and, by extension, on the parent’s report) would be net of the potential differences on the self-report by those same exogenous variables. A graphical representation of the conditional LDS model is presented in Fig. 1B. The non-linear combinations of the conditional effects were obtained as appropriate for each of the levels of the exogenous variables, and the 95% confidence intervals of these conditional effects were plotted to visualise the direction of effects.

Analyses were conducted in R using lavaan version 0.6–12 [41]. In our main analyses, we used maximum likelihood (ML) estimation, which is compatible with applying survey weights. However, in order to account for item-level missingness and the non-normality of indicators, we conducted a separate set of analyses using MLR estimation and full information maximum likelihood (FIML) and report these in Additional file 2.

Results

Descriptive statistics

A descriptive comparison of the parent and adolescent version of the SDQ and item-level means, and standard deviations (SD) are presented in Table 1. In general, adolescent report showed higher item mean levels for difficulties and lower item mean levels for pro-social behaviour than parent report.

Table 1 Parent and adolescent version items, means, and standard deviations from the SDQ

Measurement invariance

Measurement invariance was tested with a series of nested multi-group confirmatory factor analyses. Partial scalar invariance was established for all four factors, by using modification indices to identify the parameters that needed to be freed to improve model fit. Table 2 displays model fit indices and the specific items that were freed are presented in the note.

Table 2 Measurement invariance models between parent and self-report

Latent difference scores

LDS models were run on the four partially invariant models (Table 3). Model fit was good in all cases. The latent difference scores were significant and negative for all difficulties (conduct, emotion, peer), showing that parents on average reported lower difficulties compared to adolescents, with the greatest discrepancy observed for conduct problems, and the smallest discrepancy observed for peer problems. The LDS for pro-social behaviour was significant and positive, showing that parents on average reported higher pro-social behaviour compared to adolescents. Variances were all significant, showing that there was significant heterogeneity in discrepancy scores. Covariances were all significant and negative, showing that in cases where the discrepancy factor was negative, the higher the intercept for self-reported difficulties, the greater discrepancy was observed, with the largest effect observed for emotion symptoms. However, for pro-social behaviour, the higher the intercept for self-report, the smaller the amount of discrepancy was observed.

Table 3 Latent difference score models between parent and self-report

Conditional models

Adolescent sex

LDS models conditional on adolescent sex (male/female) were tested by including sex as a predictor of both the self-reported and the discrepancy factors (Table 4). Sex was associated with three of the self-reported factors, as males reported higher conduct problems than females, and females reported higher emotion symptoms and pro-social behaviour than males. Sex was associated with all of the discrepancy factors, most strongly with emotion symptoms. The parameter estimates by group showed that while parents reported lower difficulties than adolescents overall, the discrepancy was greater among female adolescents for emotion and peer problems, and slightly greater among males for conduct problems. Parents reported higher pro-social behaviour than adolescents overall, but the discrepancy was greater in male adolescents (Fig. 2).

Table 4 LDS models conditional on adolescent’s sex

Fig. 2

Predicted mean levels of SDQ factors by reporter and adolescent sex (95% confidence intervals)

Parental psychological distress

LDS models conditional on parental level of psychological distress (low/high) were then tested (Table 5). Parental psychological distress was associated with all of the self-reported factors, as adolescents reported higher difficulties and lower pro-social behaviour if their parent had a higher level of distress. Parental distress was associated with all of the discrepancy factors. The parameter estimates by group showed that parents reported lower difficulties than adolescents overall, but the discrepancy was greater if parents had a lower level of psychological distress, with the largest discrepancy observed for emotion symptoms. The same effect, albeit smaller, was observed for pro-social behaviour, as parents reported higher pro-social behaviour than adolescents, but the discrepancy was greater if parents had a lower level of psychological distress (Fig. 3).

Table 5 LDS models conditional on parent’s psychological distress

Fig. 3

Predicted mean levels of SDQ factors by reporter and level of parent psychological distress (95% confidence intervals)

Parental level of education

Finally, LDS models conditional on parental level of education (low/high) were tested (Table 6). Parental education was associated with three of the self-reported factors, as adolescents reported lower conduct and peer problems, and higher pro-social behaviour if their parent had a higher level of education. Parental education was associated with two of the discrepancy factors, as conditional effects were observed for emotion and conduct problems. The parameter estimates by group showed that parents reported lower emotion and conduct problems than adolescents, and the discrepancy was greater if parents had a higher level of education (Fig. 4).

Table 6 LDS models conditional on parent’s level of education

Fig. 4

Predicted mean levels of SDQ factors by reporter and level of parent education (95% confidence intervals)

Sensitivity analysis

We ran sensitivity analyses applying full information maximum likelihood estimation with robust standard errors to all LDS models (Additional file 2), as this was not possible in combination with survey weights. Estimates varied slightly, but results remained the same across all unconditional and conditional models. However, there was one case where the discrepancy factor became non-significant in the unconditional model for emotion symptoms, although the point estimate remained very similar.

Discussion

In this study, parent-adolescent informant discrepancy was investigated for the first time on the SDQ, in the largest nationally representative birth cohort study of adolescents in the UK. Informant discrepancies were estimated using LDS modelling, and it was found that parents reported lower levels of difficulties (emotion, peer, conduct), and higher levels of pro-social behaviour than adolescents, which was in line with previous research in large scale general population samples in Chile [12] and Japan [24]. Findings are discussed below in the order they were presented in the results.

We were unable to find support for the proposed five-factor structure of the SDQ, which is not an uncommon finding [3, 15, 32, 34]. Previous confirmatory factor analyses conducted on the SDQ using MCS data have supported the five-factor structure at ages 5, 7, 11, and 14, but not age 17 [32,33,34], calling into question the acceptability of the SDQ factor structure in older adolescents. Despite this, most research using the SDQ examines individual mean scores of the subscales, therefore the approach taken in this study to examine factors independently can be considered valid. However, model fit was poor for the hyperactivity-inattention scale, therefore this construct was not explored further, perhaps because this scale is not unidimensional in older adolescents.

Measurement invariance was tested prior to estimating LDS models. Most factors showed weak (metric) invariance, while strong (scalar) invariance was not observed in any model. This is not an uncommon finding, as scalar level invariance is very difficult to support [29], but can be approximated through partial invariance [21]. The fact that full scalar invariance was not supported is perhaps unsurprising given that the wording is slightly different for some items between the parent and self-report version of the SDQ. For example, item 16 of the parent version from the emotion subscale (“nervous or clingy in new situations”) could have different connotations to the self-report version (“I am nervous in new situations”), especially among older adolescents. Indeed, this was one of the items that had to be freed across reporters. Despite these limitations, partial scalar invariance was established for all four remaining factors, by allowing one or two items to be freely estimated, which is in line with recommendations from the literature [38, 48]. While we acknowledge that this is not the ideal scenario, it better reflects the measurement attributes of the scale, rather than assuming invariance (and equal measurement properties across items) that underlies typical comparisons on the raw sum or mean scores [30].

Discrepancy findings

To our knowledge, this is the first study to estimate latent difference scores among parents and adolescents on the SDQ. The largest parent-adolescent discrepancy was observed for conduct problems (almost 1 standard deviation difference), despite that externalising problems are often said to be more easily observed than internalising problems [2]. Parents may have underreported conduct problems due to either social desirability [23], or perhaps unawareness of adolescent’s behaviour, especially as participants were aged 17, when independence from parents is likely to be greater than at younger ages.

Discrepancies were also observed for emotion symptoms and pro-social behaviour (about half a standard deviation), with a minor amount of discrepancy observed for peer problems. The fact that parents tend to report more positive and less negative outcomes than adolescents suggests that on average parents have a general positive bias about their children. In light of this, future research would benefit from including a third more impartial reporter, such as teachers or peers, which could shed light on this further. An age effect could have also played a part in our study, as adolescents may have underreported positive and overreported negative outcomes due a general negative self-referential bias, which is characteristic of older adolescents [43]. Therefore, future research would benefit from assessing parent-adolescent discrepancy across different stages of development.

Conditional effects

Conditional effects were observed for adolescent sex, whereby parent-adolescent discrepancy was greater for emotion symptoms and peer problems in female adolescents, and greater for conduct problems and pro-social behaviour in male adolescents. The largest effect was observed for emotion symptoms, as discrepancy was more than twice as large in parent-daughter dyads compared to parent-son dyads. This finding suggests that the large sex differences observed in adolescent emotion symptoms between males and females [36, 37], could be partly overestimated using self-report measures. However, emotion symptoms reflect internalising problems that may be more difficult to observe, particularly for parents with lower awareness of the associated signs and symptoms. Therefore, parents may be more likely to under-report symptoms in females, due to the higher degree of prevalence and severity in female adolescents.

Large sex differences were also observed for pro-social behaviour, whereby parent-adolescent discrepancy was greater in male adolescents. This could be related to the fact that items used to measure pro-sociality (e.g., kindness, consideration) are more analogous with female identity, and therefore possibly underreported by male adolescents. A recent study in a general population school sample of younger adolescents (median age 11 years) in Switzerland examined teacher–child informant discrepancy on pro-social behaviour and found that teachers reported lower pro-sociality across sexes, but to a greater extent in males [32]. Although this effect was in the opposite direction, as adolescents reported lower pro-sociality than parents in our study, together, this suggests that informant discrepancies for pro-sociality are likely to be greater in males, perhaps due to a general bias within society that associates pro-social behaviour with female identity.

The effect of parental psychological distress on discrepancy was significant for all four factors, as compared to parents with low psychological distress, reports from parents with higher levels of distress were more consistent with their children’s reports. This was evident for reports of pro-social behaviour, as well as all difficulties, suggesting that parents with higher levels of psychological distress may be less biased and more accurate at identifying both strengths and difficulties. This is in contrast to early research that questioned parental reports of children’s behaviour and adjustment when parents showed mental health difficulties [4, 39]. Our findings support other research in this area, that parents with mental health difficulties show more accurate reporting of children’s behaviour [8], perhaps because these parents are more sensitive and attuned to signs of maladjustment. It is also possible that awareness of children’s difficulties could be a causal factor explaining higher levels of parental psychological distress, although the direction of effects could not be disentangled in this study.

Parental level of education was associated with two of the discrepancy factors, as parent-adolescent discrepancy for emotion and conduct problems was greater at higher levels of parental education. This is somewhat surprising, especially since higher levels of parental education are known to be protective of children’s mental health and adjustment [43, 46]. Indeed, we found that adolescents themselves reported fewer emotion and conduct problems if their parent had a higher level of education. Therefore, the greater discrepancy cannot be attributed to a higher intercept for these scores. This finding suggests that higher educated parents may have a larger positive bias about their children than parents with a lower level of education. However, these conditional effects were small and reflect a novel finding, therefore should be replicated in other samples before drawing firm conclusions.

Strengths and limitations

To our knowledge, this is the first study to examine informant discrepancies on the SDQ using a latent difference score approach. This approach has a number of advantages over traditional methods, including the possibility to test (rather than assume) measurement invariance, and calculate a true difference score that is as free from measurement error as possible [14]. In addition, the large and nationally representative sample used enable conclusions to be generalised to the population. A limitation of the study was the missing data observed on the SDQ. Nevertheless, a substantial proportion of the sample were analysed, and survey weights were constructed specifically for this study to help restore sample representativeness. Some benefits of the SDQ include its wide breadth of domains, covering internalising, externalising, and pro-social behaviour. However, model fit for the five-factor structure was found to be poor, therefore future research may wish to use other scales with better psychometric properties. We would expect the findings regarding parent-adolescent informant discrepancy to be consistent across other measures of adolescent adjustment.

Conclusion

In a large general population sample from the UK, using novel methodology, this study found that adolescents reported more negative and less positive outcomes on the SDQ compared to parents. Discrepancy between reporters was found to be conditional on various factors, including adolescent sex, and parental level of psychological distress and education, showing that specific informant biases are likely to impact measurement of adolescent adjustment at each reporter level. To build on this research, future work would benefit from including more reporters, such as peers and teachers to give a fuller picture, as well as investigating discrepancy across different stages of adolescence, and including alternative multi-informant measures.