Psychometric properties of online administered parental strengths and difficulties questionnaire (SDQ), and normative data based on combined online and paper-and-pencil administration

Annual or cumulative prevalence rate of mental health problems and psychiatric illness among youth are high (e.g., [1–3]). Children with early onset of either internalizing problems such as anxiousness and withdrawn behavior (e.g., [4]) or persistent externalizing problems such as defiant and disruptive behavior (e.g., [5–7]) are also at higher risk for continued severe and debilitating mental health problems during adolescence and adulthood [8, 9]. Early interventions in terms of parent management training (PMT) programs (e.g., [10–12]) for troubled children have been found to reduce the risk for developing later mental health problems. To reliably identify children with elevated risk for continued externalizing and internalizing problems, and to be able to evaluate the effectiveness of interventions targeting these children and their parents, we need reliable and valid instruments.

Most available instruments with strong psychometrics in this context focus heavily on children’s difficulties only (e.g., Child Behavior Checklist, CBCL: [13]), instead of capturing both strengths and problems. The Strengths and Difficulties Questionnaire SDQ: [14–16] is a well-known, frequently used instrument in research [17, 18], that can be completed in five minutes by parents or teachers for assessing psychological problems and prosocial behaviors among children aged 3–16 years. A unique aspect of the SDQ is its focus on not only problems, but also on the strengths. Scores derived from the SDQ are highly correlated with CBCL, and it is significantly better than the CBCL in detecting inattention and hyperactivity when they both are compared to a semi-structured interview [19]. The SDQ has been evaluated widely internationally and is considered to be an instrument with good psychometrics (for a review, see [20]). Although some studies have replicated its original factor structure (e.g., [21]), other studies have failed to reproduce the same factor structure (e.g., [22–24]), and normative data for parent-ratings of older children in Sweden are missing.

Increased use of online data collection in both screening and intervention studies, as well as in clinical practice, raises new questions about the psychometrics of instruments administered online and applicability of norms obtained through traditional mode of data collection (i.e., paper-and-pencil). Some systematic differences have been noted in response to the questionnaire administered online versus on paper-and-pencil [25, 26]. Joinson [27] found that people consistently gave higher severity rating on their online report compared to paper-and-pencil. However, most studies have found high correlations between scores obtained from these different modes of data collection and that the clinical relevance of found discrepancies are generally negligible (e.g., [28–31]). Research on the psychometrics of instruments measuring child behaviors online is scarce, and that is especially true for the SDQ. Furthermore, we need to know whether norms obtained from the online surveys are different from those collected by paper-and-pencil. Last, but not least, Swedish norms for children and adolescents aged 10–13 are not yet available, and thus needed. Consequently, the aim of the present study was to 1) examine the psychometric properties of the online administered parental rating of the SDQ in terms of its internal consistency, factor structure, and concurrent validity with another instrument measuring similar constructs, and 2) provide parental norms for the SDQ from a nationwide representative Swedish sample of 10–13 year old children.

Use of questionnaires such as the SDQ is a cost-effective way of collecting data from different informants. Data from such instruments might be a good starting point for a decision to collect more data for selection and intervention purposes. However, the usefulness of these instruments is dependent on their psychometric properties and availability of norms from the general population.

The psychometrics of the SDQ in previous research have been fair to good given the mixed finding in reproducing its original factor structure. In total, satisfactory internal consistency and test-retest reliability have been reported (for a review, see [20–22, 39, 40]). The present study is the first one to provide more information on the psychometrics of the SDQ when administered online. Given the large skewness and kurtosis of three of its items, polychoric ordinal alpha was used to investigate the internal consistency. The SDQ subscales showed good to excellent reliability, ranging from .85 to .91. This is higher than what other studies have reported e.g., .57 to .77 [32], .58 to .76 [41] and .59 to .80 [42]. These studies have not used polychoric ordinal alpha, but instead Cronbach’s alpha, which might explain some of the differences. The reported internal reliability pattern is very similar across the studies, Hyperactivity-Inattention subscale has the strongest reliability and Peer Problems subscale has the weakest. The SDQ correlated significantly in expected direction with the DBD, which supports the validity of the SDQ. In addition, the confirmatory factor analysis of data obtained from online administration of the SDQ resulted in an excellent fit. The validity of the five-factor model was supported, which supports its construct validity.

Norms for the SDQ (parent version) for the targeted age of children is scarce. In a previous validation study of the SDQ [43] data were obtained from 263 randomly selected parents of children 5–15 years old in the general population, but the authors did not present the norms in detail besides a figure presenting mean value of the Total Difficulties score of SDQ and subscales for the entire sample. Comparing norms from the present study to an Australian study [42] with somewhat younger children shows good similarities, with the exception of slightly lower mean value of reported Emotional Symptoms in the present study for girls (1.6) compare to the Australian study (2.0). The same pattern emerged in comparing norms for boys in the current study to the Australian norms. Present norms correspond very well with those obtained from other Scandinavian countries as well (for a review, see [44]). Comparing the central tendency figures in the present study with those obtained in Denmark (Aarhus) and Norway (Akershus), shows that data in the present study fit slightly better with the Norwegian norms, and is slightly higher than the Danish norms. However, the differences are small and the pattern of data in the present study confirms the conclusion made by Obel et al. (2004) that the SDQ scores are very similar across Nordic countries. An important finding in the present study is that norms obtained from online administration of the SDQ are very similar to those using paper-and-pencil. This study therefore supports the idea that the SDQ can be used for online data collection without any concerns about its psychometrics or how the obtained scores can be interpreted in relation to the norms from the general population.

The present study had some limitations that are worth mentioning for future replications. The first was a larger response through paper-and-pencil condition than via the Internet, instead of a fairly equal response rate. However, we did not find any differences between those responding via the Internet compared to those who did it by using paper-and-pencil. The parent’s educational level is slightly higher than the average educational level of parents in Sweden. This seems to be a common bias in research studies. In other words, parents with higher education are more willing to participate in such studies than those with lower education. Nevertheless, we found no substantial differences (i.e., with at least medium or close to medium effect sizes) on the SDQ between respondents with higher versus lower education. The percentage of girls and boys at each age interval was also compared to the national data retrieved from the Statistics Sweden (http://www.scb.se) on all girls and boys aged 10–13 years. At each age interval in the total population, about 48.5% are girls and 51.5% boys. In the study sample the mean percentage of girls at these ages is about 51% (boys 49%). Gender distribution of boys and girls in the sample might thus be viewed as representative of the general population of children at these ages. The response rate (just above 51%) was low despite several reminders and some incentives to increase the response rate. It would also have been informative to have some data on the age of the responding parents. Unfortunately this parameter was not included in the questionnaire. Lack of information regarding the characteristics of children (e.g., native language, order and number of siblings) is another limitation of the study. Such information would have been helpful for investigating the representativeness of the sample in relation to national data. Finally, some studies have shown that those who do not respond to surveys regarding psychological problems tend to suffer from such problems to a higher extent than those who are more willing to participate (e.g., [45]). Translated into the context of the current study, parents whose children exhibit some conduct problems might be less willing to participants in studies such as the current one. On the other hand, obtained norms in our study are in line with norms based on studies with less drop-out. Nevertheless, our findings need to be interpreted with caution as a risk for positive selection cannot be ruled out. As the participating parents reported a higher level of education than the general population, and a larger portion of them were married, available norms from the present study should also be viewed in light of these limitations.

The present study is the first to provide data on the psychometrics of the SDQ when administered online and it makes the SDQ more useful in the context of screening, assessment and evaluation of interventions due to provision of norms from the general population.

Written informed consent was obtained from all the participants.

AB is a Ph.D. candidate, and clinical psychologist, and licensed psychotherapist. PE is Med. Dr., clinical psychologist and licensed psychotherapist. Her major research area is about psychopathology and treatment of disruptive behavior among children and adolescents. AG (Ph.D.) is a professor of clinical psychology, clinical psychologist and licensed psychotherapist. His main focus in research is on psychopathology, prevention and treatment of eating disorders and conduct problems.