Open Access

Do school teachers and primary contacts in residential youth care institutions recognize mental health problems in adolescents?

Child and Adolescent Psychiatry and Mental Health201610:19

https://doi.org/10.1186/s13034-016-0109-4

Received: 4 April 2016

Accepted: 22 June 2016

Published: 29 June 2016

Abstract

Background

Studies show that adolescents living in residential youth care (RYC) institutions experience more mental health problems than others. This paper studies how well teachers and primary contacts in RYC institutions recognize adolescents’ mental health problems as classified by The Child and Adolescent Psychiatric Assessment diagnostic interviews (CAPA).

Methods

All residents between 12 and 23 years of age living in RYC institutions in Norway and enrolled in school at the time of data collection were invited to participate in the study. Of the 601 available children, 400 participated in the study, namely 230 girls, mean age = 16.9 years, SD = 1.2 and 170 boys, mean age = 16.5 years, SD = 1.5. The Child Behavior Checklist (CBCL) and Teacher’s Report Form (TRF) were used. The sensitivity and specificity of these instruments were studied.

Results

We observed a significant gap between the mental health problems diagnosed by the CAPA interviews and the problems reported by primary contacts on the CBCL and by teachers on the TRF. The CBCL showed a higher sensitivity than the TRF, whereas the TRF showed a higher specificity than the CBCL. Both primary contacts and teachers classified externalizing problems fairly well such as ADHD in both genders and conduct disorder in girls. Both teachers and primary contacts, however, had more problems detecting internalizing problems. Teachers may have a tendency to view most students as healthy and to underestimate the severity of their problems, whereas primary contacts may tend to overestimate the number of problems and view adolescents as sicker than they really are.

Conclusion

The Child Welfare System should revise their intake procedures to detect possible problems early on and to introduce the necessary treatment. It is important to identify factors that increase healthy school adaption in order for these adolescents to accomplish school in a proper way since education is important for a successful adult life.

Keywords

Mental health Adolescents Residential youth care Primary contacts Teachers

Background

Compared with other children, adolescents in contact with the Child Welfare System (CWS) tend to be less successful later in life across a wide range of areas [1, 2]. These adolescents experience problems with mental health, drug addiction, crime, poor education, and unemployment [3, 4]. According to Harpin et al. [5], out-of-home youth in Ireland had greater risks (suicidal risk, mental health distress) and fewer protective factors (feeling that parents and other adults care about them and a sense of school connectedness) than those in the comparison group. Several studies have confirmed that CWS clients have more mental health problems than others [6, 7]. A recent Norwegian study reported that 76.2 % of the youth living in residential youth care (RYC) in Norway fulfilled the symptoms, onset, duration and impairment criteria for at least one DSM-IV diagnosis. That study reported higher prevalence rates for depressive and anxiety psychiatric disorders than for behavioral disorders [8]. The most frequent diagnoses or diagnostic categories observed were depression and dysthymia (37.3 %), followed by any anxiety disorder (34.9 %), Attention Deficit Hyperactivity Disorder (ADHD) (32.3 %) and Asperger’s Syndrome (AS) (23.2 %); however, only 37 % reported receiving help for these diagnoses. According to Levitt [9], there is a significant gap between children in the CWS population who need services and children who receive services, as the majority of child welfare agencies do not systematically screen children in the CWS for mental health problems.

Adolescents who have been removed from home because of a lack of adequate parental care rarely have access to consistent educational support, which is a resource that is taken for granted by most adolescents who live with their parents [10]. There is no reason to think that adolescents in RYC are better off in this area. In general, adolescents in out-of-home care are at a high risk of having poor educational outcomes [1113], and they have lower rates of school attendance [14], more cases of drop-outs [15] and lower grades compared to children living at home [16].

School failure is one of the more serious negative outcomes for young people in CWS. International studies have consistently shown that they score significantly below their peers on a range of school outcome measures [17]. In a Norwegian study by Clausen and Kristofersen [12], 35 % of former CWS clients had completed high school, compared to 80 % of a non-client sample. Similar results have also been found in Sweden [13]. Jaffee and Gallop [18] found that relatively few CWS adolescents (approximately 40 %) function normally in school and that even fewer are resilient across several domains, i.e., school achievement, mental health, and social competence. In addition, caregivers’ attitudes towards school may influence children’s success in school [19]. Marginalization and social exclusion are considered to be outcomes of a lack of coping in school [20, 21], as education plays a major role in an individual’s ability to successfully settle into adult life.

It has also been reported that adolescents’ secondary school careers are negatively affected by the presence of acute psychosocial health problems [22]. Kessler et al. [23] have reported that in the United States, adolescents with psychiatric disorders account for 14.2 % of high school dropouts. Furthermore, externalizing problems are reported to impair educational attainment [24]. Poor educational attainment has also been found to predict the onset of schizophrenia spectrum disorders [25].

Adolescents in RYC do not have parents to attend to their needs, and they depend more on other people, for example on primary contacts in RYC institutions or teachers, to disclose their problems. Adolescents spend a substantial amount of time in school, and it is therefore important for teachers to help detect serious problems. However, because of residential instability, adolescents in care tend to experience multiple school transfers [26], which makes it difficult for teachers to observe symptoms over time. On the other hand, for some adolescents in RYC, teachers may be among the more stable persons in their life.

Studies have shown that teachers in Scandinavia generally report relatively low levels of emotional/behavioral problems among school-aged children [27]. However, we do not know how well teachers detect mental health problems in children living in RYC, who, according to studies, suffer from far more mental health problems than the general population [6].

Teachers are reported to be more accurate in identifying children who are at risk of externalizing disorders than those at risk of internalizing disorders [28]. Internalizing problems such as feelings of depression or loneliness are presumably less observable and depend more on interpretation by informants than externalizing problems such as fighting or teasing. Recognizing mental health problems, however, is very important among adolescents living in RYC, in which more than 70 % of adolescents have been found to meet the criteria for at least one psychiatric disorder [8, 29].

Previous studies on teacher’s reports of adolescent mental health problems most often included adolescents living with families and often focused on younger children. To our knowledge, no studies have focused on adolescents living in RYC institutions and their situations at school. Earlier studies tended to include children in the CWS system in general.

The aim of the present study which is part of a larger study on adolescents in RYC was to explore whether mental health problems, as assessed by The Child and Adolescent Psychiatric Assessment (CAPA) [30], among adolescents living in RYC institutions were detected by primary contacts at their institutions and their teachers. The research question was whether adolescents’ internalizing (affective and anxiety) disorders, conduct disorder (CD) and ADHD problems as reported by teachers and primary contacts were consistent with the diagnostic categories identified in CAPA [30]. As the symptoms of externalizing (CD) problems and ADHD are more easily identified as disruptive, we hypothesized that teachers and primary contacts would more easily detect these two categories than internalizing problems.

Method

Participants

All residents between 12 and 23 years of age living in RYC institutions in Norway and enrolled in school at the time of data collection were invited to participate in the study. The age group 12–23 was chosen because that was the age group available in the RYC institutions. Unaccompanied minors without asylum in Norway and youth placed in acute care were considered to be in such a high state of crisis that collecting their data was not prioritized, and they were therefore excluded from the study, see flowchart of the study, Fig. 1. Youth who lacked sufficient proficiency in Norwegian to be interviewed were also excluded. For more details about the sample, see Jozefiak et al. [8].
Fig. 1

Flowchart of number of participants in the study

Setting

RYC institutions in Norway are organized by The Norwegian Directorate for Children, Youth and Family under the Ministry of Children and Equality. The directorate is responsible for all RYC institutions, but the institutions can be both publicly and privately owned. A Norwegian RYC institution is typically a small unit (3–5 residents) in which youth are encouraged to live as close to a normal life as possible, attending school and participating in leisure activities.

The CWS decide, as part of their intake procedures, what kind of care is best suited for each child, mostly foster care or RYC. For older children it is more difficult to find foster care so adolescents are mostly placed in RYC. It differs how long the children stay in RYC. Intentionally they stay as short as possible, however, for some their home situation is not good enough for moving back. Most of the children have contact with their biological families during the stay.

At the institutions each child is assigned a primary care giver among the available RYC staff during the stay. The RYC staff often holds a bachelor degree in social, health or pedagogical areas, however, about a third of the staff is without higher education. The work of the staff is based on a milieu therapeutic model and shows a generally limited knowledge of psychiatric diagnosis and treatment.

Procedures

A database of all RYC institutions in Norway was created by the project team based on information from The Norwegian Directorate for Children, Youth and Family Affairs. The RYC institutions were randomly selected and contacted in a random order. Data collection was conducted by four trained research assistants in the RYC institutions between June 2011 and July 2014 and lasted approximately 4 h per youth. Due to the length of CAPA and the adolescents’ challenges related to concentration and stamina, not all residents were able to complete the psychiatric interview. The child’s primary contact reported on each resident’s mental health problems using The Child Behavior Checklist (CBCL) [31]. The adolescents in RYC attended the local schools. All students are assigned to a homeroom teacher who has a special responsibility for the adolescent in school, including filling in forms and offering student-parents meetings minimum twice a year. This teacher collects information from other teachers about subjects other than his own. The person at school working closest with the child (homeroom teacher or teacher assistant) filled out the Teacher’s Report Form (TRF) [32].

The few participant 19 years old (N = 5; 1.8 %) were assessed with the Achenbach System of Empirically Based Assessment (ASEBA) 11–18 year versions [31]. This was assumed to give more similar and comparable information across age-groups than using another instrument for the oldest.

Participants were recruited using procedures approved by the Norwegian Regional Committee for Medical and Health Research Ethics, and written consent was obtained. The parents have the custody of adolescents when the placement is voluntary, and the CWS service has the custody of adolescents placed involuntary. Informed written consent was signed by the adolescents regardless of their age. According to the Norwegian Health Research Legislation at the age of 16, the adolescents are considered old enough to sign their own consent. For adolescents under the age of 16, written consent was also provided by parents or CWS.

Measures

Achenbach et al. [33] constructed several measures within their package Achenbach System of Empirically Based Assessment (ASEBA). Three of those were used in the present study: CBCL, TRF, and CAPA. We have not found any studies reporting on the associations between The Child and Adolescent Psychiatric Assessment (CAPA) [30] and The Child Behavior Checklist (CBCL) [31] or the Teacher’s Report Form (TRF) [31, 32]. However, there have been some studies on the associations between scores on the CBCL and TRF. A large study in 21 societies by Rescorla et al. [34] found that CBCL scores were relatively higher than the TRF scores on most scales.

The Child Behavior Checklist (CBCL) consists of 118 Likert-type and two open-ended items rated on a 0–2 scale (0 = not true, 1 = somewhat or sometimes true, or 2 = very true or often true). For the present study, we used the following eight syndrome scales from the 2001 version [31] of the checklist for children and adolescents aged 6–18 years: Anxious/depressed, Withdrawn/depressed, Somatic complaints, Social problems, Thought problems, Attention problems, Rule-breaking behavior and Aggressive behavior. The Norwegian version of the CBCL has shown satisfactory reliability and validity (alphas of 0.93, 0.84 and 0.89 for the total scale and Internalizing and Externalizing subscales, respectively) [35]. According to the Multicultural Supplement to the ASEBA manual [36], Norway is included in Group 3, and the norms and cut-offs were set according to this group; see Table 1.
Table 1

Cut-offs of the different diagnoses according to the cultural norm in the Multicultural Supplement to the manual for the ASEBA school-age forms and profiles

 

Boys

Girls

Borderline

Clinical range

Borderline

Clinical range

CBCL category (cultural norm 1)

 Affective problems

5

8

5

8

 Anxiety problems

3

5

4

5

 Conduct problems

5

8

5

8

 ADHD problems

7

10

6

8

TRF category (cultural norm 2)

 Affective problems

7

9

5

8

 Anxiety problems

3

5

3

5

 Conduct problems

9

14

5

10

 ADHD problems

17

22

12

17

Teacher’s Report Form (TRF) [32]

To date, this is one of the most used measures of emotional/behavioral problems in school. The TRF consists of teacher’s ratings of a child’s academic performance, adaptive characteristics and conduct problems. Teachers were asked to rate the degree of a child’s emotional and behavioral problems during the previous 2 months on a 0–2 scale (0 = not true as far as they know; 1 = somewhat or sometimes true; 2 = very true or often true).

The scale consisted of 118 problem items plus 2 open-ended items (not used here). Total problems scores thus range from 0 to 236. The TRF has been found to have internal consistency in 21 countries with a strong construct validity alpha [34, 36]. According to the Multicultural Supplement to the ASEBA manual [36], Norway is included in Group 3, and the norms and cut-offs were set according to this group; see Table 1.

The Child and Adolescent Psychiatric Assessment (CAPA)

The CAPA is an interviewer-based semi-structured psychiatric interview that collects data on the onset dates, duration, frequency, and intensity of symptoms of a wide range of psychiatric diagnoses according to the DSM-IV [30]. The interview serves as a guide to determine whether a symptom is present at pre-specified levels, and the interviewer is expected to probe until she or he can decide whether the symptom is present. Information concerning the frequency, onset, intensity and duration is obtained. Moreover, functional impairment is captured. The test–retest reliability of the assessment has been shown to be adequate [30]. Interviewers (N = 4) had at least a bachelor’s degree in a relevant field and extensive experience working with children and families. The inter-rater reliability of the rater pairs as estimated by Gwet’s AC1 (and agreement rate) ranged between 0.74 and 1.0, except for substance abuse, which had an AC1 of 0.69. Gwet’s AC1 was calculated in AgreeStat (supplied commercially by Gwet at http://www.agreestsat.com/agreestat.html) [8].

Statistics

Throughout this study, we considered the diagnoses from the CAPA interview as the diagnostic standard. First, we studied the sensitivity and specificity of the CBCL and TRF for each diagnosis of the diagnostic groups. In this context, a CBCL score equal to or above the gender-specific borderline cut-off value in Table 1 was regarded as a positive CBCL, and the same method was applied for the TRF.

Second, we conducted ROC (receiver operating diagnostic curve) analyses. When different cut-off values are used for the CBCL (or TRF) scores, different pairs of sensitivity and specificity values emerge. An ROC curve connects these paired values of specificity and sensitivity. The area under the ROC curve, AUC, is a measure of the ability of the value to discriminate between clinical cases and non-clinical cases. The AUC equals 1 if there is perfect discrimination, and an AUC of 0.5 indicates discrimination that is no better than chance. We regarded an AUC below 0.7 as poor, between 0.7 and 0.8 as acceptable, between 0.8 and 0.9 as excellent, and above 0.9 as outstanding discrimination, as recommended by Hosmer et al. [37]. One interpretation of the AUC is as follows: if one randomly picks a diseased individual and a non-diseased individual, the AUC is the probability that the diseased individual scores higher than the non-diseased individual on the scale. The statistical analyses were conducted using SPSS 22 and Stata 13. A two-sided p value <0.05 indicated statistical significance.

Results

Attrition

Of the 601 available children, 201 refused participation, representing an attrition rate of 33 %. Thus 400 children participated in the study, consisting of 230 girls with a mean age of 16.9 years, SD = 1.2, and 170 boys with a mean age of 16.5 years, SD = 1.5. In total, 86 (of 98 eligible) institutions participated, resulting in a response rate of 88 % [8]. The demographic infomation about the sample attending school is shown in Table 2.
Table 2

Demographic information of the sub-sample attending school (n = 282)

Demographics

n

%

Boys

132

46.8

Girls

150

53.2

Age in years

 12–15

99

35.1

 16–18

178

63.1

 19

5

1.8

Prevalence of diagnostic categories from Child and Adolescent Psychiatric Assessment (CAPA)

 Affective disorders

78

27.7

 Anxiety disorders

77

27.3

 ADHD

90

31.9

 Conduct disorder

37

13.1

Sensitivity and specificity

The comparisons between the CAPA diagnosis and the CBCL scores in the four diagnostic categories showed a sensitivity ranging from 0.60 to 0.82, whereas the TRF showed a sensitivity ranging from 0.39 to 0.54; see Table 3. The primary contacts’ reports consistently showed a higher sensitivity across all diagnostic categories compared with those of the teachers. The opposite pattern was found when comparing specificity. The agreement between the CAPA diagnosis in the four diagnostic categories and the CBCL showed a specificity ranging from 0.30 to 0.74, whereas the TRF showed a specificity ranging from 0.70 to 0.88. The primary contacts’ reports had a consistently lower specificity across all diagnostic categories than those of the teachers, see Table 3.
Table 3

Cross-tables of results from the CAPA interview, set as the “gold standard,” indicating psychiatric diagnosis in the four categories, affective, anxiety, and conduct disorder, and ADHD, compared to reports on the CBCL and TRF corresponding to DSM-oriented scales

CAPA—diagnostic categories

CBCL—corresponding DSM-oriented scales

CAPA—diagnostic categories

TRF—corresponding DSM-oriented scales

Yes

No

Total

Estimate

Yes

No

Total

Estimate

Total sample

 Affective disorder

Yes

61

13

74

Sensitivity

0.82

Affective disorder

Yes

22

27

49

Sensitivity

0.45

No

69

70

139

Specificity

0.50

No

26

66

92

Specificity

0.72

Total

130

83

213

  

Total

48

93

141

  

 Anxiety disorder

Yes

45

29

74

Sensitivity

0.61

Anxiety disorder

Yes

22

28

50

Sensitivity

0.44

No

55

84

139

Specificity

0.60

No

27

64

91

Specificity

0.70

Total

100

113

213

  

Total

49

92

141

  

 Conduct disorder

Yes

25

7

32

Sensitivity

0.78

Conduct disorder

Yes

15

13

28

Sensitivity

0.54

No

127

54

181

Specificity

0.30

No

29

85

114

Specificity

0.75

Total

152

61

213

  

Total

44

98

142

  

 ADHD

Yes

40

27

67

Sensitivity

0.60

ADHD

Yes

23

36

59

Sensitivity

0.39

No

38

108

146

Specificity

0.74

No

10

73

83

Specificity

0.88

Total

78

135

213

  

Total

33

109

142

  

BOYS

 Affective disorder

Yes

14

6

20

Sensitivity

0.70

Affective disorder

Yes

5

9

14

Sensitivity

0.36

No

34

40

74

Specificity

0.54

No

9

44

53

Specificity

0.83

Total

48

46

94

  

Total

14

53

67

  

 Anxiety disorder

Yes

17

9

26

Sensitivity

0.65

Anxiety disorder

Yes

10

8

18

Sensitivity

0.56

No

29

39

68

Specificity

0.57

No

9

40

49

Specificity

0.82

Total

46

48

94

  

Total

19

48

67

  

 Conduct disorder

Yes

15

6

21

Sensitivity

0.71

Conduct disorder

Yes

7

12

19

Sensitivity

0.37

No

51

22

73

Specificity

0.30

No

6

43

49

Specificity

0.88

Total

37

57

94

  

Total

13

55

68

  

 ADHD

Yes

19

12

31

Sensitivity

0.61

ADHD

Yes

9

20

29

Sensitivity

0.31

No

12

51

63

Specificity

0.81

No

2

37

39

Specificity

0.95

Total

31

63

94

  

Total

11

57

68

  

GIRLS

 Affective disorder

Yes

47

7

54

Sensitivity

0.87

Affective disorder

Yes

17

18

35

Sensitivity

0.49

No

35

30

65

Specificity

0.46

No

17

22

39

Specificity

0.56

Total

82

37

119

  

Total

34

40

74

  

 Anxiety disorder

Yes

28

20

48

Sensitivity

0.58

Anxiety disorder

Yes

12

20

32

Sensitivity

0.38

No

26

45

71

Specificity

0.63

No

18

24

42

Specificity

0.57

Total

54

65

119

  

Total

30

44

74

  

 Conduct disorder

Yes

10

1

11

Sensitivity

0.91

Conduct disorder

Yes

8

1

9

Sensitivity

0.89

No

76

32

108

Specificity

0.30

No

23

42

65

Specificity

0.65

Total

86

33

119

  

Total

31

43

74

  

 ADHD

Yes

21

15

36

Sensitivity

0.58

ADHD

Yes

14

16

30

Sensitivity

0.47

No

26

57

83

Specificity

0.69

No

8

36

44

Specificity

0.82

Total

47

72

119

  

Total

22

52

74

  

Sensitivity and specificity are reported for the four diagnostic groups for both CBCL and TRF. The table presents results for the total sample and for boys and girls separately

For boys, as shown in the middle panel of Table 3, the agreement between a CAPA diagnosis in the four diagnostic categories and the primary contacts’ report on the CBCL showed a sensitivity ranging from 0.61 to 0.81. The agreement between a CAPA diagnosis in the four diagnostic categories and the teachers’ report on the TRF showed a sensitivity ranging from 0.31 to 0.56. The CBCL consistently showed a higher sensitivity across all diagnostic categories than the TRF. The opposite pattern was found when comparing specificity. Primary contacts’ reports on the CBCL showed a specificity ranging from 0.54 to 0.81. Teachers’ reports on the TRF showed a specificity ranging from 0.83 to 0.95. The CBCL completed by the primary contacts consistently showed a lower specificity across all diagnostic categories compared with the TRF completed by the teachers.

For girls, as observed in the bottom panel of Table 3, the agreement between a CAPA diagnosis in the four diagnostic categories and the primary contacts’ report on the CBCL showed a sensitivity ranging from 0.59 to 0.96. The agreement between a CAPA diagnosis in the four diagnostic categories and the teachers’ report on the TRF showed a sensitivity ranging from 0.37 to 0.89. The primary contacts’ reports consistently showed a higher sensitivity across all diagnostic categories compared with those of the teachers. The opposite pattern was found when comparing specificity. Primary contacts’ reports on the CBCL showed a specificity ranging from 0.31 to 0.69. Teachers’ reports on the TRF showed a specificity ranging from 0.56 to 0.79. For all diagnostic categories, except for the CAPA Anxiety subscale, the CBCL completed by the primary contacts showed a consistently lower specificity across all diagnostic categories than the TRF completed by the teachers.

ROC

Figure 2 shows the ROC with the corresponding sensitivity and specificity for all possible cut-off values, for each of the diagnostic categories, for the CBCL and the TRF, and separately for boys and girls. In all four diagnostic categories and in both genders, the ROC curves for the CBCL and TRF were quite similar, indicating that the CBCL and TRF scales discriminated between adolescents equally as well. In fact, there were no significant differences between any of the eight pairs of CBCL and TRF comparisons (p values from 0.16 to 0.98). Both primary contacts’ reports on the CBCL and teachers’ reports on the TRF identified ADHD acceptably well in both genders and CD in girls, with AUCs between 0.7 and 0.8. Both primary contacts’ and teachers’ reports poorly detected affective disorders and anxiety in both genders and CD in boys, with AUCs below 0.7. One exception was the CBCL for affective disorders in girls, with an AUC = 0.73.
Fig. 2

ROC curves (receiver operating diagnostic curve) connecting corresponding values of sensitivity and specificity for the CBCL and TRF with the CAPA diagnostic standard. The enlarged symbols (square and triangle) represent the sensitivity and specificity obtained using the defined cut-off values in Table 1

Discussion

This paper focused on how well teachers and primary contacts in RYC institutions identified adolescents’ mental health problems as determined by the CAPA diagnostic interview. We observed that there was a significant gap between the mental health problems diagnosed by the CAPA interviews and the problems reported by the primary contact on the CBCL and by the teachers on the TRF. In general, we observed that the reports from the primary contacts showed a higher sensitivity across all diagnostic categories than those of the teachers. This indicates that when adolescents do have a psychiatric disorder, primary contacts are better at identifying this than teachers. Teachers’ ratings showed a higher specificity across all diagnostic categories than those of primary contacts. This indicates that when adolescents do not have a psychiatric disorder, teachers are better at recognizing this than the primary contacts. Ideally, when using any screening instrument, both sensitivity and specificity should be high, thus ensuring that the right individuals are identified with a diagnosis and avoiding overdiagnosis.

As adolescents living in RYC institutions do not have parental educational support, it is up to the institutional staff to meet this need. It is important to identify factors that facilitate healthy adaptation to school for these adolescents to succeed in school. Improving mental health problems is one of these factors, as mental health issues are an important barrier to learning [24]. Teachers were quite good at identifying adolescents who did not meet the criteria for a diagnosis (specificity). This suggests that teachers have a tendency to view these students as healthy and may underestimate the severity of their problems.

On the other hand, adolescents’ primary contacts may overestimate the prevalence of mental health problems and view individuals as more sick than they are. A disadvantage of this perspective is that adolescents may be treated as sick when they are not; they may thus be given fewer responsibilities and fewer demands instead of being encouraged to push themselves and do their best. Underestimating these adolescents and feeling sorry for them because of their previous challenges and experiences is understandable. However, they, more than others, need people to believe in them, encourage them to work, and believe in their ability to succeed. In addition, caretakers’ attitudes towards school have been reported as being important for children to succeed in school [19]. In general, studies have reported that adolescents in out-of-home care are at a high risk for poor educational outcomes [11, 13]. This is a serious problem, as we also know that education plays a major role in successful adult life [38]. Enhancing the educational performance of young people in RYC requires daily and pervasive educational support and encouragement [39]. Both teachers and care persons should work together towards these goals.

Primary contacts were quite good at identifying adolescents who had a diagnosis (sensitivity), whereas teachers had a tendency to underestimate these problems, which is consistent with results from Larsson and Drugli [27] on school-aged children. Adolescents in RYC are dependent on caretakers and teachers to identify their problems. The present study shows that primary contacts detect adolescents’ mental health problems to a certain degree. However, as very few of the adolescents from our study report receiving help for their problems from mental health clinics (37 %, [8]), the institutions do not seem to refer these adolescents to treatment. If RYC personnel or teachers do not act, these adolescents will not receive the help that they need as few adolescents call on medical or other services themselves if they suffer anxiety or feel depressed. This situation is noteworthy. The availability of psychiatric services for this group is further discussed in another paper from the same study [8].

With CBCL sensitivities ranging from 0.60 to 0.82 across the four diagnostic categories based on primary contacts’ reports and sensitivities of the TRF ranging from 0.39 to 0.54 according to teachers, many mental health problems may go undetected. The fact that approximately 76 % of the adolescents in the present study qualified for at least one psychiatric DSM-IV diagnosis [8] shows that there is a gap between the true prevalence of mental health problems (CAPA diagnosis) and the problems reported on the CBCL and TRF.

Suffering from mental health problems prevents individuals from fully focusing on schoolwork. It is therefore very important for caretakers and teachers who see these adolescents every day to recognize their mental health problems and refer them to treatment. These adolescents may also need adjustments in school to promote optimal learning. This is important because education is reported to provide better prospects for an individual’s future [40].

Sensitivity and specificity measure the performance of the cut-off values used to identify adolescents within a borderline range. The fact that the CBCL had a higher sensitivity than the TRF, whereas the TRF had a higher specificity than the CBCL, could be explained by the higher cut-off points for the TRF compared to the CBCL as illustrated in the ROC curves. The area under the ROC curve is a measure of the ability of the value to discriminate between clinical cases and non-clinical cases. This area was similar for the CBCL and TRF for all diagnostic groups.

The hypothesis that externalizing problems would be easier to detect was confirmed, consistent with previous literature [28]. Both primary contacts and teachers classified externalizing problems fairly well such as ADHD in both genders and CD in girls, with AUCs between 0.7 and 0.8, which is considered acceptable [37]. It is understandable that externalizing problems are easier to detect, as some symptoms manifest themselves as disturbing elements that can easily be observed because they may interrupt regular activities. Externalizing problems are also reported to impair educational attainment [24]. Furthermore, learning and behavior problems are often related [41], and these adolescents may be at risk for developing behavioral issues if help is not introduced early enough. It is worth noting that several adolescents had externalizing problems that were undetected by caretakers and teachers, and these adolescents also need help.

Both teachers and primary contacts struggled more in recognizing internalizing problems. They poorly identified affective disorders and anxiety in both genders and CD in boys, with AUCs of their corresponding scales below 0.7, which is considered poor [37]. One exception was the use of the CBCL to detect affective disorders in girls, with an AUC = 0.73, indicating that primary contacts detected these problems at an acceptable level. Internalizing problems, such as depression and anxiety, are more hidden and could be concealed in subtle behavior. These problems are often disguised as withdrawal and passive behavior, and staff and teachers could easily think that adolescents with these problems are shy or want to be left alone. However, it is an important problem that serious diagnoses such as depression and anxiety are so modestly detected by caretakers and teachers. Who else is in a position to identify these problems among these adolescents?

Strengths and limitations

This study is part of a larger study on adolescents in RYC, was based on three different sources of information, and used standardized international measurements. The study is the first one to look at teachers’ and primary care providers’ ability to recognize mental health struggles in adolescents in RYC. We did not have the permission to include the adolescents’ parents as informants, thereby limiting the knowledge about early development and family functioning before placement in RYC. The study has a fairly high participation rate of 67 %. However, limitations could be sample bias as many institutions refused to participate, also many adolescents refused participation or were unable to complete the measures.

Conclusion

In the present study, there was a mismatch between the DSM-IV diagnoses among adolescents in RYC and the problems reported by their primary contacts and teachers. Primary contacts’ reports showed a higher sensitivity than those of teachers, whereas the teachers’ TRF scores showed a higher specificity in detecting mental health problems in adolescents in RYC than the primary contacts’ CBCL reports. Both primary contacts and teachers recognized externalizing problems such as ADHD fairly well in both genders as well as CD in girls. Both teachers and primary contacts, however, had more problems detecting internalizing problems. It would be important to create interventions with primary contacts, teachers, and youth to educate and raise awareness about emotional problems. Peer identification is another strategy which also has shown effectiveness amongst youth and might be a resource worth considering in the future. Further studies should investigate this topic more carefully to ensure that these vulnerable adolescents receive sufficient help for their problems. CWS should revise their intake procedures so that possible problems are detected early and that the necessary treatment is introduced.

Abbreviations

CWS: 

Child Welfare System

RYC: 

residential youth care

CAPA: 

The Child and Adolescent Psychiatric Assessment diagnostic interviews

CBCL: 

The Child Behavior Checklist

TRF: 

Teacher’s Report Form

CD: 

conduct disorder

ADHD: 

Attention Deficit Hyperactivity Disorder

AS: 

Asperger’s Syndrome

ROC: 

receiver operating diagnostic curve

AUC: 

the area under the ROC curve

Declarations

Authors’ contributions

All authors participated in the design of the study draft. AMU drafted the intro, methods and discussion sections. SL performed the statistical analyses, and NSK and SL drafted the result section of the manuscript. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Ethics approval and consent to participate

Participants were recruited using procedures approved by the Norwegian Regional Committee for Medical and Health Research Ethics, and written consent was obtained. For persons under 16, informed consent from the primary caregiver was also provided.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Faculty of Medicine, Regional Centre for Child and Youth Mental Health and Child Welfare—Central Norway, Norwegian University of Science and Technology (NTNU)

References

  1. Horwitz AV, Widom CS, McLaughlin J, White HR. The impact of childhood abuse and neglect on adult mental health: a prospective study. J Health Soc Behav. 2001;42(2):184–201.View ArticlePubMedGoogle Scholar
  2. Kim J, Cicchetti D. A longitudinal study of child maltreatment, mother-child relationship quality and maladjustment: the role of self-esteem and social competence. J Abnorm Child Psychol. 2004;32(4):341–54.View ArticlePubMedGoogle Scholar
  3. Andersson G. Family relations, adjustment and well-being in a longitudinal study of children in care. Child Fam Soc Work. 2005;10(1):43–56.View ArticleGoogle Scholar
  4. Vinnerljung B, Hjern A, Lindblad F. Suicide attempts and severe psychiatric morbidity among former child welfare clients–a national cohort study. J Child Psychol Psychiatry. 2006;47(7):723–33.View ArticlePubMedGoogle Scholar
  5. Harpin S, Kenyon DB, Kools S, Bearinger LH, Ireland M. Correlates of emotional distress in out-of-home youth. J Child Adolesc Psychiatr Nurs. 2013;26(2):110–8.View ArticlePubMedGoogle Scholar
  6. Ford T, Vostanis P, Meltzer H, Goodman R. Psychiatric disorder among British children looked after by local authorities: comparison with children living in private households. Br J Psychiatry. 2007;190(4):319–25.View ArticlePubMedGoogle Scholar
  7. Lawrence CR, Carlson EA, Egeland B. The impact of foster care on development. Dev Psychopathol. 2006;18(01):57–76.View ArticlePubMedGoogle Scholar
  8. Jozefiak T, Kayed NS, Rimehaug T, Wormdal AK, Brubakk AM, Wichstrøm L. Prevalence and comorbidity of mental disorders among adolescents living in residential youth care. Eur Child Adolesc Psychiatry. 2016;25(1):33–47.View ArticlePubMedGoogle Scholar
  9. Levitt JM. Identification of mental health service need among youth in child welfare. Child Welf. 2009;88(1):27.Google Scholar
  10. Shin SH. Building evidence to promote educational competence of youth in foster care. Child Welf. 2003;82(5):615–32.Google Scholar
  11. Berridge D. Theory and explanation in child welfare: education and looked-after children. Child Fam Soc Work. 2007;12(1):1–10.View ArticleGoogle Scholar
  12. Clausen S-E, Kristofersen LB. Barnevernsklienter i Norge. NOVA rapport. 2008;3:2008.Google Scholar
  13. Vinnerljung B, Öman M, Gunnarson T. Educational attainments of former child welfare clients—a Swedish national cohort study. Int J Soc Welf. 2005;14(4):265–76.View ArticleGoogle Scholar
  14. Davis RJ. College access, financial aid, and college success for undergraduates from foster care. National Association of Student Financial Aid Administrators. 2006.Google Scholar
  15. Shin SH. Developmental outcomes of vulnerable youth in the child welfare system. J Hum Behav Soc Environ. 2004;9(1–2):39–56.View ArticleGoogle Scholar
  16. Trout AL, Hagaman J, Casey K, Reid R, Epstein MH. The academic status of children and youth in out-of-home care: a review of the literature. Child Youth Serv Rev. 2008;30(9):979–94.View ArticleGoogle Scholar
  17. Ferguson HB, Wolkow K. Educating children and youth in care: a review of barriers to school progress and strategies for change. Child Youth Serv Rev. 2012;34(6):1143–9.View ArticleGoogle Scholar
  18. Jaffee SR, Gallop R. Social, emotional, and academic competence among children who have had contact with child protective services: prevalence and stability estimates. J Am Acad Child Adolesc Psychiatry. 2007;46(6):757–65.View ArticlePubMedPubMed CentralGoogle Scholar
  19. Shonk SM, Cicchetti D. Maltreatment, competency deficits, and risk for academic and behavioral maladjustment. Dev Psychol. 2001;37(1):3.View ArticlePubMedGoogle Scholar
  20. Blyth E, Milner J. Exclusion from school: a first step in exclusion from society? Child Soc. 1993;7(3):255–68.View ArticleGoogle Scholar
  21. McCrystal P, Percy A, Higgins K. Exclusion and marginalisation in adolescence: the experience of school exclusion on drug use and antisocial behaviour. J Youth Stud. 2007;10(1):35–54.View ArticleGoogle Scholar
  22. Uiters E, Maurits E, Droomers M, Zwaanswijk M, Verheij RA, van der Lucht F. The association between adolescents’ health and disparities in school career: a longitudinal cohort study. BMC Public Health. 2014;14(1):1.View ArticleGoogle Scholar
  23. Kessler RC, Foster CL, Saunders WB, Stang PE. Social consequences of psychiatric disorders, I: educational attainment. Am J Psychiatry. 1995;152(7):1026–32.View ArticlePubMedGoogle Scholar
  24. Sagatun Å, Heyerdahl S, Wentzel-Larsen T, Lien L. Mental health problems in the 10th grade and non-completion of upper secondary school: the mediating role of grades in a population-based longitudinal study. BMC Public Health. 2014;14(1):1.View ArticleGoogle Scholar
  25. Chong SA, Subramaniam M, Lee I-M, Pek E, Cheok C, Verma S, et al. Academic attainment: a predictor of psychiatric disorders? Soc Psychiatry Psychiatr Epidemiol. 2009;44(11):999–1004.View ArticlePubMedGoogle Scholar
  26. Emerson J, Lovitt T. The educational plight of foster children in schools and what can be done about it. Remedial Spec Educ. 2003;24(4):199–203.View ArticleGoogle Scholar
  27. Larsson B, Drugli MB. School competence and emotional/behavioral problems among Norwegian school children as rated by teachers on the Teacher Report Form. Scand J Psychol. 2011;52(6):553–9.View ArticlePubMedGoogle Scholar
  28. Dwyer SB, Nicholson JM, Battistutta D. Parent and teacher identification of children at risk of developing internalizing or externalizing mental health problems: a comparison of screening methods. Prev Sci. 2006;7(4):343–57.View ArticlePubMedGoogle Scholar
  29. Gearing RE, Schwalbe CS, MacKenzie MJ, Brewer KB, Ibrahim RW. Assessment of adolescent mental health and behavioral problems in institutional care: discrepancies between staff-reported CBCL scores and adolescent-reported YSR scores. Adm Policy Ment Health Men Health Serv Res. 2015;42(3):279–87.View ArticleGoogle Scholar
  30. Angold A, Costello EJ. The child and adolescent psychiatric assessment (CAPA). J Am Acad Child Adolesc Psychiatry. 2000;39(1):39–48.View ArticlePubMedGoogle Scholar
  31. Achenbach TM, Rescorla L. Manual for the ASEBA school-age forms & profiles: child behavior checklist for ages 6–18, teacher’s report form, youth self-report: an integrated system of multi-informant assessment. Burlington, Vt: ASEBA; 2001.Google Scholar
  32. Achenbach TM, Rescorla L. ASEBA school-age forms and profiles. Aseba Burlington; 2001.Google Scholar
  33. Achenbach TM, Dumenci L, Rescorla LA. DSM-oriented and empirically based approaches to constructing scales from the same item pools. J Clin Child Adolesc Psychol. 2003;32(3):328–40.View ArticlePubMedGoogle Scholar
  34. Rescorla LA, Bochicchio L, Achenbach TM, Ivanova MY, Almqvist F, Begovac I, et al. Parent–teacher agreement on children’s problems in 21 societies. J Clin Child Adolesc Psychol. 2014;43(4):627–42.View ArticlePubMedGoogle Scholar
  35. Nøvik T. Validity of the child behaviour checklist in a Norwegian sample. Eur Child Adolesc Psychiatry. 1999;8(4):247–54.View ArticlePubMedGoogle Scholar
  36. Achenbach TM, Rescorla LA. Multicultural supplement to the manual for the ASEBA school-age forms & profiles. Burlington VT: University of Vermont Research Center for Children, Youth, & Families; 2007.Google Scholar
  37. Hosmer DW Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. New York: Wiley; 2013.View ArticleGoogle Scholar
  38. McCrystal P, Percy A, Higgins K. Substance use behaviors of young people with a moderate learning disability: a longitudinal analysis. Am J Drug Alcohol Abuse. 2007;33(1):155–61.View ArticlePubMedGoogle Scholar
  39. Gharabaghi K. A culture of education: enhancing school performance of youth living in residential group care in Ontario. Child Welf. 2011;90(1):75.Google Scholar
  40. Kristofersen LB. Barnevernbarnas helse. 1993;35:115–123.Google Scholar
  41. Perry KE, Weinstein RS. The social context of early schooling and children’s school adjustment. Educ Psychol. 1998;33(4):177–94.View ArticleGoogle Scholar

Copyright

© The Author(s) 2016

Advertisement