Brief report: large individual variation in outcomes of autistic children receiving low-intensity behavioral interventions in community settings
Child and Adolescent Psychiatry and Mental Health volume 9, Article number: 6 (2015)
Despite widespread awareness of the necessity of early intervention for children with autism spectrum disorders (ASDs), evidence is still limited, in part, due to the complex nature of ASDs. This exploratory study aimed to examine the change across time in young children with autism and their mothers, who received less intensive early interventions with and without applied behavior analysis (ABA) methods in community settings in Japan.
Eighteen children with autism (mean age: 45.7 months; range: 28–64 months) received ABA-based treatment (a median of 3.5 hours per week; an interquartile range of 2–5.6 hours per week) and/or eclectic treatment-as-usual (TAU) (a median of 3.1 hours per week; an interquartile range of 2–5.6 hours per week). Children’s outcomes were the severity of autistic symptoms, cognitive functioning, internalizing and externalizing behavior after 6 months (a median of 192 days; an interquartile range of 178–206 days). In addition, maternal parenting stress at 6-month follow-up, and maternal depression at 1.5-year follow-up (a median of 512 days; an interquartile range of 358–545 days) were also examined.
Large individual variations were observed for a broad range of children’s and mothers’ outcomes. Neither ABA nor TAU hours per week were significantly associated with an improvement in core autistic symptoms. A significant improvement was observed only for internalizing problems, irrespective of the type, intensity or monthly cost of treatment received. Higher ABA cost per month (a median of 1,188 USD; an interquartile range of 538–1,888 USD) was associated with less improvement in language-social DQ (a median of 9; an interquartile range of −6.75-23.75).
To determine an optimal program for each child with ASD in areas with poor ASD resources, further controlled studies are needed that assess a broad range of predictive and outcome variables focusing on both individual characteristics and treatment components.
Autism spectrum disorders (ASDs) are persistent disabling neurodevelopmental disorders that are clinically evident from early in life. Accordingly, many countries have given greater public attention to ASDs and allocated more public funds to implement and develop community services or promote research in this field. Among them, early identification and subsequent intervention for ASDs are considered key issues. A recent systematic review of early intensive intervention concluded that Lovaas-based approaches, early intensive behavioral intervention variants and the early intensive comprehensive approach (the Early Start Denver Model) resulted in some improvement in cognitive, language, and adaptive functioning in some young children with ASD compared with broadly defined eclectic treatments .
The growing body of evidence on early interventions for children with ASD suggests that there is great variability in children’s response to treatment [1-6]. However, the responder’s characteristics for each treatment have not been well identified, which makes it difficult for clinicians to recommend any specific form of intervention as the best option for an individual child with ASD.
In Japan, existing intervention services are generally insufficient in terms of their quantity and quality to meet the identified needs of young children with ASDs and their families. To complement existing services, various ABA-based techniques combined with parental training are provided at a limited number of universities and private agencies in metropolitan areas, although of a lower intensity. In a recent study, Hiraiwa  retrospectively examined the severity of autism in 60 young Japanese children with autism and found that there was a significant improvement in children receiving low intensity one-to-one treatment using various methods based on the principle of ABA (≥7 hours per week but less than the recommended intensity when compared with those receiving treatment-as-usual (TAU)); here the ABA methods included discrete trial teaching (DTT), verbal behavior (VB), and PRT provided by therapists and/or parents. Apart from case studies, Hiraiwa’s study  has been the only study to examine children receiving ABA in Japan. However, it used only one, less sensitive child measure, and family functioning that might have had an influence on children’s progress [8,9] was not measured.
The aim of this study was to thus explore individual outcome variations across time in young autistic children and their mothers, who received less intensive early interventions with and without ABA methods in community settings in Japan. The outcomes were assessed in terms of both child and family functioning using standardized instruments.
Seventeen children were recruited through notices posted in a specialized pediatric clinic located in a suburb of Tokyo, where one of the authors (M.H.) prescribes ABA therapy for children diagnosed with autism. In addition, three research volunteer families were contacted because they lived near the National Center of Neurology and Psychiatry (NCNP). All 20 children met the following criteria: (1) a diagnosis of autistic disorder according to DSM-IV-TR criteria corroborated by the Japanese versions of the Autism Diagnostic Interview-Revised (ADI-R)  and the Autism Diagnostic Observation Schedule (ADOS)  evaluated by an experienced child psychiatrist or psychologist with a research license; (2) an absence of medical conditions or obvious motor delay; (3) a chronological age below 7 years; (4) entry into an ABA and/or TAU program at two to five years of age. Of the 20 children, 18 (14 boys) participated in both intake and a 6-month follow-up assessment (Figure 1).
Participants’ characteristics (age, gender, scores at T1) are shown in Table 1. All participants were living with both parents. The number of siblings was similar to the national average . Parental educational levels were higher than the national average . Family income varied widely but the peak and mean shifted higher than those of the national average . The percentage of women who were full-time housewives was higher than the national average .
The study protocol was approved by the NCNP Ethics Committee. Written informed consent was obtained from the parents of each participating child.
Sixteen participants received ABA-based treatments, with 11 also receiving supplemental TAU (Table 1). Five children received only ABA and two received only TAU. In addition to DTT, various ABA techniques such as VB, PRT, either alone or in combination, were provided in a one-to-one setting by highly trained therapists supervised by the program consultant. Neither therapists nor supervisors were involved in this study. Information regarding the content, hours per week, and the monthly cost of received treatment was obtained from mother-completed questionnaires at T1, T2, and T3 (Table 1). Our participants received a near average to above-average intensity of ABA as a group, and paid monthly fees to the agency/agencies that ranged from approximately US $175 to $5,875 (based on parental information, according to the currency exchange rate at the time of this study). In contrast, TAU was either free of charge or the monthly fees that were paid were less than $125. Hours of ABA/TAU per week or the cost of received treatment per month were not significantly associated with any of the child and family characteristics (child’s age, number of siblings, parental education, income). Although all parents were taught the basics of ABA and about various behavioral techniques to augment the effect of the intervention, additional ABA therapy carried out by parents themselves at home was not examined in this study.
TAU consisting of one-to-one or group programs was provided by local community-based day nurseries or specialized private preschools. The programs were organized and provided by a team that included a psychologist, nursery school teacher, community nurse, and child care staff. The frequency and hours per week of TAU provided by the community were limited across the study areas (Table 1). The TAU content was diverse with some of it including the use of picture cards or schedules, sensory integration therapy, or group-based social skills training.
Regarding child measures, although testers (licensed clinical psychologists with a master’s degree or doctoral degree) were blind to the intensity of the child’s treatment, sometimes blindness to the type of treatment was compromised unintentionally. Autistic symptoms were assessed using the Japanese version of ADOS . Since the use of Calibrated Severity Scores (CSS) as an indicator of autism severity has been shown to be more valid than the ADOS raw total score [16,17], CSS were calculated from raw ADOS scores [16,18].
A child’s development was assessed using the Kyoto Scale of Psychological Development Test (KSPD) , which is widely used in Japanese clinical settings for young and/or developmentally delayed children and comparable to the Bayley Scales of Infant Development second edition (BSID-II)  (KSPD cognitive-adaptive (C-A) DQ and the BSID-II Cognitive facet, language-social (L-S) DQ and the Language facet, postural-motor (P-M) DQ and the Motor facet) . Total DQs assessed by the KSPD are considered comparable to IQ scores for children with autism .
Children’s internalizing and externalizing behavior problems were measured using the Japanese version of the parent-rated Child Behavioral Checklist (CBCL) . T-scores were used as outcome measures.
Maternal mental health was assessed using the Parenting Stress Index (PSI) and a two-question case-finding instrument (TQI). The PSI, a self-report 120-item questionnaire comprising Child and Parent domains, assesses dysfunctional parenting in parents of preschool children . The TQI consisting of two questions is a depression screening tool originally included in the Primary Care Evaluation of Mental Disorders Procedure (PRIME-MD) . The utility of the number of yes answers has been previously demonstrated for Japanese adults .
Eighteen participants completed both the T1 assessment (demographic information, ADI-R, ADOS, KSPD, CBCL, and PSI) and 6-month follow-up (T2) assessment (ADOS, KSPD, CBCL, and PSI). At T3 approximately 1 year after T2, the TQI and questionnaire about the received treatment were mailed to mothers, with 16 (88.9%) mothers completing and returning them (Figure 1). Time intervals T1-T2 and T2–T3 had a median of 192 days and an interquartile range of 28 days, and a median of 354 days and an interquartile range of 147 days, respectively. Performance-based tests were administered at the NCNP.
Wilcoxon’s paired-sample test was used to compare outcome measures at T1 and T2. Since the non-normality of treatment variables was confirmed using the Shapiro-Wilk test, correlations between the predictor variables including treatment variables and child/mother measures at T1, and score changes between T1 and T2 were examined by calculating Spearman’s correlation coefficients. A Mann–Whitney test was used to compare predictor variables between participants whose mothers answered yes to one or both depression items at T3 and those whose mothers answered no to both questions. A p-value < .05 was considered statistically significant. The statistical analysis was performed using SPSS version 18.0 (SPSS Inc., Chicago, USA).
Table 2 provides details of each participant’s measures at T1, T2 and T3. As shown in Table 1 and Table 2, levels of children’s cognitive functioning, behavior problems and their mothers’ parenting stress at T1, treatment hours per week, treatment cost per month, and T1-T2 change in child and mother measures varied greatly in this sample. Table 3 shows the correlations between the predictor variables and T1-T2 improvement for 18 pairs of children with autism and their mothers.
Overall, a significant improvement was observed for internalizing t-scores (p = .001) only. The L-S DQ changes approached near significance (p = .08) (Table 2).
Change in children’s behaviors
As shown in Table 3, ABA hours per week were significantly correlated with an improvement in P-M DQ only (p = .036). TAU hours per week were not associated with any change. The monthly fee paid for ABA was significantly negatively correlated with an improvement in L-S DQ (p = .027), although it was positively correlated with an improvement in ADOS CSS, which approached statistical significance (p = .064). The improvement in the child measures listed in Table 3 were not significantly associated with clinical characteristics assessed at T1, although DQs at T1 were negatively correlated with the changes in C-A and L-S DQs (p = .043, .053, respectively). The changes in each child measure, ADOS CSS, KSPD DQ, and CBCL, were not significantly correlated with each other, whereas among the KSPD DQs the changes in C-A DQ were correlated with those in L-S DQ and P-M DQ (r s = .51, .45, p = .031, .064, respectively).
Change in mother’s parenting stress
Neither ABA nor TAU hours per week were significantly associated with an improvement in the PSI Child or Parent domain scores. ABA plus TAU hours per week were associated with a reduction in PSI Parent scores, which approached statistical significance (p = .075) (Table 3). The monthly cost of ABA was not significantly correlated with change in either PSI Child or PSI Parent domain scores. A reduction in the PSI Child domain scores was significantly correlated with an improvement in children’s C-A DQ (r s = .67, p = .002) and CBCL internalizing scores (r s = .69, p = .001), while a reduction in the PSI Parent domain scores was significantly correlated with children’s CBCL internalizing scores (r s = .52, p = .026).
Mothers’ depression items at T3
The frequency distribution of TQI positive items (n = 16) was similar to that in a recent Japanese adult sample . Participants whose mothers answered yes to either one or both depression items (n = 9) did not significantly differ in either ABA or TAU hours per week, the monthly ABA cost between T1 and T3 (not shown), or family characteristics when compared with the other children (n = 7), but had a significantly increased ADOS CSS (p = .046) and a lower total DQ (p = .071) at T1.
We prospectively monitored the developmental progress of 18 children diagnosed as having autistic disorder who received various combinations of ABA (median 3.5 hours per week, range 0–12 hours per week) and/or TAU (median 0.5 hours per week, range 0–21.3 hours per week), and assessed their autistic symptoms, cognitive functioning, internalizing and externalizing problems at intake and 6-month follow-up, and their mothers’ mental health at intake, 6-month follow-up, and 1.5-year follow-up. Large individual variations in outcomes were observed in this study, which is consistent with the findings from previous research [2,6,27]. A significant improvement at the group level was observed only for internalizing problems, irrespective of the type and intensity of received treatments or ABA cost per month. Changes in children’s autistic symptoms, cognitive or language functioning and mothers’ parenting stress were not associated with either ABA or TAU hours per week at an individual level.
However, it was impossible to tease out the effect of ABA from that of TAU in this study. Recent studies have reported that young children with ASDs who received 2 years of lower intensity one-to-one behavioral treatment (4–15 hours per week, where the average hours per week were much higher than those in this study) showed significant progress in a broad range of parameters compared to children who had received TAU treatment [28,29]. The question of whether the intensity or duration of low intensity ABA-based treatment of the type delivered in this study is associated with an improvement in child and family functioning should be examined in future prospective controlled studies.
Regarding the outcome predictors in children, initial IQ has been identified as a strong predictor in 4 of the 11 studies of early intensive behavioral interventions  and in a naturalistic study : that was not the case in this study (although our findings may have been affected by the small sample size). As ASD treatment predictors as well as goals can vary according to the socio-cultural context (similar to general mental health issues ), future intervention studies should include more diverse race/ethnic/cultural factors to better understand their effects .
This study has a number of methodological limitations. First, the sample size was small. Second, we obtained information on ABA and TAU treatment only through parents. We therefore lacked information on its specific form or quality. The fidelity of ABA programs delivered was not monitored. And we did not systematically evaluate parental involvement in home-based ABA therapy. Third, this study did not have a control group receiving a different type of treatment. Fourth, the blindness of the assessment was not perfect. The strengths of this study include a uniform assessment protocol with well-standardized measures of child diagnostic and developmental status as well as parental mental health [1,2].
As emphasized by Howlin et al. , treatment should not demand extensive sacrifice in terms of time, money, or any other aspect of family life, instead it should benefit all involved. Although our preliminary results should be interpreted with caution, they suggest that in countries such as Japan with poor ASD resources, we need to focus on individual characteristics and to think about what components should comprise an optimal program for the child with autism.
Applied behavior analysis
Two-question case-finding instrument
Warren Z, McPheeters ML, Sathe N, Foss-Feig JH, Glasser A, Veenstra-Vanderweele J. A systematic review of early intensive intervention for autism spectrum disorders. Pediatrics. 2011;127:e1303–11.
Howlin P, Magiati I, Charman T. Systematic review of early intensive behavioral interventions for children with autism. Am J Intellect. 2009;114:23–41.
Camarata S. Early identification and early intervention in autism spectrum disorders: accurate and effective? Int J Speech Lang Pathol. 2014;16:1–10.
Eaves LC, HH H. The very early identification of autism: outcome to age 4 1/2-5. J Autism Dev Disord. 2004;34:367–78.
Magiati I, Charman T, Howlin P. A two-year prospective follow-up study of community-based early intensive behavioral intervention and specialist nursery provision for children with autism spectrum disorders. J Child Psychol Psychiatry. 2007;48:803–12.
Fernell E, Hedvall Å, Westerlund J, Carlsson LH, Eriksson M, Olsson MB, et al. Early intervention in 208 Swedish preschoolers with autism spectrum disorder. A prospective naturalistic study. Res Dev Disabil. 2011;32:2092–101.
Hiraiwa M. An evaluation of ABA therapy for children having autism using PARS. Shonika Shinryo. 2012;75:159–66 (in Japanese).
Robbins FR, Dunlap G, Plienis AJ. Family characteristics, family training, and the progress of young children with autism. J Early Intervention. 1991;15:173–84.
Osborne LA, McHugh L, Sauders J, Reed P. Parenting stress reduces the effectiveness of early teaching interventions for autism spectrum disorders. J Autism Dev Disord. 2008;38:1092–103.
Tsuchiya KJ, Matsumoto K, Yagi A, Inada N, Kuroda M, Inokuchi E, et al. Reliability and validity of autism diagnostic interview-revised, Japanese version. J Autism Dev Disord. 2013;43:643–62.
Kuroda M, Inada N, Yukihiro R, Uchiyama T, Hirose K, Uno Y, et al. Autism Diagnosis Observation Schedule (ADOS-G): Reliability and validity of the Japanese version of ADOS-G, module 1–4. In: Uchiyama T, editor. Annual report of research supported by health and labour sciences research grants. Fukushima: Fukushima University; 2013. p. 31–8 (in Japanese).
National Institute of Population and Social Security Research: Marriage process and fertility of Japanese married couples-Report on the fourteenth Japanese National Fertility Survey in 2010 volume I [http://www.ipss.go.jp/syoushika/bunken/data/pdf/207616.pdf] (in Japanese).
United Nations Development Programe-Human Development Reports: 2014 Human Development Statistical Tables [http://hdr.undp.org/sites/default/files/hdr14-report-en-1.pdf].
Ministry of Health, Labour and Welfare: Summary Report of Comprehensive Survey of Living Conditions 2011 [http://www.mhlw.go.jp/english/database/db-hss/dl/report_gaikyo_2011.pdf].
The Japanese Society of Child Health: Heisei 12 nenndo Yoji kenkod Chosahoukokusho [http://www.jschild.or.jp/book/report_2000.html#one] (in Japanese).
Gotham K, Pickles A, Lord C. Standardizing ADOS scores for a measure of severity in autism spectrum disorders. J Autism Dev Disord. 2009;39:693–705.
Schumwzy S, Farmer C, Thurm A, Joseph L, Black D, Golden C. The ADOS calibrated severity score: Relationship to phenotypic variable and stability over time. Autism Res. 2012;5:267–76.
Gotham K, Risi S, Dawson G, Tager-Flussberg H, Joseph R, Carter A, et al. A replication of the Autism Diagnostic Observation Schedule (ADOS) revised algorithms. J Am Acad Child Adolesc Psychiatry. 2008;47:642–51.
Society for the Kyoto Scale of Psychological Development Test. The Kyoto scale of psychological development test 2001. Kyoto: Nakanishiya Shuppan; 2008 (in Japanese).
Bayley N. Bayley scales of infant development second edition. San Antonio, Texas: The Psychological Corporation; 1993.
Tatsuta N, Suzuki K, Sugawara T, Nakai K, Hosokawa T, Satoh H. Comparison of Kyoto scale of psychological development and bayley scales of infant development second edition among Japanese infants. J Spec Educ Res. 2013;2:17–24.
Koyama T, Osada H, Tsujii H, Kurita H. Utility of the Kyoto scale of psychological development in cognitive assessment of children with pervasive developmental disorders. Psychiatry Clin Neurosci. 2009;63:241–3.
Itani T, Kanbayashi Y, Nakata Y, Kita M, Fujii H, Kuramoto H, et al. Development of child behavior checklist/4-18 Japanese version. Shoni no Seishin to Shinkei (Psychiatria et Neurologia Paediatrica Japonica). 2001;41:243–52 (in Japanese).
Kanematsu Y, Araki A, Narama M, Shirahata N, Maru M, Arayashiki R. PSI parenting STRESS index manual. Tokyo: Shadanhojin Koyomondai Kenkyukai; 2006.
Spitzer RL, Williams JB, Kroenke K, Linzer M, deGruy FV, Hahn SR, et al. Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study. JAMA. 1994;272:1749–56.
Adachi Y, Aleksic B, Nobata R, Suzuki T, Yoshida K, Ono Y, et al. Combination use of beck depression inventory and two-question case-finding instrument as a screening tool for depression in the workplace. BMJ Open. 2012;2:e000596.
Darrow C, Pry R, Pernon E, Michelon C, Aussilloux C, Baghdadli A. Outcome of young children with autism-does the amount of intervention influence developmental trajectories? Autism. 2010;14:1362–3613.
Eldevik S, Eikeseth S, Jahr E, Smith T. Effects of low-intensity behavioral treatment for children with autism and mental retardation. J Autism Dev Disord. 2006;36:211–24.
Peters-Scheffer N, Didden R, Mulders M, Korzillius H. Effectiveness of low intensity behavioral treatment for children with autism spectrum disorder and intellectual disability. Res Autism Spectr Disord. 2013;7:1012–25.
Saxena S, Funk M, Chisholm D. World health assembly adopts comprehensive mental health action plan 2013–2020. Lancet. 2013;381:1970–1.
Wong C, Odom SL, Hume K, Cox AW, Fettig A, Kucharczyk S, Brock ME, Plavnick JB, Fleury VP, Schultz TR (2013): Evidence-based practices for children, youth, and young ddults with Autism Spectrum Disorder. Chapel Hill: The University of North Carolina, Frank Porter Graham Child Development Institute, Autism Evidence-Based Practice Review Group. http://autismpdc.fpg.unc.edu/sites/autismpdc.fpg.unc.edu/files/2014-EBP-Report.pdf.
This study was supported by research grants from the Ministry of Health, Labor and Welfare of Japan to Y. Kamio (H20-KOKORO-004 and ID11103316). The funding source had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. We thank Andrew Stickley, and ThinkSCIENCE K.K., Tokyo, Japan for language assistance. We would like to thank Yoshiyuki Tachibana, Naoko Inada, Eriko Hiramoto, and Yuka Sakurai for data collection. Special thanks are due to Ryoji Yukihiro for statistical advice.
The authors declare that they have no competing interests.
YK designed the study, conducted the statistical analysis and drafted the manuscript. HH made substantive contributions to the conceptualization of the manuscript, supported the literature review, and participated in the statistical analysis. AM participated in data collection, reviewed all the data and helped to draft the manuscript. MH conceived of the study, and participated in its design, recruitment of participants, and helped to draft the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Kamio, Y., Haraguchi, H., Miyake, A. et al. Brief report: large individual variation in outcomes of autistic children receiving low-intensity behavioral interventions in community settings. Child Adolesc Psychiatry Ment Health 9, 6 (2015). https://doi.org/10.1186/s13034-015-0039-6