Abstract

ObjectivesEvaluate psychometric properties of the Pediatric Insomnia Severity Index (PISI), a brief measure of insomnia severity. Methods Clinically referred youth (n = 462; 283 males, 179 females, mean age = 7.28 ± 2.05 years) and their caregiver(s) completed sleep evaluation including the PISI, Children's Sleep Habits Questionnaire, and sleep disorders inventory for students. Tests of reliability and validity and confirmatory factor analysis (CFA) were conducted to assess PISI psychometric properties. Exploratory analyses were conducted to examine insomnia severity by insomnia diagnosis. Results Measures of internal consistency for the PISI factor scores varied. CFA indicated that a two-factor model had optimal fit relative to a single-factor solution. Overall, convergent and discriminant validity of PISI factors were supported. Insomnia severity varied by diagnosis. Conclusions Findings provide preliminary support for the reliability and validity of the PISI within a large pediatric sample and for its clinical utility as a brief measure of insomnia severity.

Introduction

Pediatric insomnia is common, tends to persist without treatment, and has considerable negative consequences (Byars & Simon, 2014; Byars, Yolton, Rausch, Lanphear, & Beebe, 2012; Pollock, 1994; Sadeh, Gruber, & Raviv, 2002). Pediatric sleep difficulties, including insomnia, are associated with academic and cognitive difficulties, internalizing and externalizing behavior problems, and risk for obesity (Chen, Beydoun, & Wang, 2008; Reid, Hong, & Wade, 2009; Touchette et al., 2007). The myriad of negative consequences associated with untreated pediatric sleep disorders has spurred efforts to improve assessment and treatment (Burnham, Goodlin-Jones, Gaylor, & Anders, 2002; Lozoff, Wolf, & Davis, 1985; Meltzer, Johnson, Crosette, Ramos, & Mindell, 2010). Empirically supported treatments for pediatric insomnia are available and efficacious (Byars & Simon, 2014; Meltzer & Mindell, 2014).

The hallmark symptoms of pediatric insomnia are difficulty falling asleep (i.e., sleep-onset problems) and/or difficulty staying asleep (i.e., sleep-maintenance problems) (Meltzer, 2010). Diagnostic nosology has specified clinical subtypes for primary insomnia that include etiologies more common in young children (e.g., behavioral insomnia of childhood) as well as older children and adolescents (e.g., psychophysiological insomnia) (American Academy of Sleep Medicine, 2005a). Behavioral insomnia of childhood is characterized by sleep-onset or -maintenance problems resulting from learned sleep associations and/or bedtime resistance/limit-setting problems (American Academy of Sleep Medicine, 2005a American Academy of Sleep Medicine). Psychophysiological insomnia is characterized by sleep disruption associated with heightened arousal and problematic learned sleep-preventing associations (American Academy of Sleep Medicine, 2005a).

Recent revision to the International Classification of Sleep Disorders diagnostic criteria for insomnia included consolidation of all insomnia diagnoses under a single chronic insomnia disorder, but sleep-onset and sleep-maintenance problems included in the general diagnostic criteria for insomnia have remained essentially unchanged since first publication of the International Classification of Sleep Disorders (ICSD; American Academy of Sleep Medicine, 2014; American Sleep Disorders Association Diagnostic Classification Steering Committee, 1990; Sateia, 2014). To date, the bulk of pediatric insomnia outcome research has demonstrated effectiveness of behavioral intervention for treating both sleep-onset and sleep-maintenance problems (Meltzer & Mindell, 2014; Morgenthaler et al., 2006). Thus, convergence of empirical evidence and clinical experience suggests that evidence-based assessment of pediatric insomnia should focus on sleep-onset and sleep-maintenance difficulties.

Despite advances in pediatric behavioral sleep medicine, sleep problems in youth often are not identified in primary care settings (Blunden et al., 2004; Meltzer et al., 2010). Lack of routine assessment of pediatric sleep disturbances may in part be because of unavailability of brief, easily administered/interpreted, and clinically valid tools for measuring insomnia severity in youth. Polysomnography is the “gold standard” for assessment of medically based sleep disorders (e.g., obstructive sleep apnea [OSA]), but does not directly measure insomnia and is not recommended as an assessment tool for behavioral sleep problems (Reite, Buysse, Reynolds, & Mendelson, 1995; Standards of Practice Committee of the American Sleep Disorders Association, 1995). Actigraphy can be used to estimate habitual sleep/wake patterns at home, but can be expensive, time intensive to score, and historically has had limited clinical use because of poor third-party reimbursement (McCrae, Taylor, Smith, & Perlis, 2010; Morgenthaler et al., 2007).

There are a number of evidence-based psychometrically sound parent- and child-report pediatric sleep measures (see Lewandowski, Toliver-Sokol, and Palermo, 2011, for comprehensive review). For example, the Children's Sleep Habits Questionnaire (CSHQ) is a well-established multidimensional parent-report measure of sleep in school-age children with strong psychometric support for assessing behavioral and medical sleep symptoms in children (e.g., bedtime resistance, sleep-disordered breathing, daytime sleepiness; Lewandowski et al., 2011; Owens, Spirito, & McGuinn, 2000). However, the time required to administer, score, and interpret this measure may be impractical for screening purposes in primary care settings, nor even as a measure of insomnia severity during routine follow-up of youth undergoing treatment for insomnia. Furthermore, the CSHQ was not specifically designed to measure insomnia symptoms. In fact, there are no insomnia-specific measures that can be used with children, and adult insomnia-specific measures (e.g., the Insomnia Symptoms Questionnaire [Okun et al., 2009]; Brief Insomnia Questionnaire [Kessler et al., 2010]; Insomnia Severity Index [Bastien, Vallieres, & Morin, 2001] cannot simply be generalized for use with youth [Lewandowski et al., 2011]).

Given the lack of insomnia-specific pediatric measures and the increasing need to document health care outcomes (Forrest & Silber, 2014), we developed the Pediatric Insomnia Severity Index (PISI). The PISI is a brief parent-proxy measure designed to efficiently monitor primary clinical symptoms of pediatric insomnia that are common treatment targets in the context of clinical care for youth (4–10 years) with sleep disturbance. The PISI consists of six items, which take <5 min to complete and score. This article examines, within the applied context of a comprehensive pediatric sleep disorders center, the psychometric properties of the PISI, including reliability and content, convergent, and discriminant validity.

Methods

Development of thePISI

The PISI was developed by a team of sleep specialists including two psychologists, four pulmonologists, a neurologist, and a pediatric nurse practitioner. The scale was developed to assess the hallmark features of pediatric insomnia while considering core measures of sleep continuity typically targeted during insomnia treatment. Group consensus was reached regarding the following ICSD-II general insomnia criteria that would guide item content: (1) difficulty falling asleep (PISI Items 1 and 2), (2) difficulty maintaining sleep (PISI Items 3 and 4), and (3) daytime impairment (PISI Item 5). In addition, prior insomnia research informed group consensus regarding selection of the following sleep continuity measurement domains that were included in the scale: (1) sleep-onset latency (PISI Item 1), (2) frequency of night wakings (PISI Item 3), and (3) sleep duration (PISI Item 6) (Perlis, Jungquist, Smith, & Posner, 2005). Subsequent to identifying foundational content domains, the team reviewed previously validated pediatric sleep measures to identify insomnia-specific items to guide item development for the brief insomnia severity measure. Group consensus was reached regarding six insomnia-specific items. PISI items cover some of the same ground as the CSHQ, but no items are identical. After the final PISI form was developed and approved by the clinical team, use in the clinic setting was started in early 2009. None of the six items originally included in the PISI was eliminated from the scale before initial clinical implementation.

Participants

All study procedures were approved and overseen by the hospital’s institutional review board. The sample included 462 youth and their caregivers, seen for behavioral sleep medicine evaluation within an accredited sleep disorders center located in a tertiary care pediatric hospital. Eligible families received print and verbal information explaining the purpose of the study. Before formal enrollment, primary caregivers provided written informed consent. Inclusion criteria were child aged 4–10 years and a clinical diagnosis of insomnia confirmed during a diagnostic sleep evaluation conducted by a licensed psychologist certified in behavioral sleep medicine or a clinical resident/fellow under the direct supervision of the sleep psychologist. During the enrollment period from June 2009 to March 2015, 482 patients were eligible for participation. In total, 11 parents did not provide consent, and nine families had incomplete data, resulting in a final sample of 462 subjects (96% of eligible subjects; 283 males, 179 females, mean = 7.28 years old, SD = 2.05 years). The majority self-identified as non-Hispanic White (78%), with remaining participants Black (11%), Hispanic (2%), Asian (1%), or Multiracial (8%). In total, 62% of parents were married, 21% unmarried, 10% divorced, 4% separated, 2% remarried, and 1% widowed. In total, 25% reported annual income <$20,000, 27% reported $20,000–49,000/year, 26% reported $50,000–99,000/year, and 22% >$100,000/year.

Procedures

Participants were referred to an accredited sleep disorders center and were triaged for evaluation based on referral question and parent-reported history gathered during a telephone intake interview. Patients had a chief complaint of insomnia and underwent a comprehensive sleep evaluation with a behavioral sleep medicine clinician. The diagnostic work-up included a clinical interview and battery of sleep screening instruments (see measures). Patients with symptoms suggestive of an organic sleep disorder (e.g., OSA) were also triaged for evaluation with a board-certified sleep physician.

Measures

Demographic Information

On an intake questionnaire, parents reported demographic information in a multiple choice format, and relevant health conditions in a yes/no format.

PISI

The development of the PISI (see Figure 1) was described above. In brief, the PISI is a six-item paper-pencil parent-proxy measure that quantifies insomnia symptom severity in children 4–10 years old. The total sleep duration item is rated on a 0–6-point scale with each rating designating an estimate of total hours slept on most nights (e.g., 6 = <5 hr of sleep). All other items are rated on a 0–6-point scale from “never” (0 times/week) to “always” (seven nights/week). Respondents are asked to characterize the patient’s sleep over the past week. Preliminary data demonstrate that the PISI is sensitive to changes in insomnia severity for patients treated with empirically supported behavioral and cognitive behavioral therapy for insomnia (Byars & Simon, 2014).
Figure 1.

Pediatric insomnia severity index.

CSHQ

The CSHQ (Owens et al., 2000) is a 33-item parent-report measure of sleep behavior and sleep disorder symptoms developed for use with children 4–10 years old. Caregivers use a 3-point Likert scale to rate the frequency of specific sleep symptoms/behaviors during the previous week (rarely to usually). The measure yields a total score and eight subscale scores for which higher scores represent more problematic sleep. The CSHQ has demonstrated validity for use with preschool and school-aged children and has been shown to differentiate clinical from control groups (Goodlin-Jones, Sitnick, Tang, Liu, & Anders, 2008; Owens et al., 2000). Published psychometrics demonstrate adequate internal consistency (0.68–0.78) and test–retest reliability (0.62–0.79) for clinical and community comparison groups (Owens et al., 2000). A total score >41 appears to be the most sensitive clinical cutoff for identifying sleep problems in children (Owens et al., 2000). Primary caregivers completed the CSHQ.

Sleep Disorders Inventory for Students

The sleep disorders inventory for students (SDIS; Luginbuehl, Bradley-Klug, Ferron, Anderson, & Benbadis, 2008) is a paper-pencil parent-report measure screening for sleep disorders that have an organic basis, including OSA, narcolepsy, periodic limb movement disorder (PLMD), and delayed sleep phase syndrome (DSPS). Items are rated using a Likert scale ranging from 1 (the child never exhibits this behavior) to 7 (child exhibits behavior multiple times per hour daily or nightly). Factor scores for each sleep disorder domain and a total sleep disorder score are computed. T-scores of 65 and higher are considered to be suggestive of a high risk for a sleep disorder. The SDIS-Child version is 41 items and is used with children 2–10 years old. Comprehensive psychometrics on the instrument demonstrate adequate internal consistency, test–retest reliability, and validity (Luginbuehl et al., 2008). Primary caregivers completed the SDIS.

Sleep Diagnosis

ICSD-II (American Academy of Sleep Medicine, 2005a) sleep diagnoses were given by a behavioral sleep medicine clinician. Every subject included in the study met ICSD-II criteria for a primary insomnia diagnoses as follows: 107 participants (23%) were diagnosed with behavioral insomnia of childhood (BIC) sleep-onset association type, 111 (24%) with BIC limit-setting type, 141 (31%) with BIC combined type, and 103 (22%) with psychophysiological insomnia. In total, 29% of participants also met criteria for a secondary sleep disorder diagnosis, and 13% met criteria for three sleep disorder diagnoses.

Analysis Plan

This study examined the psychometric properties of a two-factor conceptual model reflecting primary clinical symptoms of pediatric insomnia (sleep-onset and sleep-maintenance problems) and relevant sleep continuity variables (sleep-onset latency, frequency of night wakings, and total sleep time). The factor domains were conceptualized as follows: (1) sleep-onset problems (aggregate of PISI Items 1, 2, and 6) and (2) sleep-maintenance problems (aggregate of PISI Items 3, 4, and 6). PISI factor scores were computed by summing item-level responses loading on each factor. One item (Item 5) was excluded from model testing because of poor inter-item correlation. In retrospect, Item 5 (sleepiness) was constructed to measure daytime consequences of pediatric insomnia, but was likely too narrowly defined in light of the broad range of secondary effects of pediatric insomnia (e.g., sleepiness, fatigue, attention, concentration or memory difficulty, impaired psychosocial, academic, or vocational functioning, mood and/or behavioral disturbance, sleepiness).

SPSS Statistics software version 23 and Mplus version 7.4 were used for data analysis. Missing data for the CSHQ and SDIS were typically the result of the omission of one or two items on a multi-item subscale. Capitalizing on the strong internal consistency of these measures, in cases where <50% of subscales items were missing, composites were prorated from the remaining items for the subscale (Beebe et al., 2007). If >50% of the item-level responses making up a subscale were missing, no data for the measure were included in the final data analysis. After proration, <2% of any given validation measure had missing data in the final sample of 462, all of whom had fully completed the PISI (to allow for a complete assessment of psychometrics for the PISI, cases were dropped in which the PISI was incomplete). Refer to Table 2 where n values are reported for completed study measures.

All statistical tests were evaluated at the p < .05 significance level and constituted two-tailed tests. Because the large sample raised the potential for statistically significant effects of dubious clinical relevance, we also examined effect sizes. Following convention, a correlation between .10 and .29 was considered a small effect, .30 and .49 a medium effect, and ≥.50 a large effect (Cohen, Cohen, West, & Aiken, 2003).

Reliability and Construct Validity of PISI

Internal consistency reliability of the PISI factor scores was examined through Cronbach’s α.

Confirmatory factor analysis (CFA) procedures were used to confirm superior model fit of a clinically relevant and conceptually derived two-factor model over a single-factor solution and to test for invariance across sex. Although we report χ2 for model fit, that statistic is considered overly sensitive in large samples, so we set a priori criteria for an acceptable fit as a root mean square error of approximation (RMSEA) ≤0.08 and a Comparative Fit Index (CFI) > 0.95 (Hu & Bentler, 1999). PISI item-level response data were treated as ordinal (ordered categorical), and CFA parameter estimates were obtained via a categorical response variable parameter estimation algorithm (i.e., WLSMV; see Muthen and Muthen [1998-2015]; p. 608). The DIFFTEST procedure in Mplus version 7.4 (Muthen and Muthen [1998-2015]; p. 450) was tested for the invariance of the proposed CFA model across gender. To avoid parameter estimation nonconvergence issues, thresholds for the five sleep questionnaire items were constrained to equality and the scale factors for the five sleep questionnaire items were fixed at unity between male and female participants (see Muthen and Muthen [1998-2015]; p. 685–686).

Convergent and Discriminant Validity of the PISI

Correlations were used to assess convergent validity (correlations with measures of conceptually linked constructs) and discriminant validity (lack of correlation with measures of conceptually distinct constructs). For parsimony, we focused on subscales with the clearest conceptual relationships. See Table I for a list of the relationships between the PISI and validation measures that were determined a priori to support convergent and discriminant validity. Finally, we assessed whether PISI factor scores varied by clinical diagnosis via an exploratory analysis of variance (ANOVA).

Table I.

Sleep Measure Subscales Hypothesized to Support Convergent and Discriminant Validity for thePISI

PISI
Sleep measureSleep-onset problemsSleep-maintenance problems
Child Sleep Habits Questionnaire
 Bedtime resistanceConvergentDiscriminant
 Sleep-onset delayConvergentDiscriminant
 Night wakingDiscriminantConvergent
 ParasomniasDiscriminantConvergent
 SDBDiscriminantConvergent
Sleep disorders inventory for students–child
 OSADiscriminantConvergent
 PLMDDiscriminantConvergent
 DSPSConvergentDiscriminant
PISI
Sleep measureSleep-onset problemsSleep-maintenance problems
Child Sleep Habits Questionnaire
 Bedtime resistanceConvergentDiscriminant
 Sleep-onset delayConvergentDiscriminant
 Night wakingDiscriminantConvergent
 ParasomniasDiscriminantConvergent
 SDBDiscriminantConvergent
Sleep disorders inventory for students–child
 OSADiscriminantConvergent
 PLMDDiscriminantConvergent
 DSPSConvergentDiscriminant

Note. Convergent validity would be supported if the PISI factor score displayed significant positive correlations with the sleep measure subscales hypothesized to be most conceptually consistent with that factor domain. This is indicated in the table with “Convergent.” Discriminant validity would be supported by minimal correlations between the PISI factor score and the sleep measure subscales hypothesized to be most conceptually distinct from that factor domain, indicated in the table with “Discriminant.”

DSPS = delayed sleep phase syndrome; OSA = obstructive sleep apnea; PISI = Pediatric Insomnia Severity Index; PLMD = periodic limb movement disorder; SDB = sleep disordered breathing.

Table I.

Sleep Measure Subscales Hypothesized to Support Convergent and Discriminant Validity for thePISI

PISI
Sleep measureSleep-onset problemsSleep-maintenance problems
Child Sleep Habits Questionnaire
 Bedtime resistanceConvergentDiscriminant
 Sleep-onset delayConvergentDiscriminant
 Night wakingDiscriminantConvergent
 ParasomniasDiscriminantConvergent
 SDBDiscriminantConvergent
Sleep disorders inventory for students–child
 OSADiscriminantConvergent
 PLMDDiscriminantConvergent
 DSPSConvergentDiscriminant
PISI
Sleep measureSleep-onset problemsSleep-maintenance problems
Child Sleep Habits Questionnaire
 Bedtime resistanceConvergentDiscriminant
 Sleep-onset delayConvergentDiscriminant
 Night wakingDiscriminantConvergent
 ParasomniasDiscriminantConvergent
 SDBDiscriminantConvergent
Sleep disorders inventory for students–child
 OSADiscriminantConvergent
 PLMDDiscriminantConvergent
 DSPSConvergentDiscriminant

Note. Convergent validity would be supported if the PISI factor score displayed significant positive correlations with the sleep measure subscales hypothesized to be most conceptually consistent with that factor domain. This is indicated in the table with “Convergent.” Discriminant validity would be supported by minimal correlations between the PISI factor score and the sleep measure subscales hypothesized to be most conceptually distinct from that factor domain, indicated in the table with “Discriminant.”

DSPS = delayed sleep phase syndrome; OSA = obstructive sleep apnea; PISI = Pediatric Insomnia Severity Index; PLMD = periodic limb movement disorder; SDB = sleep disordered breathing.

Table II.

Sleep Questionnaire Descriptive Statistics (Mean±SD)

Pediatric Insomnia Severity Scale, raw scores (n = 462)
 Factor scores
 Sleep-onset problems9.41 ± 3.93
 Sleep-maintenance problems8.30 ± 3.32
Child Sleep Habits Questionnaire, raw scores (n = 459)
 Subscales
 Bedtime resistance10.83 ± 3.22
 Sleep-onset delay2.25 ± 0.83
 Night waking6.52 ± 1.63
 Parasomnias11.46 ± 2.61
 Sleep-disordered breathing3.83 ± 1.28
Sleep disorders inventory for students–child version, T-score (n = 460)
 Subscales
 Obstructive sleep apnea57.53 ± 9.80
 Periodic limb movement disorder60.49 ± 8.94
 Delayed sleep phase syndrome63.50 ± 14.36
Pediatric Insomnia Severity Scale, raw scores (n = 462)
 Factor scores
 Sleep-onset problems9.41 ± 3.93
 Sleep-maintenance problems8.30 ± 3.32
Child Sleep Habits Questionnaire, raw scores (n = 459)
 Subscales
 Bedtime resistance10.83 ± 3.22
 Sleep-onset delay2.25 ± 0.83
 Night waking6.52 ± 1.63
 Parasomnias11.46 ± 2.61
 Sleep-disordered breathing3.83 ± 1.28
Sleep disorders inventory for students–child version, T-score (n = 460)
 Subscales
 Obstructive sleep apnea57.53 ± 9.80
 Periodic limb movement disorder60.49 ± 8.94
 Delayed sleep phase syndrome63.50 ± 14.36
Table II.

Sleep Questionnaire Descriptive Statistics (Mean±SD)

Pediatric Insomnia Severity Scale, raw scores (n = 462)
 Factor scores
 Sleep-onset problems9.41 ± 3.93
 Sleep-maintenance problems8.30 ± 3.32
Child Sleep Habits Questionnaire, raw scores (n = 459)
 Subscales
 Bedtime resistance10.83 ± 3.22
 Sleep-onset delay2.25 ± 0.83
 Night waking6.52 ± 1.63
 Parasomnias11.46 ± 2.61
 Sleep-disordered breathing3.83 ± 1.28
Sleep disorders inventory for students–child version, T-score (n = 460)
 Subscales
 Obstructive sleep apnea57.53 ± 9.80
 Periodic limb movement disorder60.49 ± 8.94
 Delayed sleep phase syndrome63.50 ± 14.36
Pediatric Insomnia Severity Scale, raw scores (n = 462)
 Factor scores
 Sleep-onset problems9.41 ± 3.93
 Sleep-maintenance problems8.30 ± 3.32
Child Sleep Habits Questionnaire, raw scores (n = 459)
 Subscales
 Bedtime resistance10.83 ± 3.22
 Sleep-onset delay2.25 ± 0.83
 Night waking6.52 ± 1.63
 Parasomnias11.46 ± 2.61
 Sleep-disordered breathing3.83 ± 1.28
Sleep disorders inventory for students–child version, T-score (n = 460)
 Subscales
 Obstructive sleep apnea57.53 ± 9.80
 Periodic limb movement disorder60.49 ± 8.94
 Delayed sleep phase syndrome63.50 ± 14.36

Results

Reliability and Construct Validity of PISI

Internal consistency analyses for the PISI factor scores (Cronbach’s α) yielded the following reliability indices: sleep-onset problems α = .81 and sleep-maintenance problems α = .62.

Fitting our conceptually coherent and clinically relevant two-factor model to the sample data using CFA yielded the following fit indices: n = 462; χ42=52.56,p < .0001; CFI = 0.99, RMSEA = 0.16, 90% confidence interval = 0.12, 0.20 (see Figure 2). Results from the categorical CFA test of invariance (“DIFFTEST”) showed the CFA parameter estimates were invariant between male and female participants (χ62=5.44p = .49).
Figure 2.

Pediatric insomnia severity index confirmatory factor analysis.

Note: Factor covariance and factor loadings for each Pediatric Insomnia Severity Index item are reported as unstandardized model results followed by standardized model results in parentheses.

Mean scores and standard deviations for study measures are presented in Table II.

Convergent and Discriminant Validity of the PISI

Results from analyses examining the convergent and discriminant validity of the PISI are presented in Table III. All hypotheses (eight of eight) regarding PISI factor scores/sleep screening subscales for convergent validity were supported by statistically significant positive correlations, and half represented medium to large effects. Although five of eight PISI discriminant validity associations were also statistically significant, all effect sizes were small and of dubious clinical significance.

Table III.

Convergent and Discriminant Validity of the Pediatric Insomnia Severity Index (PISI)

PISI
Sleep measureSleep-onset problems
Sleep-maintenance problems
ExpectedActualExpectedActual
Child Sleep Habits Questionnaire Subscales
 Bed resistance+0.27***00.02
 Sleep-onset delay+0.66***00.17**
 Night waking0−0.12*+0.37***
 Parasomnias00.03+0.16**
 Sleep-disordered breathing00.00+0.09*
Sleep disorders inventory for students–child subscales
 Obstructive sleep apnea00.16**+0.21***
 Periodic limb movement disorder00.27***+0.30***
 Delayed sleep phase syndrome+0.70***00.17**
PISI
Sleep measureSleep-onset problems
Sleep-maintenance problems
ExpectedActualExpectedActual
Child Sleep Habits Questionnaire Subscales
 Bed resistance+0.27***00.02
 Sleep-onset delay+0.66***00.17**
 Night waking0−0.12*+0.37***
 Parasomnias00.03+0.16**
 Sleep-disordered breathing00.00+0.09*
Sleep disorders inventory for students–child subscales
 Obstructive sleep apnea00.16**+0.21***
 Periodic limb movement disorder00.27***+0.30***
 Delayed sleep phase syndrome+0.70***00.17**

Note. *p < .05, **p < .01, ***p < .001. Expected associations reflect those detailed in Table 1 that would reflect convergent validity evidence because of significant positive (+) associations versus evidence of discriminant validity via minimal associations (0). To aid interpretation, correlations between .30 and .49 (medium effects) are in bold, while those of .50 or greater (large effects) are in bold and underlined.

Table III.

Convergent and Discriminant Validity of the Pediatric Insomnia Severity Index (PISI)

PISI
Sleep measureSleep-onset problems
Sleep-maintenance problems
ExpectedActualExpectedActual
Child Sleep Habits Questionnaire Subscales
 Bed resistance+0.27***00.02
 Sleep-onset delay+0.66***00.17**
 Night waking0−0.12*+0.37***
 Parasomnias00.03+0.16**
 Sleep-disordered breathing00.00+0.09*
Sleep disorders inventory for students–child subscales
 Obstructive sleep apnea00.16**+0.21***
 Periodic limb movement disorder00.27***+0.30***
 Delayed sleep phase syndrome+0.70***00.17**
PISI
Sleep measureSleep-onset problems
Sleep-maintenance problems
ExpectedActualExpectedActual
Child Sleep Habits Questionnaire Subscales
 Bed resistance+0.27***00.02
 Sleep-onset delay+0.66***00.17**
 Night waking0−0.12*+0.37***
 Parasomnias00.03+0.16**
 Sleep-disordered breathing00.00+0.09*
Sleep disorders inventory for students–child subscales
 Obstructive sleep apnea00.16**+0.21***
 Periodic limb movement disorder00.27***+0.30***
 Delayed sleep phase syndrome+0.70***00.17**

Note. *p < .05, **p < .01, ***p < .001. Expected associations reflect those detailed in Table 1 that would reflect convergent validity evidence because of significant positive (+) associations versus evidence of discriminant validity via minimal associations (0). To aid interpretation, correlations between .30 and .49 (medium effects) are in bold, while those of .50 or greater (large effects) are in bold and underlined.

PISI ScoresAcross Diagnostic Subgroups

PISI factor scores according to insomnia diagnosis are presented in Table IV. One-way ANOVAs indicated that insomnia severity scores varied by diagnosis for sleep-onset problems [F(3,458) = 9.34, p ≤ .001] and sleep-maintenance insomnia [F(3,458) = 4.07, p = .007]. Patients with BIC limit-setting type insomnia displayed the greatest insomnia severity scores for both sleep-onset problems and sleep-maintenance problems relative to all other diagnoses, followed by BIC combined type, psychophysiological insomnia, and BIC sleep-onset type in that order.

Table IV.

PISI Factor Scores by Diagnosis (Mean±SD)

PISI factorsInsomnia diagnosis
BIC sleep-onset associationBIC limit settingBIC combinedPsychophysiological insomnia
Sleep-onset problems7.88 ± 3.9010.31 ± 3.6310.08 ± 3.909.09 ± 4.09
Sleep-maintenance problems7.50 ± 3.059.05 ± 3.308.38 ± 3.458.21 ± 3.32
PISI factorsInsomnia diagnosis
BIC sleep-onset associationBIC limit settingBIC combinedPsychophysiological insomnia
Sleep-onset problems7.88 ± 3.9010.31 ± 3.6310.08 ± 3.909.09 ± 4.09
Sleep-maintenance problems7.50 ± 3.059.05 ± 3.308.38 ± 3.458.21 ± 3.32

Note. BIC = behavioral insomnia of childhood; PISI = Pediatric Insomnia Severity Index.

Table IV.

PISI Factor Scores by Diagnosis (Mean±SD)

PISI factorsInsomnia diagnosis
BIC sleep-onset associationBIC limit settingBIC combinedPsychophysiological insomnia
Sleep-onset problems7.88 ± 3.9010.31 ± 3.6310.08 ± 3.909.09 ± 4.09
Sleep-maintenance problems7.50 ± 3.059.05 ± 3.308.38 ± 3.458.21 ± 3.32
PISI factorsInsomnia diagnosis
BIC sleep-onset associationBIC limit settingBIC combinedPsychophysiological insomnia
Sleep-onset problems7.88 ± 3.9010.31 ± 3.6310.08 ± 3.909.09 ± 4.09
Sleep-maintenance problems7.50 ± 3.059.05 ± 3.308.38 ± 3.458.21 ± 3.32

Note. BIC = behavioral insomnia of childhood; PISI = Pediatric Insomnia Severity Index.

Discussion

This article reports the psychometric properties of the PISI based on testing with 462 youth with clinically diagnosed insomnia. Overall, the current study findings provide preliminary support for reliability and validity of the PISI. Although initial conceptualization, design, and clinical implementation of the PISI preceded publication of the Journal of Pediatric Psychology (JPP) guidelines for measure development (Holmbeck & Devine, 2009), the process for developing the PISI was fundamentally consistent with the most important JPP criteria including (1) clearly establishing a research and/or clinical need for the instrument and (2) using a rigorous, conceptually driven, and empirically based methodology for developing item/scale content. In addition, the current study analyses and resulting psychometric data presented herein, largely align with the JPP guidelines for evaluating (1) reliability, (2) factor structure, (3) convergent and discriminant validity, and (4) clinical utility of newly developed measures (Holmbeck & Devine).

Measures of internal consistency for the PISI factor scores varied with the sleep-onset factor (Cronbach’s α = .81) showing acceptable reliability and the sleep-maintenance factor (Cronbach’s α = .62) showing less optimal reliability (Nunnally & Bernstein, 1994). Reliability levels were related to the brevity of the scales and to interitem correlations that varied by factor. The sleep-onset factor was characterized by medium to large interitem correlations (.43–.86), while marginal reliability for the sleep-maintenance factor was a function of statistically significant, but small to medium interitem correlations (.22–.45).

An interpretive approach that focused on the hallmark symptoms of pediatric insomnia was partially supported by CFA. A single-factor solution demonstrated less optimal fit, both conceptually and statistically relative to the proposed two-factor model. The conceptually driven two-factor model (sleep-onset problems and sleep-maintenance problems) that is reflective of current insomnia nosology had an excellent CFI. However, the RMSEA suggested poorer fit.

Although reliability indices and fit statistics were not uniformly supportive of the proposed two-factor structure, on balance, given the clinical utility of differentiating sleep-onset difficulties from sleep-maintenance difficulties, we recommend the two-factor solution as the most suitable interpretive approach.

Findings regarding convergent validity of the PISI are promising and provide preliminary support for use of the PISI as a brief instrument to examine both sleep-onset and sleep-maintenance problems in pediatric insomnia patients. All (eight of eight) of the hypothesized correlations supporting convergent validity of the PISI were confirmed. Sleep-onset problems on the PISI were significantly linked to established validation measures of bedtime resistance, sleep-onset delay, and DSPS (reflective of sleep-onset insomnia symptoms). Sleep-maintenance problems on the PISI were significantly linked to established validation measures of night wakings, difficulties falling back asleep, and symptoms related to sleep fragmentation, including OSA, parasomnias, and restless sleep. Of note, those validation measures are nested within much longer questionnaires that are, because of their length and scoring complexity, difficult to integrate into routine screening in general pediatric settings or in serial assessments to track insomnia treatment effects.

It is noteworthy that significant effects observed for convergent validity analyses with PISI factors and screening subscales most aligned with pediatric insomnia symptoms (e.g., sleep-onset delay [r = .66], protracted sleep-onset secondary to delayed circadian rhythm [r = .70], and night waking [r = .37]) were largest and represented medium to large effect sizes. Although bedtime resistance is a common feature of pediatric insomnia, the small effect observed for bedtime resistance (r = .27) may be related to clinical presentations when bedtime resistance disrupts the prebed routine, but does not necessarily directly impact sleep-onset latency once the child is eventually in bed. Convergent validity associations with screening subscale measures of sleep fragmentation (i.e., parasomnias, OSA, and PLMD) were in the expected direction, but most (75%) represented small to negligible effects (range 0.09–0.21). Although OSA, parasomnias, and PLMD may share sleep fragmentation as a common problem domain along with insomnia, they are distinct disorders with different cardinal symptoms from insomnia and thus may not be optimal convergent validity measures.

Discriminant validity testing with the PISI was less supportive of a priori expectations. However, when there is shared method variance (in this case, the same response format and same reporter for the PISI and its validation measures), it is not unusual to see some degree of correlation between measures; what is more important is that convergent validity analyses show notably stronger correlations than discriminant validity analyses (Campbell & Fiske, 1959), which was overwhelmingly the case here. As noted, convergent validity associations with direct measures of insomnia symptoms represented medium to large effect sizes, whereas discriminant validity associations tended to be small to negligible, with statistical significance reached primarily because of the large sample.

Although the PISI was not intended to be used as a diagnostic tool nor a scale for clarifying insomnia etiology, we explored how scores varied across clinically confirmed insomnia diagnoses. Both PISI factor scores varied significantly by diagnosis, with patients with BIC limit-setting type insomnia generally faring worst, and BIC sleep-onset type generally faring best. However, there was considerable within-diagnosis variability, and PISI scores will likely vary depending on the unique circumstances of the child. Use of the PISI factor scores for assessing insomnia severity may help to prioritize insomnia treatment targets.

Clinical Use of the PISI

To be clear, the PISI is not a replacement for thorough diagnostic evaluation. Sleep medicine practice guidelines make clear the critical importance of understanding the primary etiologic mechanisms for insomnia to guide empirically supported therapies. Formal sleep disorders evaluation requires a comprehensive sleep assessment that considers relevant developmental, psychological, psychiatric, and medical factors that may impact sleep along with sleep-specific variables including the sleep environment, routines, schedule, and sleep hygiene. Insomnia evaluation should always include a thorough sleep history and screening for sleep disorders in the context of a clinical interview (Chesson et al., 2000). Other sleep measures obtained may include actigraphy, sleep diaries, parent-proxy or self-report questionnaires, symptom checklist, or psychological screening measures (Chesson et al.).

However, such intensive diagnostic procedures are not well suited for brief screening of insomnia symptoms or ongoing assessment during the course of clinical care for pediatric patients. The PISI was developed to fill that void. Overall, the present data support its reliability and validity, which are impressive considering its striking brevity and ease of interpretation. These latter features, though more difficult to quantify than psychometrics, are essential in busy clinical settings. The ease in which the PISI is integrated clinically is attested to by its continuous use in a busy behavioral sleep medicine clinic for the better part of a decade, where it is administered both at baseline (as described here) and across follow-up visits to help guide treatment decisions. Parents can readily complete the PISI at the time of check-in, and clinicians can quickly compute the sleep-onset (sum of Items 1, 2, and 6) and sleep-maintenance (sum of Items 3, 4, and 6) factor scores, allowing an immediate point of reference for clinical conversations and symptom tracking over time. Preliminary data indicate the PISI is sensitive to treatment changes and thus may have promise as a tool that can aide in decisions about treatment termination (Byars & Simon, 2014).

Study Limitations and Future Directions

It is important to consider the study findings within the context of specific methodological limitations. Findings are potentially biased by shared method variance. All validity testing for the PISI was conducted with sleep-specific paper-pencil measures completed by primary caregivers. Although tests of convergent validity were significant and represented medium to large effect sizes for insomnia-specific items, potential for reporter bias on measures with overlapping/similar content must be considered. In addition, while discriminant validity analyses demonstrated lack of correlation or weak associations between insomnia-specific symptoms on the PISI and validated measures of symptoms from conditions distinct from insomnia (e.g., OSA, PLMD), findings from tests of discriminant validity that examined dissimilar core insomnia symptoms on the PISI and CSHQ were not measuring distinct constructs and thus should be interpreted with caution. In addition, as previously discussed, CFA findings were mixed, though on balance the two-factor model seemed most suitable for interpretation.

The current study involved a clinical sample within a single sleep disorders center. Further examination in multisite clinical settings as well as community and primary care settings will help determine whether the PISI can discriminate between those children with and without sleep problems, as well as the best cutoff scores as a screener. In addition, it remains to be seen if the PISI could be used as a measure to differentiate insomnia patients from patients with other sleep disorders commonly seen for evaluation in comprehensive sleep disorders centers (e.g., OSA, parasomnia, restless legs syndrome/PLMD). Also, it would be helpful to examine validity with other sleep measures, such as sleep diaries or actigraphy. Further validation with a more racially diverse sample, and using Spanish and other language translations of the measure is recommended to ensure that the PISI is valid across cultures and racial/ethnic groups.

Even so, the PISI is the only empirically supported brief insomnia-specific measure developed for use with pediatric patients. The current study findings provide preliminary support for the reliability and validity of the PISI as a measure of insomnia severity in a large clinical sample of children and adolescents referred for sleep evaluation. These findings, combined with previous evidence demonstrating sensitivity of the PISI to treatment changes, suggest that it has promise for use in pediatric sleep medicine clinics and as a screener in broader clinical care.

Disclosures

The authors have no financial or other relationship that might lead to a conflict of interest. The study did not involve the use of off-label or investigational drugs.

Funding

This research study was completed at Cincinnati Children’s Hospital Medical Center without external financial support. Internal divisional funds (Behavioral Medicine and Clinical Psychology) were used to support completion of the study. The study did not involve the use of off-label or investigational drugs. None of the authors have any conflicts of interest.

Conflicts of interest: None declared.

Acknowledgments

The authors thank the staff in Cincinnati Children’s Hospital Medical Center (CCHMC), Division of Behavioral Medicine and Clinical Psychology—Data Core for their assistance with data management and storage and the CCHMC Sleep Disorders Center Faculty and Staff for their collaboration in providing clinical care for children with sleep disorders. We also thank the many families that participated in the study.

References

American Academy of Sleep Medicine
. (
2005a
).
International Classification of Sleep Disorders: Diagnostic and Coding Manual
. (Vol. 2).
Westchester, IL
:
American Academy of Sleep Medicine
.

American Academy of Sleep Medicine
. (
2014
).
International Classification of Sleep Disorders
(3rd ed.).
Darien, IL
:
American Academy of Sleep Medicine
.

American Sleep Disorders Association Diagnostic Classification Steering Committee
. (
1990
).
International Classification of Sleep Disorders: Diagnostic and Coding Manual
.
Rochester
:
American Sleep Disorder Association
.

Bastien
C. H.
,
Vallieres
A.
,
Morin
C. M.
(
2001
).
Validation of the insomnia severity index as an outcome measure for insomnia research
.
Sleep Med
,
2
,
297
307
.

Beebe
D. W.
,
Lewin
D.
,
Zeller
M.
,
McCabe
M.
,
MacLeod
K.
,
Daniels
S. R.
,
Amin
R.
(
2007
).
Sleep in overweight adolescents: Shorter sleep, poorer sleep quality, sleepiness, and sleep-disordered breathing
.
Journal of Pediatric Psychology
,
32
,
69
79
. doi: 10.1093/jpepsy/jsj104

Blunden
S.
,
Lushington
K.
,
Lorenzen
B.
,
Ooi
T.
,
Fung
F.
,
Kennedy
D.
(
2004
).
Are sleep problems under-recognised in general practice?
Archives of Disease in Childhood
,
89
,
708
712
. doi: 10.1136/adc.2003.027011

Burnham
M. M.
,
Goodlin-Jones
B. L.
,
Gaylor
E. E.
,
Anders
T. F.
(
2002
).
Nighttime sleep-wake patterns and self-soothing from birth to one year of age: A longitudinal intervention study
.
Journal of Child Psychology and Psychiatry
,
43
,
713
725
.

Byars
K.
,
Simon
S. L.
(
2014
).
Practice patterns and insomnia treatment outcomes from an evidence-based pediatric behavioral sleep medicine clinic
.
Clinical Practice in Pediatric Psychology
,
2
,
337
349
. doi: 10.1037/cpp0000068

Byars
K.
,
Yolton
K.
,
Rausch
J.
,
Lanphear
B.
,
Beebe
D. W.
(
2012
).
Prevalence, patterns, and persistence of sleep problems in the first 3 years of life
.
Pediatrics
,
129
,
e276
e284
. doi: 10.1542/peds.2011-0372

Campbell
D. T.
,
Fiske
D. W.
(
1959
).
Convergent and discriminant validation by the multitrait-multimethod matrix
.
Psychological Bulletin
,
56
,
81
105
.

Chen
X.
,
Beydoun
M. A.
,
Wang
Y.
(
2008
).
Is sleep duration associated with childhood obesity? A systematic review and meta-analysis
.
Obesity (Silver Spring)
,
16
,
265
274
. doi: 10.1038/oby.2007.63

Chesson
A.
Jr,
Hartse
K.
,
Anderson
W. M.
,
Davila
D.
,
Johnson
S.
,
Littner
M.
,
Wise
M.
,
Rafecas
J.
(
2000
).
Practice parameters for the evaluation of chronic insomnia. An American Academy of Sleep Medicine report. Standards of Practice Committee of the American Academy of Sleep Medicine
.
Sleep
,
23
,
237
241
.

Cohen
J.
,
Cohen
P.
,
West
S. G.
,
Aiken
L. S.
(
2003
).
Applied Multiple Correlation/Regression Analysis for the Behavioral Sciences
.
Mahwah, NJ
:
Lawrence Erlbaum Associates
.

Forrest
C. B.
,
Silber
J. H.
(
2014
).
Concept and measurement of pediatric value
.
Academic Pediatrics
,
14(Suppl 5)
,
S33
S38
. doi: 10.1016/j.acap.2014.03.013

Goodlin-Jones
B. L.
,
Sitnick
S. L.
,
Tang
K.
,
Liu
J.
,
Anders
T. F.
(
2008
).
The children's sleep habits questionnaire in toddlers and preschool children
.
Journal of Developmental and Behavioral Pediatrics
,
29
,
82
88
.

Holmbeck
G. N.
,
Devine
K. A.
(
2009
).
Editorial: An author's checklist for measure development and validation manuscripts
.
Journal of Pediatric Psychology
,
34
,
691
696
. doi: 10.1093/jpepsy/jsp046

Hu
L. -t.
,
Bentler
P. M.
(
1999
).
Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives
.
Structural Equation Modeling
,
6
,
1
55
. doi: 10.1080/10705519909540118

Kessler
R. C.
,
Coulouvrat
C.
,
Hajak
G.
,
Lakoma
M. D.
,
Roth
T.
,
Sampson
N
,
Shahly
V.
,
Shillington
A.
,
Stephenson
J. J.
,
Walsh
J. K.
,
Zammit
G. K.
(
2010
).
Reliability and validity of the brief insomnia questionnaire in the America insomnia survey
.
Sleep
,
33
,
1539
1549
.

Lewandowski
A. S.
,
Toliver-Sokol
M.
,
Palermo
T. M.
(
2011
).
Evidence-based review of subjective pediatric sleep measures
.
Journal of Pediatric Psychology
,
36
,
780
793
. doi: 10.1093/jpepsy/jsq119

Lozoff
B.
,
Wolf
A. W.
,
Davis
N. S.
(
1985
).
Sleep problems seen in pediatric practice
.
Pediatrics
,
75
,
477
483
.

Luginbuehl
M.
,
Bradley-Klug
K. L.
,
Ferron
J.
,
Anderson
W. M.
,
Benbadis
S. R.
(
2008
).
Pediatric sleep disorders: Validation of the sleep disorders inventory for students
.
School Psychology Review
,
37
,
409
431
.

McCrae
C. S.
,
Taylor
D. J.
,
Smith
M. T.
,
Perlis
M. L.
(
2010
).
The future of behavioral sleep medicine: A report on the presentations given at the Ponte Vedra Behavioral Sleep Medicine Consensus Conference, March 27-29, 2009
.
Behavioral Sleep Medicine
,
8
,
74
89
. doi: 10.1080/15402001003622792

Meltzer
L. J.
(
2010
).
Clinical management of behavioral insomnia of childhood: Treatment of bedtime problems and night wakings in young children
.
Behavioral Sleep Medicine
,
8
,
172
189
. doi: 10.1080/15402002.2010.487464

Meltzer
L. J.
,
Johnson
C.
,
Crosette
J.
,
Ramos
M.
,
Mindell
J. A.
(
2010
).
Prevalence of diagnosed sleep disorders in pediatric primary care practices
.
Pediatrics
,
125
,
e1410
e1418
. doi: 10.1542/peds.2009-2725

Meltzer
L. J.
,
Mindell
J. A.
(
2014
).
Systematic review and meta-analysis of behavioral interventions for pediatric insomnia
.
Journal of Pediatric Psychology
,
39
,
932
948
. doi: 10.1093/jpepsy/jsu041

Morgenthaler
T. I.
,
Lee-Chiong
T.
,
Alessi
C.
,
Friedman
L.
,
Aurora
R. N.
,
Boehlecke
B.
,
Brown
T.
,
Chesson
A. L.
Jr.
,
Kapur
V.
,
Maganti
R.
,
Owens
J.
,
Pancer
J.
,
Swick
T. J.
,
Zak
R.
(
2007
).
Practice parameters for the clinical evaluation and treatment of circadian rhythm sleep disorders. An American Academy of Sleep Medicine report
.
Sleep
,
30
,
1445
1459
.

Morgenthaler
T. I.
,
Owens
J.
,
Alessi
C.
,
Boehlecke
B.
,
Brown
T. M.
,
Coleman
J.
Jr.
,
Friedman
L.
,
Kapur
V. K.
,
Lee-Chiong
T.
,
Pancer
J.
,
Swick
T. J.
;
American Academy of Sleep Medicine
. (
2006
).
Practice parameters for behavioral treatment of bedtime problems and night wakings in infants and young children
.
Sleep
,
29
,
1277
1281
.

Muthen
L. K.
,
Muthen
B. O.
(
1998–2015
).
Mplus User's Guide
(7th ed.).
Los Angeles, CA
:
Muthen & Muthen
.

Nunnally
J. C.
,
Bernstein
I. H.
(
1994
).
Psychometric Theory
.
New York
:
McGraw-Hill
.

Okun
M. L.
,
Kravitz
H. M.
,
Sowers
M. F.
,
Moul
D. E.
,
Buysse
D. J.
,
Hall
M.
(
2009
).
Psychometric evaluation of the Insomnia Symptom Questionnaire: A self-report measure to identify chronic insomnia
.
Journal of Clinical Sleep Medicine
,
5
,
41
51
.

Owens
J. A.
,
Spirito
A.
,
McGuinn
M.
(
2000
).
The Children's Sleep Habits Questionnaire (CSHQ): Psychometric properties of a survey instrument for school-aged children. Sleep.
23
,
1043
1051
.

Perlis
M. L.
,
Jungquist
C.
,
Smith
M. T.
,
Posner
D.
(
2005
).
Cognitive Behavioral Treatment of Insomnia: A Session-By-Session Guide
.
New York, NY
:
Springer Science+Business Media, LLC
.

Pollock
J. I.
(
1994
).
Night-waking at five years of age: Predictors and prognosis
.
Journal of Child Psychology and Psychiatry
,
35
,
699
708
.

Reid
G. J.
,
Hong
R. Y.
,
Wade
T. J.
(
2009
).
The relation between common sleep problems and emotional and behavioral problems among 2- and 3-year-olds in the context of known risk factors for psychopathology
.
Journal of Sleep Research
,
18
,
49
59
. doi: 10.1111/j.1365-2869.2008.00692.x

Reite
M.
,
Buysse
D.
,
Reynolds
C.
,
Mendelson
W.
(
1995
).
The use of polysomnography in the evaluation of insomnia
.
Sleep
,
18
,
58
70
.

Sadeh
A.
,
Gruber
R.
,
Raviv
A.
(
2002
).
Sleep, neurobehavioral functioning, and behavior problems in school-age children
.
Child Development
,
73
,
405
417
.

Sateia
M. J.
(
2014
).
International classification of sleep disorders-third edition: Highlights and modifications
.
Chest
,
146
,
1387
1394
. doi: 10.1378/chest.14-0970

Standards of Practice Committee of the American Sleep Disorders Association
(
1995
).
Practice parameters for the use of polysomnography in the evaluation of insomnia
.
Sleep
,
18
,
55
57
.

Touchette
E.
,
Petit
D.
,
Seguin
J. R.
,
Boivin
M.
,
Tremblay
R. E.
,
Montplaisir
J. Y.
(
2007
).
Associations between sleep duration patterns and behavioral/cognitive functioning at school entry
.
Sleep
,
30
,
1213
1219
.