Anthony Coetzer-Liversage*, Aradhana Srinagesh*, Shayna S. Bassett*,†, Manshu Yang*, Mary Clair-Michaud‡, Rosemarie A. Martin¶, Damaris J. Rohsenow¶, Christopher W. Kahler¶, Peter M. Monti¶, Warren Hurlbut§ and L. A. R. Stein*, ¶, ‡
ABSTRACT
Group interventions are commonly used for treating youth. Assessing group processes during intervention has presented challenges. The present study evaluated the psychometric properties of a group process-group level measure. This measure examined behaviours (e.g., positive or deviant) of incarcerated youth during group substance use interventions. Adolescents and counsellors completed a series of questions after each group session (n = 584 sessions). Observers rated group behaviour from 153 video-recorded sessions. The results supported internal consistency (median α = 0.78), inter-rater reliability (median α = .63), and validity (e.g., r = 0.12–0.82, p < 0.01). Importantly, a measure with demonstrated reliability and validity at the group level can be part of quality control for researchers and practitioners when individual-level ratings are not needed or too costly.
Key Words Group assessment; group psychotherapy; substance use; deviancy training; youth.
Group-based peer (GBP) interventions are the predominant approach for treating adolescents with substance use issues (Wendt & Gone, 2018). These interventions are favoured due to efficient resource utilization, facilitation of crucial developmental tasks (e.g., emotion regulation; interpersonal skills), creation of environments mirroring youths’ daily lives, and perception of being less intimidating compared to individual therapy (Engle et al., 2010).
Despite widespread use of group treatments for youth, there is conflicting evidence regarding their efficacy. Hogue et al. (2018) conducted a review of youth interventions and found no support for concerns in the literature, suggesting that group treatment might be harmful. They also highlighted the likely effectiveness of group-based cognitive behavioural therapy (CBT). However, in a subsequent review, Hogue et al. (2021) cautioned that groups could have iatrogenic effects if youth are encouraged to express and reinforce non-normative or deviant behaviour by their peers. Peer acceptance is crucial during adolescence, and behaviours modelled and reinforced by peers significantly influence adolescents’ own behaviours. Thus, group activities can profoundly impact outcomes for youth, with the behaviour of group leaders playing a crucial role in mitigating deviant behaviour (Hogue et al., 2021). The conflicting evidence underscores the importance of investigating the underlying mechanisms of group change and peer influence in group treatment to minimize unintended negative effects and optimize interventions (Kaminer, 2005).
Assessing group processes has been challenging due to the adaptation of assessments originally designed for individual psychotherapy, such as the Working Alliance Inventory (Horvath & Greenberg, 1989) and the Empathy Scale (Persons & Burns, 1985) (Jensen et al., 2012). While these tools capture elements of the client-provider relationship, they lack validation for understanding group dynamics, including peer relationships. Additionally, some measures (e.g., Group Climate Questionnaire-Short Form (MacKenzie, 1983); Curative Climate Instrument (Fuhriman et al., 1986); Groupwork Engagement Measure (Macgowan, 2006)) have psychometric support but were not specifically validated for use with youth, overlooking adolescent development and peer influence (Dishion et al., 1999, 2001). Adolescents’ social behaviours are significantly influenced by peer interactions (Spear, 2000; Steinberg, 2008), where association with peers engaging in problematic behaviour can escalate it (Utržan et al., 2017), while exposure to prosocial peers can mitigate anti-social behaviours (Dishion et al., 1999). Hence, a tailored measure of group processes considering youth developmental stage and peer influence is imperative (Dishion et al., 1999, 2001).
Dishion et al. (2001) used an observational measure to assess group processes in 12 sessions of CBT for youth. They employed a Likert scale (0 = not observed to 4 = very true for client) to code deviancy training, positive group involvement, peer rejection, and counsellor praise. Connectedness to peer counsellors was assessed by recording the average number of older peer counsellors with whom the client had a positive relationship across the sessions. Reliability was measured with split-half reliabilities ranging from 0.53 to 0.87 (p < 0.001), while internal consistencies ranged from α = 0.86 to 0.89 (none reported for connectedness or counsellor praise). Stein et al. (2014) improved on this measure by creating versions for youth and counsellor, providing detailed validity data, and simplifying coding to only the third and last sessions out of 10 GBP sessions for incarcerated youth. However, both measures code youth at the individual level (i.e., behaviour of specific youth, as compared to behaviour of the group). Ability to efficiently code group behaviour as a whole may have appeal for stakeholders who wish to ask broader programming questions without attention to specific youth (e.g., are groups functional; are groups decompensating?). Simply stated, an individual-level point of reference tracks performance of individual youth, whereas a group-level point of reference tracks performance of the group.
Despite progress toward developing measures for group process, additional research is needed to assess group-based treatment in youth correctional facilities. Therefore, the purpose of the present study was to examine reliability and validity of a group process measure modelled after the one used by Dishion et al. (2001). Adolescence is a sensitive developmental period, and although counsellors may provide contingent praise and attention to promote positive outcomes in youth (Handwerk et al., 2000) or offset peer rejection (Dishion et al., 2001), peers may selectively reinforce deviant behaviour or ignore such behaviour in favour of more normative or prosocial behaviour (Dishion et al., 1999). While the measure described above by Dishion et al. (2001) accounts for both peer socialization and counsellor behaviours, it was developed for at-risk (e.g., referrals by teachers) as compared to incarcerated youth; individual youth were coded on each of the 12 sessions by an observer at three 15-minute increments; and no formal validity studies were conducted on the measure. A measure that may use counsellor and youth reports in addition to observers, which is coded only once at the end of a session, and that is validated holds promise to better understand groups and maintain their effectiveness. The present study evaluated the psychometric properties of a group process-group level (GP-GL) measure. Coding for group behaviour as a whole (e.g., amount of deviancy or prosocial behaviour in a group), as compared to coding degree of anti- or pro-sociality in individual youth, may assist stakeholders to efficiently evaluate and maintain group programming.
Data were collected as part of an institutional review board-approved randomized clinical trial comparing two group-based treatments for incarcerated adolescents with a history of alcohol and/or cannabis use (Stein et al., 2014, 2020). Participants, recruited from a Northeastern United States youth correctional facility, faced charges ranging from truancy to violent offences. These adolescents, 14 and 19 years old and sentenced to the facility for 4–12 months, received comprehensive services, including group and individual treatment on a variety of topics. Consent procedures involved obtaining assent from youth aged below 18 years and consent from legal guardians.
The eligibility criteria were as follows: prior to incarceration used (1) cannabis or drank alcohol at least monthly or binge drank (≥5 standard drinks for boys, ≥4 for girls) at least once, (2) cannabis or drank in the 4 weeks before the offence, or (3) cannabis or drank in the 4 weeks before incarceration.
The baseline sample (N = 205) of participants is described as follows: 40% Hispanic, 39.5% African American, 35.1% White, 8.8% Native American, 3.9% Pacific Islander, 3.9% Asian American, 7.3% self-identified as other; 89.3% male; mean (M) age = 17.1 years (standard deviation (SD) = 1.0); M = 2.5 (SD = 2.3) weeks previously detained/incarcerated; and 32.2% and 61.5% alcohol and cannabis dependent, respectively (American Psychiatric Association, 2000).
Following baseline assessment, adolescents were randomly assigned to either 2 sessions of motivational interviewing followed by 10 CBT group sessions or 2 sessions of relaxation training followed by 10 Substance Education and Twelve-Step Introduction (SET) group sessions. A total of 584 group sessions were conducted with fidelity procedures to ensure adherence to treatment protocols. These gender-segregated groups, lasting approximately 75 minutes, utilized rolling admission, and occurred one to three times weekly, with two to seven participants (median of three) in each group. See the study by Stein et al. (2020) for more information about treatments and fidelity procedures.
Counsellors (n = 6) underwent approximately 160 hours of manualized training, covering readings, role-plays, and feedback for both intervention types. One had a bachelor’s degree, four had a master’s degree, and one had a doctoral degree. A doctoral-level clinical psychologist provided weekly supervision to all counsellors and reviewed all study intervention files. All group sessions were videotaped, and fidelity coding was performed proactively throughout the study. Counsellors with fidelity scores below threshold received additional individual supervision and tracking until acceptable fidelity was achieved.
GP-GL questionnaire, is an adaptation of Dishion et al.’s (2001) original measure, featuring five scales and eight associated indices. Scales are as follows: (1) Deviancy training (six items) measures extent to which youths received group attention for problem behaviour (e.g., “members gave each other explicit attention for counter-norm talk or problem behaviour”); (2) Positive group involvement (four items) describes engagement with session curriculum (e.g., members spent group time on normative talk); (3) Peer rejection (three items) reflects extent to which youths displayed rejecting behaviours to one another (i.e., members negatively interacted or seemed to reject each other); (4) Counsellor praised positive behaviour (three items) describes extent to which youths were encouraged or praised by the counsellor for positive behaviour or commitment to prosocial goals (i.e., extent to which members were personally encouraged by counsellor); (5) Connectedness to counsellor – counsellor/observer version, indicates the number of youths who seemed to have a positive relation with the counsellor; and Connectedness to counsellor – adolescent version (one item), describes extent to which members seemed to have a positive relationship with the counsellor. Items were rated from 0 (no examples, not observed in group) to 4 (multiple examples, very true for group). Means are obtained per scale. Better group behaviour is indicated by lower scores on Deviancy and Peer rejection and higher scores on positive group involvement, Counsellor praised positive behaviour, and Connectedness to counsellor.
Regarding the eight indices, counsellor/observer forms requested the number of youth per group and the number of youth (1) who gave attention to/received attention for counter-norm behaviour (Deviancy), (2) who gave attention to/received attention for normative or positive behaviour (Positive involvement), (3) who appeared to reject/be rejected by other group members (Peer rejection), (4) who the counsellor gave attention for normative or positive behaviour (Counsellor praised positive behaviour), and (5) who the counsellor gave attention for counter-norm behaviour. These questions produced indices reflecting percentages of youth engaging in each behaviour (e.g., percentage who received attention for positive behaviour).
The form was completed following each group session by the counsellor, and a youth (randomly chosen by a project director before the session) who reported their impressions of the session. All sessions (n = 584) were video-recorded, and observer supervisors completed a subset of sessions (n = 153) including double-coding (n = 64).
Counsellors completed the grid (Sampl & Kadden, 2001) at the end of every group session, rating each session overall for quality and quantity of group participation (0 = none to 3 = high). After each session, counsellors rated individual youth in terms of clinical status (1 = poor to 5 = excellent), number of disruptive behaviours (aggressive, interrupts, profanity, sexually inappropriate, glorifying drug use, etc.), and quality/quantity of participation. Average rating among youth was calculated per session for clinical status, disruptive behaviours, and for quality and quantity of participation.
Group-level analyses were conducted using SPSS 22.0.0 (IBM, Armonk, USA). Internal consistencies (α) were calculated for scales. Convergent validity was assessed by correlating scores across respondent versions. Expected positive and negative relationships between scales were examined. Intra-class correlation coefficients (ICCs) were calculated for inter-rater reliability (Cicchetti, 1994; Koo & Li, 2016). Analysis of variance (ANOVA) was used to compare group treatments (CBT vs. SET). Effect sizes (η2) were reported. No specific hypotheses were established. Group variables were examined as covariates, but none met criteria for inclusion. Refer to the supplement for detailed analysis.
Internal consistencies were calculated for the Deviancy, Positive involvement, Peer rejection, and Counsellor praise scales (Table I). The resulting α coefficients ranged from very good to good (Ursachi et al., 2015): Deviancy (α = 0.97, 0.97, and 0.82 for observer (O), counsellor (C), and adolescent (A), respectively), Positive involvement (α = 0.75, 0.90, and 0.74 for O, C, and A, respectively), Peer rejection (α = 0.76, 0.89, and 0.86 for O, C, and A, respectively), and Counsellor praise (α = 0.74 and 0.74 for O and A, respectively). However, Counsellor praise showed poor internal consistency for counsellor version (α = 0.47).
Correlations between scales and indices that should be related are presented in Tables II and III. Effect sizes were interpreted as follows: small, r = 0.10; medium, r = 0.30; and large, r = 0.50 (Cohen, 1988). In Table II, substantial correlations were observed across observer, counsellor, and adolescent respondents in the Deviancy scale, with noteworthy correlations with both counsellor and observer indices (e.g., percentage of youth who got reinforced for anti-social behaviour according to counsellor correlated with the observer Deviancy scale with r = 0.64). Similarly, the Peer rejection scale demonstrated noteworthy correlations with indices. Correlations were somewhat more moderate for Positive involvement and Counsellor praise scales with indices. Over 70% of significant correlations were of medium effect size or better. In Table III, the Connectedness to counsellor scale evidenced noteworthy correlations with both counsellor and observer indices, though this was moderated for youth ratings. Over 60% of significant correlations were of medium effect size or better. Table IV contains correlations among scale scores across adolescent, counsellor, and observer versions, demonstrating medium to large correlations across the three versions within a scale (i.e., same-trait-different-method correlations). However, there were low correlations between adult and adolescent ratings for Counsellor praise and Positive involvement.
TABLE II Correlations between scales and indices
TABLE III Correlations between indices and connectedness to counsellor scale
TABLE IV Correlations among scales
Table V contains correlations between scale scores and counsellor reports using the Group Report Grid, showing over 50% of significant correlations were of medium effect size or better. Consistent validity support was observed for Deviancy, Positive involvement, and Peer rejection, some support for Counsellor connection, and relatively less support for Counsellor praise. Across scales, observer ratings demonstrated least support.
TABLE V Correlations among scales and Group Report Grid constructs (n = 584)
Table VI contains ICC observer results for scales and indices; the median ICCs were 0.54 and 0.64, respectively (fair to good; Cicchetti, 1994).
TABLE VI Intra-class correlation coefficients (ICCs) for observer ratings (n = 64)
Adolescent scales (Table VII) showed no significant effects. However, for counsellors and observers, CBT groups produced significantly higher Deviancy scores compared to SET groups, while SET groups had higher scores on the Positive involvement scale than CBT groups. Moreover, compared to SET, CBT had significantly higher percentages of adolescents reinforcing or being reinforced for anti-social behaviour. For counsellor indices, CBT had significantly higher percentages of adolescents reinforcing positive behaviour, and experiencing peer rejection, compared to SET. In addition, for observers, CBT was associated with higher Peer rejection scale scores and higher percentage of youth reinforced for anti-social behaviour by counsellors, compared to SET. Generally, effect sizes were larger for observers than counsellors, with counsellors’ effect sizes being small.
TABLE VII Between-group ANOVA results
This study provided psychometric analyses of a group process measure using five scales and eight indices. Internal consistencies (Table I) ranged from acceptable to very good (α = 0.74–0.97) except for Counsellor praise as rated by counsellors (α = 0.47). Scales generally related well to similar constructs across versions (Tables II and III), and scales (Table IV) related as expected to each other (e.g., adolescent ratings of Positive involvement correlated well with observer ratings of Counsellor praise), indicating good convergent validity.
When relating scales to counsellor ratings of behaviours following sessions (e.g., quality of involvement), observer ratings of Peer rejection, Positive involvement, and Deviancy showed little criterion-related validity (Table V). On the other hand, adolescent and counsellor ratings on these scales evidenced good validity (e.g., adolescent and counsellor ratings on Deviancy correlated inversely with quality of participation). Adolescent ratings of Counsellor praise and Connection did not show good validity evidence as compared to observer ratings for these scales. Counsellor ratings of Counsellor praise also evidenced little validity, whereas counsellor ratings of Connection showed validity (e.g., Connection was correlated inversely to session misbehaviours). Agreement between observers (Table VI) was fair to excellent across scales and indices.
Compared to SET, both observers and counsellors had higher deviancy scores for CBT groups (Table VII), consistent with those in the study by Dishion et al. (2001). Only observers reported counsellors reinforcing anti-social behaviour in CBT as compared to SET, which may (with replication) be noteworthy for supervision. This aligns with findings suggesting therapists may struggle to detect certain behaviours compared to observers (Carroll et al., 2000). Only counsellor estimates for percentages of youth reinforcing positive behaviour and rejecting peers were higher in CBT than in SET, though effect size was small. While this suggests counsellors are more sensitive than observers in detecting such processes, further research is needed. Adolescent ratings of group process did not meaningfully differ between groups.
The results resemble those of Engle et al. (2010) who analyzed a group-based adolescent substance intervention and found associations between leader empathy (related to connection or alliance; Horvath, 1994) and participant behaviours (both deviant and positive). While their approach began with more labour-intensive coding of individual youth, our measures directly coded group-level data incorporating scales for rejection and counsellor praise.
Some observer correlations were not statistically significant, possibly due to a smaller sample of observer ratings (n = 153) compared to counsellor and adolescent ratings (n = 584). However, larger samples may have led to significant results, prompting inclusion of effect sizes to enhance interpretation. Future research could explore additional items for assessing connectedness, in the GP-GL measure, which currently includes only one item for Connection to Counsellor. While this study was focused on validating the GP-GL measure, significant relationships between constructs suggest further research, including examining predictive validity and applying the measure to other youth interventions and settings. Additionally, reliability data for counsellor ratings on Group Report Grid were unavailable, though counsellors received training and supervision.
The GP-GL measure offers a streamlined approach for assessing group dynamics in adolescent substance use treatment, particularly in resource-limited settings. Utilizing this measure can improve the efficiency and effectiveness of group interventions for adolescents with substance use issues (Hogue et al., 2021; Jensen et al., 2012).
Analyses for all three versions indicated sound psychometric properties for the GP-GL measure. If pressed for counsellor time, adolescent ratings may suffice, although somewhat stronger validity was found for counsellor and observer versions. In choosing between observer and counsellor versions, settings with limited resources may choose the counsellor version to reduce time needed from an observer. In these cases, in order to encourage honest responding, it is important that counsellors feel supported and that ratings are not to be used punitively.
Supplemental information is linked to the online version of the paper at https://www.journalcswb.ca/index.php/cswb/article/view/384/supp_material.
This work was supported by the National Institute on Drug Abuse under grant number R01 DA-018851 (PI, Stein).
The authors have no conflicts of interest to declare.
This work is the sole responsibility of the authors and does not necessarily represent the official views of the National Institutes of Drug Abuse.
This study was approved by the Institutional Review Board of the University of Rhode Island, Kingston, RI 02881.
All procedures in this study were conducted in accordance with Institutional Review Board of the University of Rhode Island, Kingston, RI 02881.
All study participants, or their legal guardian, provided informed written consent prior to study enrollment.
*Social Sciences Research Center, University of Rhode Island, Kingston, RI 02881, USA;
†Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA;
‡Department of Children, Youth & Families, The Rhode Island Training School, Cranston, RI 02920, USA;
¶Center for Alcohol and Addiction Studies, Brown University School of Public Health, Providence, RI 02903, USA;
§Administration of Justice, Salve Regina University, Newport, RI 02840, USA.
Supplemental information is linked to the online version of the paper at https://www.journalcswb.ca/index.php/cswb/article/view/384/supp_material.
American Psychiatric Association (APA). (2000). Diagnostic and statistical manual of mental disorders: DSM-IV-TR. American Psychiatric Association.
Carroll, K. M., Nich, C., Sifry, R. L., Nuro, K. F., Frankforter, T. L., Ball, S. A., Fenton, L., Rounsaville, B. J. (2000). A general system for evaluating therapist adherence and competence in psychotherapy research in the addictions. Drug and Alcohol Dependence, 57, 225–238. https://doi.org/10.1016/S0376-8716(99)00049-6
Crossref PubMed
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. https://doi.org/10.1037/1040-3590.6.4.284
Crossref
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.
Dishion, T. J., McCord, J., Poulin, F. (1999). When interventions harm: Peer groups and problem behavior. American Psychologist, 54, 755–764. https://doi.org/10.1037/0003-066X.54.9.755
Crossref
Dishion, T. J., Poulin, F., Burraston, B. (2001). Peer group dynamics associated with iatrogenic effects in group interventions with high-risk young adolescents. In D. W. Nagle C. A. Endley (Eds.), The role of friendship in psychological adjustment (pp. 79–92). Jossey-Bass.
Engle, B., Macgowan, M., Wagner, E., Amrhein, P. (2010). Markers of marijuana use outcomes within adolescent substance abuse group treatment. Research on Social Work Practice, 20(3), 271–282. https://doi.org/10.1177/1049731509347855
Crossref
Fuhriman, A., Drescher, S., Hanson, E., Henrie, R., Rybicki, W. (1986). Refining the measurement of curativeness: An empirical approach. Small Group Behavior, 17, 186–201. https://doi.org/10.1177/104649648601700204
Crossref
Handwerk, M., Field, C., Frisman, P. (2000). The iatrogenic effects of group intervention for antisocial youth. Journal of Behavioral Education, 10(4), 223–238. https://doi.org/10.1023/A:1012299716053
Crossref
Hogue, A., Henderson, C., Becker, S., Knight, D. (2018). Evidence base on outpatient behavioral treatment for adolescent substance use, 2014–2017. Journal of Clinical Child and Adolescent Psychology, 47(4), 499–526. https://doi.org/10.1080/15374416.2018.1466307
Crossref PubMed PMC
Hogue, A., Henderson, C., Ozechowski, T., Becker, S., Coatsworth, D. (2021). Can the group harm the individual? Clinical Psychology: Science Practice, 28(1), 40–51. https://doi.org/10.1111/cpsp.12307
Horvath, A. O. (1994). Empirical validation of Bordin’s pan-theoretical model of the alliance: The Working Alliance Inventory perspective. In A. O. Horvath L. S. Greenberg (Eds.), The working alliance: Theory, research, and practice (pp. 109–130). Wiley.
Horvath, A. O., Greenberg, L. S. (1989). Development and validity of the Working Alliance Inventory. Journal of Counseling Psychology, 36, 223–233. https://doi.org/10.1037/0022-0167.36.2.223
Crossref
Jensen, D. R., Abbott, M. K., Beecher, M. E., Griner, D., Golightly, T. R., Cannon, J. A. N. (2012). Taking the pulse of the group: The utilization of practice-based evidence in group psychotherapy. Professional Psychology: Research and Practice, 43(4), 388–394. https://doi.org/10.1037/a0029033
Crossref
Kaminer, Y. (2005). Challenges and opportunities of group therapy for adolescent substance abuse: A critical review. Addictive Behaviors, 30, 1765–1774. https://doi.org/10.1016/j.addbeh.2005.07.002
Crossref PubMed
Koo, T. K., Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Crossref PubMed PMC
Macgowan, M. J. (2006). The Group Engagement Measure: A review of its conceptual and empirical properties. Journal of Groups in Addiction and Recovery, 1(2), 33–52. https://doi.org/10.1300/J384v01n02_04
Crossref
MacKenzie, K. R. (1983). The clinical application of group measure. In R. R. Dies K. R. MacKenzie (Eds.), Advances in Group Psychotherapy: Integrating research and practice (pp. 159–170). International Universities Press.
Persons, J. B., Burns, D. D. (1985). Mechanisms of action of cognitive therapy: Relative contribution of technical and interpersonal intervention. Cognitive Therapy and Research, 9, 539–551. https://doi.org/10.1007/BF01173007
Crossref
Sampl, S. Kadden, R. (2001). Motivational enhancement therapy and cognitive behavioral therapy for adolescent cannabis users: 5 Sessions, Cannabis Youth Treatment (CYT) Series, Volume 1. DHHS Pub. No. (SMA) 01-3486. Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration.
Spear, L. P. (2000). The adolescent brain and age-related behavioral manifestations. Neuroscience Biobehavioral Reviews, 24(4), 417–463. https://doi.org/10.1016/s0149-7634(00)00014-2
Crossref PubMed
Stein, L. A., Clair, M., Martin, R. A., Soenksen, S., Lebeau, R., Rohsenow, D. J., Kahler, C. W., Hurlbut, W., Monti, P. M. (2014). Measuring behaviors of individual adolescents during group-based substance abuse intervention. Substance Abuse, 35(4), 408–417. https://doi.org/10.1080/08897077.2014.949337
Crossref PubMed PMC
Stein, L. A. R., Martin, R., Clair-Michaud, M., Lebeau, R., Hurlbut, W., Kahler, C. W., Monti, P. M., Rohsenow, D. (2020). A randomized clinical trial of motivational interviewing plus skills training vs. Relaxation plus education and 12-Steps for substance using incarcerated youth: Effects on alcohol, marijuana and crimes of aggression. Drug and Alcohol Dependence, 207, 107774. https://doi.org/10.1016/j.drugalcdep.2019.107774
Crossref PubMed PMC
Steinberg, L. (2008). A social neuroscience perspective on adolescent risk-taking. Developmental Review, 28, 78–106. https://doi.org/10.1016/j.dr.2007.08.002
Crossref PubMed PMC
Ursachi, G., Horodnic, I. A., Zait, A. (2015). How reliable are measurement scales? External factors with indirect influence on reliability estimators. Procedia Economics and Finance, 20, 679–686. https://doi.org/10.1016/S2212-5671(15)00123-9
Crossref
Utržan, D. S., Piehler, T. F., Dishion, T. J. (2017). The role of deviant peers in oppositional defiant disorder and conduct disorder. In J. E. Lochman and W. Matthys (Eds.), The Wiley handbook of disruptive and impulse-control disorders. Wiley. https://doi.org/10.1002/9781119092254.ch21
Crossref
Wendt, D. C., Gone, J. P. (2018). Complexities with group therapy facilitation in substance use disorder specialty treatment settings. Journal of Substance Abuse Treatment, 88, 9–17. https://doi.org/10.1016/j.jsat.2018.02.002
Crossref PubMed PMC
Correspondence to: L. A. R. Stein, Department of Psychology, The University of Rhode Island, 130 Flagg Road, Kingston, RI 02881, USA. Telephone: 401.874.4261. E-mail: larstein@uri.edu
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. For commercial re-use, please contact sales@sgpublishing.ca.
Journal of CSWB, VOLUME 9, NUMBER 4, December 2024