Evaluating Georgia Part C Implementation Outcomes

Project ImPACT Coaching Fidelity

Providers' Project ImPACT therapy sessions will be video recorded and will be scored retrospectively for use of core Project ImPACT strategies. The research team will measure the project fidelity by watching recorded videos of the sessions, which will allow the team to capture how effectively the adult implements a strategy. This will be scored using the Project ImPACT fidelity checklist. Provider fidelity for each intervention strategy will be rated by a trained coder using a 5-point Likert scale, with scores of 4 or greater indicating appropriate fidelity. An average score of 4 or higher on all of the strategies indicates that fidelity has been met for Project ImPACT.

Provider Co-Production Quality

Following each therapy session, providers will complete a brief 20-item survey in which they will be asked what their session looked like and the rationale for any adaptations described. Providers will first indicate how long in minutes their Project ImPACT session was. They will also indicate which Project ImPACT coaching activities they completed in their session (yes/no). Providers will then be asked to indicate which of 10 additional non-Project ImPACT topics they covered in their session and subsequently how long in minutes they spent covering the additional topics. The following information will then be calculated for each session: 1) Percent of possible coaching elements delivered; 2) Percent of time spent covering additional topics; 3) Percent of time delivering other topics.

Project ImPACT Self-Efficacy.

Providers will rate their self-efficacy using a 7-item measure that asks about their perceived skills delivering Project ImPACT core strategies. Providers will rate each item using a 10-point Likert scale (1 = not at all confident, 5 = somewhat confident, and 10 = completely confident). An increase in the mean score indicates that the intervention has successfully enhanced self-efficacy.

Perceived Characteristics of the Intervention Scale (Project ImPACT Perceptions)

All providers will complete the Perceived Characteristics of the Intervention Scale (PCIS). The PCIS is a 12-item questionnaire that measures the perceived characteristics of an intervention that are thought to impact its uptake and use. The current study will use an abbreviated measure that includes the following dimensions: Relative Advantage (e.g., "Project ImPACT is more effective than other therapies I have used"), Compatibility (e.g., "Project ImPACT is aligned with my clinical judgement"), Complexity (e.g., "Project ImPACT is easy to use"), and Potential for Reinvention (e.g., "Project ImPACT can be adapted to meet the needs of my patients"). Participants will rate each item on a 7-point Likert scale (1 = Strongly disagree, 7 = Strongly agree). Higher scores generally indicate more positive perceptions of the intervention.

The Family Outcome Survey (caregivers)

The Family Outcome Survey (FOS) measures family outcomes expected to change through participation in early intervention programs for children with disabilities and their caregivers. The FOS includes five outcomes: understanding your child's strengths, needs, and abilities; knowing your rights and advocating for your child; helping your child develop and learn; having support systems; and accessing the community. The FOS comprises 24 items, with four to six items per outcome. Responses range from not at all (1) to completely (5). To get overall scores, all the responses are summed for the items within each of the five outcome areas and the three helpfulness indicators. The Early Childhood Technical Assistance (ECTA) Center recommends that scores of 5 or greater on the family outcome items be considered an indicator of outcome attainment. Scores below 5 may indicate areas requiring further improvement

Parent Observation of Social Interaction

Parent Observation of Social Interaction (POSI). The POSI is a 7-item measure that asks caregivers to rate their child's use of social communication skills, including communication, gesturing, and play, with each question scored as either 0 or 1. A score of 3 or more indicates a positive screen, suggesting the child may be at risk and warrants further evaluation.