MOST Full Factorial Analysis

A Multiphase Optimization Strategy (MOST) factorial analysis. Adapted from code from MS and based on Spring et al. (2020).

Important!!!

MS used mixed effects models for H4L, but for triple E, the engagement index (outcome) is only measured once as a composite of app engagement. There is no baseline EI measure and therefore no repeated-measures structure on the outcome

Triple E project employs the MOST framework using a 2^4 factorial experimental design analysed via Analysis of Covariance (ANCOVA).

The MOST approach is designed to evaluate individual intervention components before they are combined into a final ‘package’. This will allow us to isolate the efficacy of each strategy (gamification, health coaching, parental resources, and text messages), rather than the app as a whole.

Primary analysis will involve standard ANCOVA main effects because it allows us to compare mean scores on the primary outcome across experimental conditions while controlling for baseline variables. Instead of comparing all 16 arms against each other (which would require a very large sample size), the analysis compares aggregates of the experimental conditions.. for example the total effect of text messaging is determined by comparing the ~50% who received that strategy vs the ~50% who did not. By using the factorial ANCOVA approach, every participant’s data contributes to the estimate of every main effect. This helps to maintain the 95% power to detect medium effect sizes (0.26) with at least 258 participants. But to accomodate 30% attrition, 336 participants were aimed to be recruited