UC Merced

SCDs assess intervention effects by measuring units repeatedly over time in both the presence and the absence of treatment. Traditionally, SCDs are assessed using visual analysis. However, this ignores any potential trend, or systematic change over time, that might occur in the data. Proper treatment of trend - both the discovery whether or not it is present, and the form it takes, are essential steps before assessing the relationship between covariates, such as a treatment effect, and the outcome of interest. Generalized Additive Models, a semi-parametric extension of the Generalized Linear Model, are proposed as an effective way of measuring the trend, regardless of the functional form, and simultaneously measuring other covariates, such as the treatment effect. This dissertation uses a simulation study to assess the performance of GAMs, as compared to GLM, in terms of bias, efficiency, Type I error rates, and proportion of significance of the estimation of treatment level effects. In addition, we assess the estimation of level of nonlinearity within the GAMs, as well as the performance of nested χ2 tests as measures of model fit. I generated several different levels of data trajectories (no trend/interaction, linear trend/interaction, partial trend /interaction- an exponent of 3.5, and septic trend/interaction). The partial condtions (both trend and interactions) resulted in unexplicable results, and so were excluded from furthe analyses. In addition, the third chapter of this dissertation is an thurough look at this subsection of results, exploring several potential problems to troubleshoot what happened. Overall, There was no perfect model to capture the full complexity of these datasets. GLMs tended to have less bias in the estimation of the level change treatment effect, although this difference was very small in the larger time point conditions (i.e. 20 or more time points). However, GLMs were unable to recover the nonlinear trajectory in any datasets generated with a nonlinear DGM, which would make the overall assessment of the effect of the treatment over time (via the interaction term) suspect. However, not even the GAMBs estimate the true level of nonlinearity accurately (assessed by estimated degrees of freedom) when the true nonlinearity is very high (e.g. septic). The same pattern is true of the RMSE results. The GLM is generally more efficient than the GAMB, but the difference between the two models becomes small in the longer time series. Similarly, the Type I error rates are generally higher in GAMB models, but the overall rates are fairly low. GAMB models were better at capturing a true treatment effect (in terms of statistical significiance) than GLMs. So even though the level change treatment effect may be slightly less biased in a GLM than in a GAMB. they are less likely to be significant, leading to different decisions and model interpretation, based on analytic model choice. Of course, if the treatment effects produced by the GAMB are severely biased, this is not a positive finding. But, when the number of time points is greater than 20, neither analytic model is largely biased, and in these cases, the differences in significance become more important. These results are discussed in detail in the second chapter of this dissertation. Finally, the fourth chapter shows applications of GAMs to both a SCD as well as two longitudinal datasets other than SCDs, to demonstrate the versatility of GAMs.