Evaluation of the Comprehensive Community Mental Health Services for Children and Their Families Program
Macro’s evaluation of systems of care for the Center for Mental Health Services has resulted in an extensive series of data sets that require the use of complex and combined multivariate analytic techniques. The data gathered for the child and family outcome study are collected longitudinally and analyzed employing generalized linear models, fixed effects and random effects regression models. Many indicators collected through evaluation are categorical and require the utilization of such techniques as logistic regression, multinomial logistic regression, and ordered Probit regressions. Person- and item-level data reduction techniques, including latent class analysis, factor analysis, and cluster analysis, are often utilized to better understand the interrelations within and across construct indicators. We collect both process and outcome measures at the system and client levels, and use this information in multilevel analyses that model the degree to which community- and system-level change influences child and family outcomes (i.e., hierarchical linear modeling). Survival analysis methods are employed to look at various administrative data and agency records (i.e., service records, criminal records) that are episodic in nature. We use log-transformed and gamma regression models to analyze cost data that are often skewed. The nature and structure of the various national evaluation data sets has required a sophisticated and state-of-the-art approach to creative and cutting edge data analytic techniques.

Evaluation of Substance Abuse Prevention Initiatives for High-Risk Youth
Macro employed a multi-stage analysis plan to evaluate a demonstration program aimed at preventing alcohol, tobacco, and illicit drug use among at-risk youth. Using a quasi-experimental design, we compared some 6,000 program participants with 4,000 comparison youths in 48 demonstration programs. The analysis included reliability and validity of the instrument scales, cross-site measurement stability, attrition analysis, correlational analysis of baseline variables, and treatment and control group equivalence analysis. We analyzed each data set—participant surveys, individual exposure to program activities, and program characteristics—and then the integrated data set. Analysis of the integrated data set involved multiple regression, LISREL, structural equation modeling, and hierarchical linear modeling.

Customer Satisfaction Survey
Macro performs quarterly statistical analyses of a customer satisfaction survey for the Defense Logistics Agency (DLA). We provide overall results along with results for each of 10 service units. Our analysis includes basic descriptive statistics, cross-tabulation, regression and correlation analyses, and cluster and factor analyses as indicated by our client’s needs. In addition, we provide the DLA with our proprietary Customer Loyalty Plus® approach, which allows the client to efficiently set priorities and allocate resources to improve customer satisfaction. To complement the statistical analysis, we code respondents’ verbatim comments using Macrosaic©, which analyzes the comments by frequency, relates the data to the quantitative data, and generates descriptive reports.