Randomized Controlled Trials
As part of the independent evaluation of U.S. Department of Education grants, our researchers have examined the effectiveness of innovative technology in classrooms. By randomly assigning treatment and control groups, monitoring implementation, and measuring results, the product developers and funders have a much better idea of what works well, for whom, and under what circumstances. We have measured student achievement outcomes through standardized reading tests and curriculum-based knowledge tests. These studies also have included classroom observations, teacher implementation logs, and close-out focus groups for formative feedback.

Action Research for Improving Classroom Practice
As evaluators for a Technology Innovation Challenge Grant, Macro developed an online data collection tool for Maryland teachers involved in action research. The initial purpose of the tool was to assist teachers in implementing classroom-based research by providing a systematic approach to project planning and data collection. The tool also allowed Macro to monitor and evaluate the effectiveness of a professional development model that uses action research. More than 250 educators who received professional development through the grant used this tool. Macro has revised this tool for wider dissemination to educational systems in Maryland and other States for use in assessing the effectiveness of school improvement strategies.

Studying the Use of Technology in Schools
Macro completed the fourth in a series of studies that a public broadcasting organization conducted to determine the use of television, video, and other technologies for educational purposes in elementary and secondary schools in the United States. Building on data that have been collected since 1977, Macro investigated the availability, use, support, and outcomes of educational technology through a nationally representative survey of schools and teachers. A module was added to the questionnaire to collect information about the use of other new technologies, especially the use of computers in multimedia and online applications. Another important feature that we added was qualitative telephone interviews with a subset of teachers (10 percent) to provide greater in-depth understanding of the classroom application of these tools. The results were published in a research volume for use by educators and policymakers in shaping budgets and policy. An attractive summary document that combined findings, pictures, and graphics was distributed to schools, public television stations, and Congress. This summary report of findings is published online, and the research team gave several presentations to public television audiences.

Cross-State Adult Education Data and Research Evaluation
For the U.S. Department of Education’s Program and Policy Studies Service, Macro performed secondary data analysis on adult education students in three States. Attendance, instruction, achievement, and economic outcomes were evaluated. To accomplish this, we obtained complete State records for 4-year adult education programs and created longitudinal transcripts for each student for each year of study. Previously, adult education programs only provided limited yearly summaries of student performance, so merging multiple years of information required us to deal with many inconsistencies in records. The result was a unique perspective on student educational progress and innovative uses of longitudinal student data. The study served to verify that the quality of student data in selected States that are known for high-quality management information systems was adequate to address research questions, to provide examples of analyses that can be accomplished, and to determine student outcomes in terms of learning gains, persistence, employment, and earnings.