blog posts and news stories

Welcome On Board, John Easton!

The Obama administration has named John Q. Easton, executive director of the Consortium on Chicago School Research, as the new director of the Institute of Education Sciences. The choice will bring a new perspective to IES. The Consortium provides research support for local reforms and improvements in the Chicago Public Schools. While Dr. Easton is quoted in Education Week as saying he will retain the rigor that IES has made a priority, the Consortium’s work points to the importance of building local capacity for research to support reform. In a paper published online, Roderick, Easton, & Sebring (2009), he and his colleagues provide a clear and detailed rationale for their approach that includes the need for combining high quality research with the ability to cut through technical details to communicate both good and bad news to decision-makers. Empirical Education congratulates John Easton, and we look forward to working with him to replicate this model for research in school districts throughout the country.

Consortium on Chicago School Research: A New Model for the Role of Research in Supporting Urban School Reform, ²⁄₂₀₀₉. Melissa Roderick, John Q. Easton, and Penny Bender Sebring.

American Recovery and Reinvestment Act

In response to the recent announcements of education stimulus funding through the American Recovery and Reinvestment Act (ARRA), Empirical Education is now offering assistance to state and local education agencies in building local capacity for rigorous evaluations. Our MeasureResults system helps state and district planners answer questions about the effectiveness of schools and of locally implemented programs. The enhanced analytic capabilities built into the MeasureResults system enables school districts to conduct their own rigorous research with findings relevant to their localized setting, using their existing datasets. Read more about how Empirical Education can work with state and local education agencies to take full advantage of ARRA funding. Also see how we partner with publishers to provide research packages along with their product offerings.

Empirical Education Partners with NWEA to Research Virtual Control Groups

Northwest Evaluation Association, the leading provider of computer adaptive testing for schools, is partnering with Empirical Education to analyze the properties of its virtual control group (VCG) technologies. Empirical has already conducted a large number of randomized experiments in which NWEA’s “Measures of Academic Progress” (MAP) served both as pretest and posttest. The characteristics of a randomly assigned control group provide a yardstick in evaluating the characteristics of the VCG. The proposed research builds on extensive theoretical work on approaches to forming comparison groups for obtaining unbiased impact estimates from quasi-experiments.

In parallel to this theoretical analysis, NWEA and Empirical Education are cooperating in a nationwide comparison group (“quasi-”) experiment to estimate the impact of a basal reading program in wide use nationally. Taking advantage of the fact that MAP is in use in thousands of schools, Empirical will identify a group of schools currently using this reading program and testing their students’ reading using MAP and then select a well matched comparison group from non-users who also test with MAP. Characteristics of the schools such as SES, percent English learner, urbanicity, ethnicity, and geographic region, as well as prior reading achievement, will be used in identifying the comparison group.

Presentation at the Society for Research in Educational Effectiveness (SREE) Explores Methods for Studying Achievement Gaps

Frequently in Empirical Education’s experimental evaluations for school districts, the question of local concern is an achievement gap identified between two student groups. The analysis of these experiments also often finds significant differences between these subgroups in how effective the intervention was (that is, if it increased or decreased the gap) while not finding a significant overall difference. In his 2005 book, Howard Bloom suggested why there may be more statistical power to detect subgroup differences than to detect the average effect. The exploration presented at SREE, which was held in Washington March 1-3, examined the statistical characteristics of eight experiments conducted over the last three years to find out whether a critical assumption of Bloom’s approach held. His assumption is that the average performance gap does not vary across the units that are randomized. The work, led by Andrew P. Jaciw, Empirical Education’s Director of Experimental Design and Analysis, found that the assumption held. This finding is important because it suggests that local experiments focusing on achievement gaps may be less expensive than experiments addressing only the overall average effect of an intervention. (Click here for a copy of the poster and handout.)

Bloom, H. S., (2005). Randomizing groups to evaluate place-based programs. In H. S. Bloom (Ed). Learning More From Social Experiments. New York, NY: Sage.

Methods for Local Experimental Evaluation of STEM Initiatives Presented to State Legislators

The National Conference of State Legislatures (NCSL) presented a seminar for education committee chairs January 9-11 in Huntsville, Alabama, home of NASA’s Marshall Space Flight Center. The topic was “Linking Research and Policy to Improve Science, Technology, Engineering and Math Education.” Empirical Education‘s president, Denis Newman, presented the company‘s research on the Alabama Math Science and Technology Initiative, an 80-school randomized experiment being conducted as part of its contract with the Regional Education Laboratory for the Southeast. The presentation also drew on findings from experiments the company has conducted to evaluate STEM initiatives elsewhere in the country to illustrate the importance of local research goals and characteristics in evaluation design. The seminar was part of a series on research funded by the Institute of Education Sciences. (Click here for a copy of the presentation.)

New MeasureResults® Webpage

Empirical Education clients and partners can now keep track of the latest MeasureResults developments on our new webpage. The webpage features video clips that explain how MeasureResults’ online interface can be used to set up and conduct school level studies, as well as details about the design and development of program features. Updates and announcements will be posted on a regular basis. In addition to serving our existing MeasureResults subscribers, Empirical Education is currently developing a custom-designed MeasureResults tool for use in more than ten school districts for a high-school math effectiveness study.