blog posts and news stories

Evidentally Rejoins Empirical Education

On 10/31/18, we formed Evidentally from a set of projects that Empirical Education called Evidence as a Service (EaaS). The idea was to build a set of products that would automate much of the labor-intensive portions of the research process, e.g., statistical analysis, data cleaning, and similar efforts. Building on the education technology (edtech) efficiencies, particularly the collection of edtech usage data, would make it possible for non-researchers such as school administrators to conduct efficacy research.

By lowering the cost of research and introducing an ease into the research process, we are able to increase the number of valid studies that could be combined in a meta-analysis for generalizable results. The notion was that, as a product company, Evidentally could attempt to get investments unavailable to services companies such as Empirical Education. Unfortunately, for several reasons, Evidentally was unable to get that investment.

The intellectual property and projects of Evidentally have been returned to Empirical Education as of 12/7/22, and Evidentally (as its own entity) was dissolved. While the team is still committed to building the Evidence as a Service suite of tools, it will be conducted as a project of Empirical Education, under the branding of the Evidentally Evidence Suite. The Evidentally product is just one piece of Empirical Education’s education evidence offerings, when an education application or curriculum needs evidence of efficacy meeting any of the Every Students Succeeds Act (ESSA) Tiers of evidence.

Studying the Impacts of CAPIT Reading: An Early Literacy Program in Oklahoma

Empirical Education’s Evidentally recently conducted a study to evaluate the impact of CAPIT Reading on student early literacy achievement. The study utilized a quasi-experimental comparison group design using data from 12 elementary schools in a suburban school district in Oklahoma during the 2019–20 school year.

CAPIT Reading is a comprehensive PK–2 literacy solution that includes a digital phonics curriculum and teacher professional development. The program is a teacher-led phonemic awareness and phonics curriculum that includes lesson plans, built-in assessments, and ongoing support.

Four schools used CAPIT to supplement their literacy instruction for kindergarten students (treatment group) while eight schools did not (comparison group). The study linked CAPIT usage data and district demographic and achievement data to estimate the impact of CAPIT on the Letter Word Sounds Fluency (LWSF) and Early Literacy Composite scores of the aimsweb reading assessment, administered by the district in August and January.

We found a positive impact of CAPIT Reading on student early reading achievement on the aimsweb assessment for kindergarten students. This positive impact was estimated at 4.4 test score points for the aimsweb Early Literacy Composite score (effect size = 0.17; p = 0.01) and 7.8 points for the LWSF score (effect size = 0.29; p < 0.001). This impact on the LWSF score is equivalent to a 29% increase in growth for the average CAPIT student from the fall to winter tests.

We found limited evidence of differential impact favoring student subgroups, meaning that this positive impact for CAPIT users did not vary according to student characteristics such as eligibility for free and reduced-price lunch, race, or gender. We found that the impact on aimsweb overall was marginally greater for special education students by 4.9 points (p = 0.09) and that the impact on LWSF scores was greater for English Language Learners by 7.4 points (p = 0.09). Impact of CAPIT reading does not vary significantly across other student groups.

Read the CAPIT Reading Student Impact Report for more information on this early literacy research.

Jefferson Education Accelerator Contracts with Empirical for Evidence as a Service™

Jefferson Education Accelerator (JEA) has contracted with Empirical Education Inc. for research services that will provide evidence of the impact of education technology products developed by their portfolio companies. JEA’s mission is to support and evaluate promising edtech solutions in order to help educators make more informed decisions about the products they invest in. The study is designed to meet level 2 or “moderate” evidence as defined by the Every Student Succeeds Act. Empirical will provide a Student Impact Report under its Evidence as a Service offering, which combines student-level product usage data and a school district’s administrative data to conduct a comparison group study. Denis Newman, Empirical’s CEO stated, “This is a perfect application of our Evidence as a Service product, which provides fast answers to questions about which kids will benefit the most from any particular learning program.” Todd Bloom, JEA’s Chief Academic Officer and Research Associate Professor at UVA’s Curry School of Education, commented: “Empirical Education is a highly respected research firm and offers the type of aggressive timeline that is sorely needed in the fast-paced edtech market.” A report on impact in the school year 2017-2018 is expected to be completed in July.