We work to make our research operations efficient while always maintaining high quality. Using software tools, our research managers get over 90% response rates from teacher surveys and securely collect and warehouse student and other school district data. Statistical programing and report production provides quick turnaround. Our research scientists and managers guide the entire life cycle of a rigorous evaluation from design to publication. Our operations team also helps other research organizations lower their costs by drawing on efficient components of our overall capabilities.
Some highlights include the following.
We are experienced in identifying, contacting, and bringing on board districts, schools, or teachers to participate in research projects. We draw upon our many district contacts established through prior work and professional meetings, and we develop new relationships and recruit new sites as needed. We use standard district agreements that are brief yet comprehensive, spelling out the study’s data requirements and identifying a point of contact.
Empirical retains an external institutional review board (IRB) and has worked closely with this organization to develop standard operating procedures, including proprietary templates for district agreements, teacher consent forms, and other research activities. We have also developed the package required by federal agencies for any research conducted under federal contracts.
We have expertise in interview and observation protocols, including quantitative and qualitative analysis. We conduct focus groups, individual participant interviews, and observations of professional development. We have recruited and trained local observers and can manage large-scale observations or testing anywhere in the country.
We have extensive experience with the various stages of data collection from school districts and state departments of education, where we deploy our SecureServer®. These stages include:
Drawing on our experience in data warehousing, cleaning, and compilation, our team takes full advantage of district student record systems to track program implementation efficiently. We coordinate the administration of testing and have technologies for scanning and tracking double data entry.
Our data warehousing is optimized for school data and can easily handle large and complex datasets. Our procedures are designed to address the many cases in which data provided by districts have not been fully verified or standardized. Our data system interfaces with our analytical and survey tools and supports a multi-level structure for districts/schools/students that allows flexible hierarchies, default structures for handling standard demographics while allowing extensions, ability to track changes such as attrition and its causes, and multi-year support, including tracking changes in teachers, students, and rosters. We have experience in integrating data from longitudinal data systems and are working on a project to convert census tract data to district/school units.
We are particularly aware of the need to present findings in appropriate contexts and to provide guidance in the interpretation of research results. Through hundreds of interactions with district personnel, we have developed expertise in bridging the world of rigorous methodology and the practical context of schools. Thus, our analyses include measures at the student, teacher, and school levels, and our reports provide an interpretation of results related to the implementation and purposes of the program under study.
Our reports, while written in accessible language, contain the necessary criteria for submission to peer-reviewed journals as well as to the US Department of Education’s What Works Clearinghouse (WWC). Because several Empirical staff members have been trained in WWC review procedures, we are proactive in how we conduct and report our research. In addition to the full reports and research summaries intended for a non-technical audience, feedback to the client—whether a school system or a publisher—includes analysis of the product features that worked well or that failed, which populations it benefited or failed for, implementation issues that influenced the outcome, and acceptance by teachers. We also can conduct reviews based on the principles of the WWC and meta-analyses of existing research literature.
We bring a deep knowledge of how school districts’ conduct and use research for decision making and can work with districts to prepare the rigorous evaluation component required by many federal and state grants. Our workshops for educators focus on understanding the value of research, as well as the application of these methodologies to support the work in their agencies. Our ability to connect insights from research to the needs of educators, particularly the need for timely evidence, broadens the value of our research capacity.
