Skip to content

DreamBox Math Study at William Penn School District: ESSA Evidence Level III

At-a-Glance

Type:

Case Study

Source:

LearnPlatform by Instructure

Grade Level:

K-5

Product:

DreamBox Math

Share This Report

essa level iii badge

In 2022, LearnPlatform, a third-party edtech research company, was contracted to explore the extent to which DreamBox Math helps accelerate learning for all students at William Penn School District in Pennsylvania.

This study examines the relationship between usage of DreamBox Math and student math outcomes. LearnPlatform designed the study to satisfy Level III requirements (Promising Evidence) according to the Every Student Succeeds Act (ESSA).

DreamBox Math provides invaluable strategies for addressing our greatest academic needs, showing educators where best to intervene, and helping each student reach their full potential.
quote filled
Edward Dunn
Curriculum Supervisor of Mathematics, William Penn School District, PA

Source

LearnPlatform by Instructure, Senior Researchers Ashley Hunt, Ph.D. and Meetal Shal, Ph.D., Director of Research Mary Styers, Ph.D. “DreamBox Math (2021-22) ESSA Evidence Level III study conducted at The William Penn School District, PA”. Discovery Education, February 2,2023.

Who Was Included in the Study?

This study included 18,51 K-6 students across eight elementary schools in William Penn School District, PA. This student sample represents a diverse and historically vulnerable student population. 

Researchers Asked:

  • How were different DreamBox Math usage patterns related to grade K-6 students’ spring 2022 math achievement?
  • Which usage pattern(s) of DreamBox Math had the greatest impact on grade K-6 students’ spring 2022 math achievement?
  • What was the overall impact of DreamBox Math on grade K-6 students’ spring 2022 math achievement?
  • How did the impact of DreamBox Math vary by student grade and racial identity?
  • What was the impact for students who had free or reduced-price lunch (FRL) status?

Study Sample Student Demographics

Students with IEPs
0 %
Students Qualify for FRL
0 %
English Language Learners
0 %

What Data Were Evaluated?

Researchers investigated differential rates of learning acceleration for K-6 students who had access to DreamBox Math during the 2021-22 academic year and completed the fall 2021 Savvas MSDA math pretest and the spring 2022 Savvas MSDA math posttest.

Analysts used 2021-22 student-level DreamBox Math usage (i.e., average weekly lessons and weekly minutes). This usage data informed the extent to which students used the program during the school year and whether students’ use of DreamBox Math related to math learning outcomes on Savvas™ MSDA (the district’s end-of-year assessment).

Study Inclusion Requirements

Students in grades K-6 who completed both:
✓ Fall 2021 Savvas MSDA math pretest
✓ Spring 2022 Savvas MSDA math posttest

DreamBox Math Usage Groups

✓ Low usage = 0 to 3.4 weekly lessons
✓ Moderate usage = 3.5 to 6.7 weekly lessons
✓ High usage = More than 6.8 weekly lessons

How Much Time Did Students Use DreamBox Math Each Week?

DreamBox Learning recommends students complete five lessons per week (about one hour). Overall, students at William Penn School District demonstrated strong usage. During the 2021-22 school year, students at WPSD completed an average of 4.2 DreamBox Math lessons per week and spent an average of 70 minutes in DreamBox Math per week. 59% of students completed 3.5+ weekly lessons and 33% of students completed the recommended 5 weekly lessons. 

Overall Distribution of DreamBox Math User Groups

What Were Some Key Takeaways from the Study?

Overall, students who completed more lessons and spent more time in DreamBox Math had higher math achievement at the end of the study.

✓ Students who completed at least 3.5 lessons (55 minutes) in DreamBox Math had higher end-of-year math achievement scores.
✓ This study controlled for students’ beginning-of-year math achievement and demographics.

DreamBox Math was equally impactful for students regardless of grade, race, and FRL status.

✓ Across all grades, students who completed more DreamBox Math weekly lessons had higher spring Savvas MSDA achievement scores.
✓ Students who identified as Black (African American) and completed more weekly lessons/spent more time (weekly minutes) in the program had higher end-of-year math achievement than those students who used the program less.
✓ Among students who had FRL status, there was a positive, statistically significant relationship that suggests students who completed more weekly lessons/spent more time using the program had higher end-of-year math achievement than those who didn’t use the program.

Education Intervention Impact

LearnPlatform researchers calculated standardized effect sizes (Hedge’s g) to determine the magnitude of changes in student outcomes. The effect size for students who used DreamBox Math more than 6.8 lessons per week was 0.29. This result suggests that DreamBox Math is 4Xs more effective than the average elementary school math intervention.*

* Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., Roberts, M., Anthony, K. S., & Busick, M. D. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms (NCSER 2013-3000). Washington, DC: National Center for Special Education Research, Institute of Educational Sciences, US Department of Education.

Research from Over 100,000 Students Shows DreamBox Math Works

At-a-Glance

Type:

Research Brief

Time Period:

2019 – 2022

Source:

Multiple

Product:

DreamBox Math

Share This Report

By personalizing instruction, DreamBox Math ensures that all students can achieve significant growth in math.

Over the past several years, Discovery Education has partnered with third-party research organizations, expert data scientists, and education researchers to analyze the efficacy of DreamBox Math across over 100,000 K–8 students. The rich data set includes students using DreamBox Math in class, at home, during accelerated programs, for intervention, and for practice.

Researchers applied a variety of methodologies to explore diverse data for students from different-sized districts, in unique parts of the nation, and facing varying learning challenges. Regardless of how researchers explored data, all types of students’ math proficiency grew with DreamBox Math. The six takeaways in this report demonstrate how versatile this program is for all learners.

Millions of Data Points

100,000+ Students

Six Key Research Takeaways

South Carolina Educators Feedback on Experience

At-a-Glance

Type:

Survey

Time Period:

September 2022

Source:

Discovery Education Research & Analytics

Grade Level:

K-12

Products:

  • Discovery Education Experience
  • Professional Learning

Share This Report

The DE Research & Analytics team conducted a feedback survey in September of 2022 to learn more about the use of Discovery Education Experience among South Carolina educators, Specifically, they wanted to glean insights into digital teaching and learning, and how professional learning plays a critical role. The final survey featured a total of 17 responses from experienced educators that taught a range of subjects and grades.

south carolina study card

DreamBox Reading Park Logic Model: ESSA Evidence Level IV​

At-a-Glance

Type:

Logic Model

Source:

LearnPlatform by Instructure

Grade Level:

K-2

Product:

DreamBox Learning Reading Park

Share This Report

essa level iv badge

Discovery Education engaged LearnPlatform by Instructure, a third-party edtech research company, to develop a logic model for DreamBox Reading Park. LearnPlatform by instructure designed the logic model to satisfy Level IV requirements (Demonstrates a Rationale) according to the Every Student Succeeds Act (ESSA).

To continue building evidence of effectiveness and to examine the proposed relationships in the logic model, Discovery Education plans to conduct an evaluation to determine the extent the Reading Park program produces the desired outcomes for grades K-2. Specifically, Discovery Education has plans to begin an ESSA Level III study.

Because of the development of the logic model and the plan for an ESSA Level III study, DreamBox Reading Park by Discovery Education meets Tier IV evidence requirements in compliance with ESSA.

Source

LearnPlatform by Instructure, Researcher Alexander Lee, Ph.D. and Senior Researcher Meetal Shal, Ph.D. “Reading Park Logic Model.” Discovery Education, January 5, 2024.

DoodleMath Logic Model: ESSA Evidence Level IV​

At-a-Glance

Type:

Logic Model

Source:

LearnPlatform by Instructure

Grade Level:

K-5

Product:

DoodleMath

Share This Report

essa level iv badge

Discovery Education engaged LearnPlatform by Instructure, a third-party edtech research company, to develop a logic model for DoodleMath. LearnPlatform by instructure designed the logic model to satisfy Level IV requirements (Demonstrates a Rationale) according to the Every Student Succeeds Act (ESSA).

To continue building evidence of effectiveness and to examine the proposed relationships in the logic model, Discovery Education plans to conduct an evaluation to determine the extent to its program produces the desired outcomes. Specifically, Discovery Education has plans to begin an ESSA Level III study.

Because of the development of the logic model and the plan for an ESSA Level III study, DoodleMath by Discovery Education meets Tier IV evidence requirements in compliance with ESSA.

Source

LearnPlatform by Instructure, Researcher Andrew Scanlan, M.A. and Senior Researcher Meetal Shal, Ph.D. “DoodleMath Logic Model.” Discovery Education, January 20, 2023.

Florida Study Shows Correlation between Discovery Education and Student Achievement

At-a-Glance

Type:

Executive Summary

Time Period:

2018-2019; 2021-2022

Source:

McREL International

Grade Level:

3-12

Products:

  • Discovery Education Experience
  • Science Techbook
  • Social Studies Techbook

Share This Report

The purpose of this study was to examine the relationship between Discovery Education’s market presence and academic achievement in the State of Florida. The study used a correlational design, where the outcomes of interest were school average state assessment scores in Math, Science, English, and Social Studies for Spring 2019 and 2022; and the focal predictor was if a school had a Discovery Education license or not (i.e., treatment or comparison).

The research question this study aimed to answer was whether or not schools with one or a combination of Discovery Education products report higher achievement scores than schools who do not have access to any DE product. The results showed instances where Discovery Education had positive effects on student achievement. 

Texas Study Shows Correlation Between Discovery Education and Student Achievement

At-a-Glance

Type:

Executive Summary

Time Period:

2018-2019; 2021-2022

Source:

McREL International

Grade Level:

3-12

Products:

  • Discovery Education Experience
  • Science Techbook
  • Social Studies Techbook
  • Mystery Science

Share This Report

The purpose of this study was to examine the relationship between Discovery Education’s market presence and academic achievement in the State of Texas. The study used a correlational design, where the outcomes of interest were school average state assessment scores in Math, Science, English, and Social Studies for Spring 2019 and 2022; and the focal predictor was if a school had a Discovery Education license or not (i.e., treatment or comparison).

The research question this study aimed to answer was whether or not schools with one or a combination of Discovery Education products report higher achievement scores than schools who do not have access to any DE product. The results showed instances where Discovery Education had positive effects on student achievement. 

North Carolina Statewide Study Summary of Findings

At-a-Glance

Type:

Case Study

Time Period:

2016-2017

Source:

Discovery Education Research & Analytics

Grade Level:

K-12

Products:

  • Discovery Education Experience
  • Science Techbook
  • Math Techbook

Share This Report

Discovery Education Research & Analysis compared partner schools in North Carolina to non-partner schools to study the extent to which achievement on state summative assessments is related to partnership with Discovery Education.

To accomplish this, Discovery Education compared the published school level data for the state of North Carolina for the 2016-2017 school year to a list of Discovery Education partners for the same time period, resulting in an analysis of achievement of DE partner schools vs. non-DE partner schools.

There are 34 performance indicators that show that partner schools perform better than non-DE partner schools. This positive correlation between DE partners and better scores found in the overall sample also holds for several different demographic subgroups.

Grade-Level Proficiency Results
% of students who met the state’s GLP benchmark (2016-2017)

Sahuarita and Freeport-McMoRan Foundation Partnership Impacts Grade-Level Proficiency​

At-a-Glance

Type:

Evaluation Study

Time Period:

Jan 2020 – Oct 2020

Source:

Decisive Data Insights

Grade Level:

K-8

Products:

  • Discovery Education Experience
  • Science Techbook

Share This Report

Sahuarita Unified School District (SUSD), located just south of Tucson, Arizona, serves 5,800 students enrolled in PK-12 grades. The district has been the recipient of funding via the Freeport-McMoRan Foundation (FCX Foundation), whose goal is to build resiliency in communities where it has a presence.

This evaluation is the first study that seeks to understand the impact of various SUSD initiatives funded by the FCX Foundation, and was led by researcher Dr. Brett Kramer of Decisive Data Insights. The study began with stakeholder engagement. The research team then collected performance data across the funded projects, analyzed the data to determine partnership effects, and tied those effects back to larger measures to inform the direction of future investments.

The investment in Discovery Education led to an increase in math, science, and reading proficiency. And, with 18 positive effects across grade and subjects, it can be concluded that use of Discovery Education led to better academic outcomes in every subject and grade level, to some degree, than what would have been attained without its use. 

Discovery Education’s Impact on Student Achievement​

At-a-Glance

Type:

Research Brief

Time Period:

2016 – 2022

Source:

Discovery Education Research & Analytics

Grade Level:

K-12

Products:

  • Discovery Education Experience
  • Science Techbook
  • Math Techbook

Share This Report

Our award-winning K-12 platform, Discovery Education Experience, empowers educators with a vast collection of multimodal resources vetted and curated to support differentiated, scaffolded learning in any environment. Experience is built upon a theory of action about how best to enhance the K-12 teaching and learning experience. Based on this theory of action, Discovery Education produced a logic model that serves as a conceptual map for Experience.

Findings from multiple studies tell a consistent and promising story of how use of Discovery Education’s curricular resources is related to outcomes that matter across the country for students overall, and especially for racial/ethnic minorities and students impacted by poverty, disabilities, and limited English proficiency. Discovery Education access is, in fact, making a significant difference in teaching and learning.

Download the full research brief to learn more about how Discovery Education makes an impact on student achievement across multiple states and supports educators with innovative ways to connect and collaborate with one another.