Number Rockets Program Description
Part D. Maximizing State-Level Funds to Support Students
The Department recognizes that States have an extraordinary opportunity to address the disproportionate impact of the COVID-19 pandemic on underserved students through the ARP Act’s required State set-asides to address the academic impact of lost instructional time, provide summer learning and enrichment programs, and provide comprehensive afterschool programs. In this section, SEAs will describe their evidence-based strategies for these resources.
Academic Impact of Lost Instructional Time: Describe how the SEA will use the funds it reserves under section 2001(f)(1) of the ARP Act (totaling not less than 5 percent of the State’s total allocation of ARP ESSER funds) on evidence-based interventions to address the academic impact of lost instructional time by supporting the implementation of evidence-based interventions, such as summer learning or summer enrichment, extended day, comprehensive afterschool programs, or extended school year programs, and ensure that such interventions respond to students’ academic, social, emotional, and mental health needs. The description must include:
1. A description of the evidence-based interventions (e.g., providing intensive or high-dosage tutoring, accelerating learning) the SEA has selected, and the extent to which the SEA will evaluate the impact of those interventions on an ongoing basis to understand if they are working;
Description of the Evidence-Based Intervention: Number Rockets
This State has identified high-dosage tutoring as an evidence-based intervention to address lost instructional time. Research-proven one-to-one and one-to-small group tutoring has been shown to be the most effective method of accelerating academic achievement. Proven tutoring models that provide their tutors with structured materials and guidance with professional development and coaching have demonstrated greater effects on student achievement than other strategies, including summer school, after school, and extended day programs (Gersten et al., 2020; Neitzel et al., in press; Nickow et al., 2020; Pellegrini et al., 2021; Wanzek et al., 2016).
This State has selected one such program, Number Rockets, for math intervention at grade 1 because of its innovativeness and its demonstrated evidence. Number Rockets is conducted in groups of 2-4 with first-grade students who are experiencing mathematics difficulties. Research, involving randomized controlled trials, demonstrate that Number Rockets improves number concepts and math reasoning, and arithmetic concepts and operations, calculation skill, and mathematics word-problem solving learning for this population of learners over what is expected within the school’s standard program.
Tutors
Tutors are typically hired by school districts. Tutors may be full time or may work part of the school day. A full-time tutor should be able to provide 9-12 40-minute sessions daily (9-24 children per week). Tutors usually conduct lessons in person. Remote delivery during school in-person closures is possible.
Students
Number Rockets is designed for students in grades 1 with risk for poor mathematics outcomes due to low math performance.
Technology
Number Rockets does not require technology for implementation.
Curriculum and Program Operation
Number Rockets is a tutoring intervention for first-grade students with mathematics learning difficulty. The program is based on the concrete-representational-abstract model, which relies on concrete objects to promote conceptual learning. Tutors deliver the program to small groups of 2-3 students three times per week during the school day in 40-minute sessions, with 30 minutes of mathematics instruction and activities, followed by 10 minutes of practice to build arithmetic fluency. The 63 program lessons cover 17 topics, and topics include worksheets and manipulatives (e.g., Base-10 blocks for place value instruction). Topics span identifying and writing numbers; understanding less than, greater than, and equal; sequencing numbers, skip counting; place value; identifying operations; writing number sentences; addition and subtraction facts; 2-digit addition and subtraction; and missing addends.
During the final 10 minutes of each tutoring session, students complete a fluency game designed to develop automatic retrieval of math facts, and students are taught efficient counting strategies as backups to automatic retrieval. Student behavior is monitored throughout each session.
Professional Development, Progress Monitoring, and Follow-up
Fuchs Tutoring offers professional development support for school and district personnel to take on the role of (1) Number Rockets tutors and (2) Number Rockets coaches.
Tutors implement the program directly with students. Coaches are matched with approximately 10 tutors each, and they support tutors through weekly observations with feedback and monthly group tutor meetings. For both roles, candidates complete a certification process and receive ongoing support from Fuchs Tutoring staff. See additional information in the chart below. Also see FuchsTutoring@air.org.
| Tutor | Coach | |
| Certification Process | College degree; tutor experience preferred Participation in 2-day Training Institute (offered twice-annually, location TBD). Brief assessment and/or performance task Submission of tutoring practice video for review, initially and on an annual basis thereafter to retain certification | College or advanced degree; demonstrated prior experience as math specialist or coach Participation in 3-day Training Institute (tutor training plus additional day for coach training; offered twice annually, location TBD) Completion of tutor certification process (assessment and tutoring practice video review) Coach competency task assessment Quarterly coach review process with Fuchs Tutoring Staff |
| Ongoing Support from Fuchs Tutoring | Match with certified coach, if needed Monthly booster training webinars Access to Number Rockets tutor online community of practice | Monthly facilitated group coach meetings Monthly booster training webinars Access to Number Rockets tutor online community of practice |
Evidence Base
Multiple randomized controlled trials have been conducted on Number Rockets and published in peer-review journals. Number Rockets studies have been reviewed by What Works Clearinghouse as part of the 2021 Institute on Education Sciences math intervention practice guide (https://ies.ed.gov/ncee/wwc/Docs/PracticeGuide/WWC2021006-Math-PG.pdf).
The mean effect size for Number Rockets within their Proven Tutoring eligible study is +0.34. For example, in a 2005 randomized controlled trial, Fuchs et al. (2013) randomly assigned 127 first graders with start-of-school-year math deficits to Number Rockets or a control group. Effect sizes comparing Number Rockets to the control favored Number Rockets on curriculum-based measurement slope (+0.40); addition and subtraction basic facts (+0.40, +0.14), Woodcock-Johnson Calculations (+0.57) and Applied Problems (+0.11), first-grade math concepts & applications (+0.67), and story problems (+0.70).
Reference: Fuchs, L.S., Geary, D.C., Compton, D.L., Fuchs, D., Schatschneider, C., Hamlett, C.L., DeSelms, J., Seethaler, P.M., Wilson, J., Craddock, C.F., Bryant, J.D., Luther, K., & Changas, P. (2013). Effects of first-grade number knowledge tutoring with contrasting forms of practice. Journal of Educational Psychology, 105, 58-77.
Extent to Which Number Rockets Will be Evaluated on an Ongoing Basis
Each Number Rockets lesson provides independent practice, which forms the basis for tutors quantifying and tracking progress. Tutors can share individual progress with the student’s classroom teacher and parents on a regular basis. The State will use a standardized measure to evaluate program impact on a large scale.
ii. How the evidence-based interventions will specifically address the disproportionate impact of COVID-19 on certain groups of students, including each of the student groups listed in question A.3.i.-viii. When possible, please indicate which data sources the SEA will use to determine the impact of lost instructional time; and how the evidence-based interventions will specifically address the disproportionate impact of COVID-19 on certain groups of students:
The small-group or individual structure in Number Rockets allows tutors to identify and target students’ individual needs. Number Rockets has demonstrated success among disadvantaged students and students from ethnic minorities, which have been hit hardest by COVID-19.
Available evidence on COVID-19 losses are especially severe in math. This increases the stakes for providing effective, structured tutoring in math. Extrapolating from the summer learning loss literature and preliminary COVID-19 slide estimates, an NWEA Brief (Northwest Evaluation Association; https://www.nwea.org/content/uploads/2020/05/Collaborative-Brief_Covid19-Slide-APR20.pdf) suggests students will return in fall 2021 with roughly 70% of the learning gains in reading relative to a typical school year; worse in math, where students are expected to start the new school year with less than 50% of the learning gains; in some grades, students are nearly a full year behind. The Brief notes that the loss is likely greater in underserved populations.
But because COVID-19 school closures have been more punctuated and frequent than is the case during summer breaks, extrapolating from the summer break literature to COVID-19 learning loss is difficult. Using their spring 2020 and fall 2021 MAP interim test data https://edsource.org/2020/early-data-on-learning-loss-show-big-drop-in-math-but-not-reading-skills/644416, NWEA found on-track reading scores, despite spring 2020 school closure, but substantial loss in math. This may underestimate pandemic learning loss, though, because 25% of students, predominately low-income Black and Hispanic students, did not take the fall tests and because students with disabilities are often excluded from interim assessments (because test accommodation are often not available and alternate assessments are not provided).
Data sources the SEA will use to determine the impact of lost instructional time
INSERT STATE’S PLANS TO EVALUATE ALL STUDENTS WHEN THEY RETURN, IF ANY. The State will use a standardized measure, XXX, to evaluate program impact on a large scale in the spring.
iii. The extent to which the SEA will use funds it reserves to identify and engage 1) students who have missed the most in-person instruction during the 2019-2020 and 2020-2021 school years; and 2) students who did not consistently participate in remote instruction when offered during school building closures:
Students are judged as candidates for Number Rockets if they are among the performing students in math at their grade according to schoolwide math screening data and the need for intervention is confirmed by the classroom teacher.
References
Gersten, R., Haymond, K., Newman-Gonchar, R., Dimino, J., & Jayanthi, M. (2020). Meta-analysis of the impact of reading interventions for students in the primary grades. Journal of Research on Educational Effectiveness, 13(2), 401–427.
Fuchs, L.S., Geary, D.C., Compton, D.L., Fuchs, D., Schatschneider, C., Hamlett, C.L., DeSelms, J., Seethaler, P.M., Wilson, J., Craddock, C.F., Bryant, J.D., Luther, K., & Changas, P. (2013). Effects of first-grade number knowledge tutoring with contrasting forms of practice. Journal of Educational Psychology, 105, 58-77.
Neitzel, A., Lake, C., Pellegrini, M., & Slavin, R. (2021). A synthesis of quantitative research on programs for struggling readers in elementary schools. Reading Research Quarterly. doi:10.1002/rrq.379
Nickow, A, Oreopoulos, P., & Quan, V. (2020). The impressive effects of tutoring on preK-12 learning: A systematic review and meta-analysis of the experimental evidence. National Bureau of Economic Research.
Pellegrini, M., Neitzel, A., Lake, C., & Slavin, R. (2021). Effective programs in elementary mathematics: A best-evidence synthesis. AERA Open, 7 (1), 1-29.
Wanzek, J., Vaughn, S., Scammacca, N., Gatlin, B., Walker, M. A., & Capin, P. (2016). Meta-analyses of the effects of tier 2 type reading interventions in grades K-3. Educational Psychology Review, 28(3), 551–576. doi:10.1007/s10648-015-9321-7