Traditionally, high school students who may be interested in engineering, science or technology fields have prepared for their future careers by taking advanced math and science courses. However, according to Rebecca Napolitano, associate professor of architectural engineering at Penn State, the datasets students use to solve problems in their high school classes bear little resemblance to real-world data that she and other engineers work with.
"Real-world data has never been clean," Napolitano said. "But that's the kind of data we've been handing K-12 students: tidy, well-behaved, pre-solved, no dead ends. Real industrial and research data is high-dimensional, noisy and genuinely messy, and we need to teach students how to reason through that."
To help prepare the next generation of engineers by supplying them with real-world data and problems and offering personalized feedback and support, the U.S. National Science Foundation (NSF) has awarded a team led by Napolitano $1.5 million over three years to develop software that collects real-time information as K-12 students work through computational exercises. The team aims to compile a real-world dataset for use in math classrooms, measure students' reasoning processes and identify ways to foster productive learning while addressing unproductive confusion.
The project is supported by the Collaboratory to Advance Mathematics Education and Learning for K-12 (CAMEL), a program launched by the NSF to "generate and share high-value, artificial intelligence (AI)-ready datasets that advance K-12 math learning by better connecting the science of learning, classroom practice and data science." According to Napolitano, her team plans to build a dataset containing real industrial data, including images from Penn State's Two-Dimensional Crystal Consortium (2DCC), a materials research facility supported by the NSF.
Napolitano said her team aims to bring their dataset to algebra and statistics classrooms in seven rural Pennsylvania school districts. If students choose to participate in the study, their learning processes will be measured using telemetry software, a type of software that automatically collects and transmits information on user activity. As participating students work through assignments on the educational dataset, the researchers' telemetry software will track every change they make in real time, including how many errors they hit and how long they pause.
Data from the telemetry software will be interpreted by human coders, who will be trained to accurately identify and categorize patterns in the measurements. Napolitano explained that these patterns could offer a distinction between different forms of academic struggle.
"We want to know, are the students iterating and refining, genuinely learning from the struggle?" Napolitano said. "If so, that's exactly what we want to protect, and we don't want to swoop in and rescue them out of it. But if they're stuck and disengaging, can we catch that in real time? What are the warning signs, and once we see them, what's the right move? Does a teacher step in, does the system nudge the student? What actually helps?"
According to Napolitano, the researchers will continue reworking the program over the duration of the award, working directly with teachers to shape protocols for analyzing the telemetry data. The team hopes for the procedures to be AI-ready by the end of the three-year period, such that a machine learning tool could predict whether a student's struggle is productive from telemetry data alone.
"Productive struggling builds critical thinking, but it's a thin line," Napolitano said. "If you push a student too long without noticing they've stopped iterating and started disengaging, you don't just lose that assignment. You can lose that student on math, or STEM entirely, for good. These are the moments we think can be pivot points for some of these students, and if we can catch them in time, we can help them."
Wangda Zuo, professor of architectural engineering at Penn State, is a co-principal investigator on the project. Zuo will be contributing expertise and real-world data to the educational dataset.
"By building real operational data into academic curricula, we hope to better prepare the next generation of engineers to advance research on sustainable and resilient infrastructures," Zuo said.
Other co-principal investigators on this work include Kathleen Hill, professor of science education at Penn State; Xiangquan Yao, associate professor of mathematics education at Penn State; and Wesley Reinhart, assistant professor of materials science and engineering at Penn State.
The awarded project was funded by the U.S. National Science Foundation under award number 2621173. This content is solely the responsibility of the authors and does not necessarily reflect the views of the funders.
