AI in Higher Education

[Jordan]: Welcome to AI in Higher Education. I’m Jordan. Today, we’re looking at a more precise question than whether AI is simply good or bad for colleges: where does it support learning, and where might it make academic performance harder to interpret?

[Jordan]: The emerging evidence points in both directions. Some AI-supported approaches are associated with greater engagement and stronger work. At the same time, large-scale grade data suggest that higher grades may not always represent equivalent gains in learning.

[Jordan]: Let’s start with a promising classroom result from Puyang Medical College in China.

[Jordan]: In a 14-week study, researchers tested a four-layer AI teaching model with 120 students in two intact classes. The model combined intelligent resource management, adaptive learning paths, multimodal interaction, and AI-supported evaluation.

[Jordan]: Students in the AI-assisted condition scored higher on a composite measure of computational thinking, with a reported effect size of 0.97. The largest gains involved abstraction and evaluation, along with generalization.

[Jordan]: In plain language, students appeared to improve not just at getting an answer, but at identifying the important features of a problem and applying conclusions more broadly.

[Jordan]: That’s encouraging, especially for institutions exploring adaptive learning in computing, health professions, and STEM. It also suggests that the more interesting question may not be, “Can students chat with an AI tool?”

[Jordan]: It may be, “What happens when AI is integrated into resources, learning pathways, interaction, and assessment as one instructional design?”

[Jordan]: But this is still a pilot, not a universal estimate of impact. It took place at one institution, with two classes. And class-level differences can’t be completely separated from the intervention itself.

[Jordan]: So the finding is worth investigating, while resisting the temptation to treat it as proof that the same model will work everywhere.

[Jordan]: A second study, published in the International Journal of Artificial Intelligence in Education, offers more direct evidence about AI feedback. In a semester-long undergraduate macroeconomics experiment, students completed eight open-ended tasks. They received either lecturer-only feedback, peer feedback, or individualized GPT-4 feedback.

[Jordan]: Compared with lecturer-only feedback, students receiving AI feedback participated more consistently, wrote longer responses, and showed greater improvement in answer content.

[Jordan]: The researchers suggest that one reason AI feedback differed from peer feedback may be its consistency and reliability.

[Jordan]: That matters because this wasn’t merely a demonstration that an AI system can generate comments. The study connected individualized feedback with observable changes in student behavior and in the quality of written work in a live course.

[Jordan]: There’s a practical lesson here. AI feedback may be particularly useful when it supports recurring, low-stakes writing and is tied to explicit content criteria.

[Jordan]: That’s a very different proposition from handing over high-stakes grading, or asking AI to make nuanced judgments about motivation, disciplinary quality, or student circumstances.

[Jordan]: The result also doesn’t automatically transfer to every discipline. But for instructors who need to provide frequent formative feedback, it offers a more concrete place to begin.

[Jordan]: Now, a third development points toward a different role for AI altogether.

[Jordan]: Cornell researchers received a $750,000 National Science Foundation grant to develop and study the Automated Critical Thinking Reflection tool, or ACTR, in physics, biology, biomedical engineering, and related STEM courses.

[Jordan]: The system isn’t being designed simply to answer students’ questions. Instead, it’s intended to provide personalized follow-up prompts. These prompts ask students to reflect on lectures and labs, question claims, and verify information.

[Jordan]: That represents an important shift: from AI as answer engine to AI as structured reflection partner.

[Jordan]: Could a system like this help students pause before accepting a plausible-sounding response? Could it make metacognitive prompts more available in large-enrollment courses, where instructors can’t conduct every individual follow-up conversation?

[Jordan]: Those are meaningful possibilities. But it’s important to keep the evidence in proportion. ACTR is a funded research project. It isn’t yet evidence of completed learning gains.

[Jordan]: Still, the design principle is useful now. The educational value of AI may depend less on how quickly it supplies an answer, and more on whether it prompts students to explain, verify, and reconsider.

[Jordan]: That brings us to the institutional level. Arcadia University has launched an AI Navigation Hub through its Center for Teaching, Learning, and Mentoring.

[Jordan]: The hub offers guidance for students, faculty, and staff on practical AI use, ethical questions, course expectations, evaluating AI outputs, restrictions on use, and AI-related career questions.

[Jordan]: Just as important, it supports both adoption and limitation. The message isn’t that every use of AI should be encouraged, or that every use should be prohibited.

[Jordan]: Why does that matter? On many campuses, relevant information is scattered across separate policy pages, workshops, student-support offices, libraries, and academic-technology units.

[Jordan]: A visible, coordinated service gives people somewhere to go when the question isn’t simply, “Is AI allowed?” It’s, “What’s appropriate here? What should I disclose? And how do I evaluate what the system gives me?”

[Jordan]: Arcadia’s example treats AI guidance as an ongoing academic and student-service function, rather than a one-time policy announcement.

[Jordan]: And then there’s the development that should make every institution pause: evidence that AI may be raising grades faster than it raises learning.

[Jordan]: A working paper from the UC Berkeley Center for Studies in Higher Education examined more than 500,000 grades from 2018 through 2025.

[Jordan]: Using a difference-in-differences design, it reported that courses with more AI-exposed assignments, particularly writing and coding, saw an increase in A grades after ChatGPT’s release of approximately 13 percentage points.

[Jordan]: That was about a 30 percent increase relative to the 2022 baseline. The increases were larger when homework carried more weight.

[Jordan]: The author says that pattern is consistent with AI substituting for student work, rather than reflecting broad gains in learning.

[Jordan]: That interpretation deserves careful attention, but also careful limits. The study doesn’t establish that AI caused every grade increase. And it doesn’t prove that learning failed to improve.

[Jordan]: What it does provide is a substantial institutional signal. If grades rise sharply in assignments that students can readily outsource, leaders need to ask what those grades now communicate.

[Jordan]: Are students demonstrating independent performance? Do course assessments include opportunities to verify that? Have grade distributions, homework weighting, and later unaided work changed?

[Jordan]: This isn’t only an academic-integrity issue. It’s also an assessment-validity issue, a curriculum issue, and potentially a transcript issue.

[Jordan]: For faculty and academic leaders, the challenge is to distinguish productive assistance from substitution. For student-services teams, the challenge is helping students understand that using a tool isn’t the same as demonstrating mastery.

[Jordan]: Taken together, these developments suggest a more balanced way to think about AI in higher education.

[Jordan]: AI may help when it provides timely feedback, adapts instruction, or prompts reflection. But those benefits depend heavily on the surrounding design: the task, the criteria, the opportunity for verification, and the evidence of independent learning.

[Jordan]: The central question isn’t whether a tool produces better-looking work. It’s whether students are becoming more capable, more reflective, and better able to perform without the tool when that matters.

[Jordan]: Worth considering: first, pilot AI feedback in low-stakes, recurring assignments before using it for grading.

[Jordan]: Institutions could compare AI feedback with lecturer and peer feedback. Then they could measure not only participation and response length, but also revision quality, persistence, and independent follow-up performance.

[Jordan]: Second, design AI interactions to require reflection and verification.

[Jordan]: Students might identify assumptions, check sources, explain why an AI response could be wrong, or complete an unaided teach-back after using the system.

[Jordan]: And third, create a visible, cross-campus AI guidance function.

[Jordan]: A jointly staffed service could connect teaching support, library expertise, accessibility services, academic integrity, advising, and career education.

[Jordan]: The larger lesson from this month’s evidence is that institutions don’t have to choose between unrestricted adoption and blanket prohibition.

[Jordan]: They can ask better questions, run bounded pilots, and pay closer attention to whether improved performance reflects improved learning.

[Jordan]: That’s all for this episode of AI in Higher Education. Thanks for listening.
[Jordan]: The voice in this episode is AI-generated.