It may be the middle of the academic year but Dr Ann Marcus Quinn, a lecturer in technical communication and instructional design, is already ripping up much of her approach to how she assesses students.
“I used to ask students to do assignments at home such as rewriting a poorly written text from, say, a school communication or company memo. Those days are gone,” says Dr Marcus Quinn, based at the University of Limerick.
“ChatGPT can do it in seconds. So, the future has to be different, more oral presentations, more in-class activities, more explanation, thinking more creatively about how to test writing skills. We have to be prepared, for this has changed assessment. You can be damn sure students have figured out its potential already.”
Chat GPT, the artificial intelligence language model from OpenAI, has been making headlines since November last year for its ability to answer complex questions within seconds.
It can write nuanced essays, poetry, generate code, translate languages, among other functions, in response to prompts from human users. The new tool quickly sparked alarm on college campuses; videos circulated online, with millions of views, showing students how to use it for assignments.
Just when academics caught their breath, GPT-4 arrived last week. The updated version is more capable still: it has performed astonishingly well on demanding tests such as bar exams (beating 90 per cent of humans) and advanced college entry tests, as well as tax returns and accountancy challenges. It can take photos as an input and deliver more natural writing – yet it also throws up false answers and is unable to reference events after September 2021.
Tech giants, meanwhile, are pouring billions into rival tools as an artificial intelligence arms race gathers pace. Just this week Google released Bard, its own AI chatbot, that it says can draft emails, poems and offer guidance.
In Irish higher education, academics like Dr Marcus Quinn, department heads and lecturers are re-examining how they assess students in light of ChatGPT, prompting a potentially huge shift in teaching and learning.
Quality and Qualifications Ireland (QQI), the watchdog for standards in Irish higher education, says many higher education institutions have initiated full reviews of policies regarding assessment and academic integrity.
“It is a matter for institutions to take time to explore the impact of these tools on the system and understand how they may harness new technological tools such as ChatGPT, while balancing out any potential risks to academic integrity,” a QQI spokeswoman recently said.
Some are redesigning their courses entirely and others are making changes that include more group work, in-class tests and random vivas, or exams where students answer questions orally.
While some saw ChatGPT as an existential threat to higher education a few months ago, the conversation has evolved quickly.
Talk of banning or restricting its use on campus has been replaced by a recognition that the technology is here to stay. Now, some are figuring out how to include it in their assignments.
“It dropped like a bomb,” says Dr Perry Share, head of student success at Atlantic Technological University. “It’s probably the biggest thing to hit higher education since the internet.
“You can see that as a threat, or else a chance to shake up higher education, to reimagine how we assess students and make it more relevant to the real world.”
The real question is can we design authentic assessments that are worth doing in the first place?
— Dr Tomás Mac Eochagáin
He is in the latter camp and estimates, broadly, that about a quarter of students are actively using AI tools in their assignments, while the remainder are either curious about it or unaware of its capabilities.
“We can’t ignore this – it isn’t going to go away,” he says. “We have to prepare students for the real world, this is going to disrupt the jobs market and require new sets of skills. Law, accountancy, marketing and others.”
Prof Michael Madden, established professor of computer science at University of Galway, says it is a hot topic among staff and students.
“Many students say they or ‘their friends’ are trying it out or testing to see if it can work. There are plenty of anecdotes about how it’s being used for assessments,” he says. “There’s lots of curiosity, lots of interest.”
Most academic policies in Irish universities treat the unauthorised or unacknowledged use of AI as a form of cheating.
Prof Madden says these tools will become part of the assessment process and notes that some students have begun to acknowledge their use in assignments, however.
Just this week, he said, the university organised a seminar for students, staff and industry on “ChatGPT: threat or opportunity?”
The history of computer science, he says, is one where roles and processes are being automated all the time. A key challenge, he says, will be ensuring students understand programming concepts before using AI tools.
Ultimately, he says, these tools will allow students to complete more complex tasks that might never be possible otherwise.
[ Ten ways Chat-GPT4 is impressive if still flawedOpens in new window ]
[ OpenAI teams up with Stripe to monetise Chat-GPTOpens in new window ]
Dr Tomás Mac Eochagáin, director of academic programmes for Griffith College, says AI should only pose a threat to generic assignments that can be easily replicated.
“The real question is can we design authentic assessments that are worth doing in the first place?” he says.
“Instead of, for example, asking a digital marketing student to do a bog-standard assignment like comparing the marketing strategies of Adidas and Nike, we should be asking students to try market test a new runner with 20 members of the public on Dame Street, gather their opinions, ask what changes would they make ... you will know immediately if they have really done this.”
Future assignments, he says, might also ask students to critically analyse what answer Chat GPT produces to a question compared to other services.
“The way I see it is that if ChatGPT can provide an answer to an assignment within a few seconds, then it was a pretty bad assignment in the first place,” he says.
The National Academic Integrity Network, a group of Irish academics established by QQI, has scheduled a range of seminars next week to discuss the implications of AI.
Where recent seminars focused on cheating threats, next week’s discussions focus more on how to harness AI’s potential for students with disabilities, rethinking assessment and whether there are opportunities to be grasped.
Billy Kelly, the network’s chair and former dean of teaching and learning at DCU, says this is recognition that AI tools are here to stay.
“We’re living in a world where this is more and more ubiquitous and part and parcel of everyday life,” he says. “In the short term, some will be scared because it disrupts traditional assessments and makes them less secure than they might have been. The real challenge for us is what do future learning outcomes look like and what is authentic assessment.
“While it might have been seen initially as an existential threat, we’re going to have to incorporate it and use it for good.”
Beyond ChatGPT: the other AI tools set to reshape learning, music and film
ChatGPT isn’t the only AI tool that has emerged to challenge how students are assessed. There are now AI tools for generating images, video and music, with more to come.
DALL·E 2: Creates realistic images and artwork in different styles based on written prompts. It is owned by Open AI, the same group behind ChatGPT.
Magenta Studio: Uses cutting-edge machine-learning techniques to generate music.
Wordtune Spices: Improves writing by adding background information, historical data or even jokes to prove your point. Owned by Israeli start-up AI21 Labs.
Runway Gen-2: Turns images, video clips or even text into pieces of video in a range of styles.
