There's a neat trick I've seen teachers do to address the concern that regurgitation isn't learning (and it sort of mirrors the technique we use in machine learning): ask a follow-up question based on the essay.
A student that has done the underpinning research to generate the essay should be able to field a closely-related question, offered in realtime. If they can't, that's concerning.
(To be sure, some students will address this approach by having the machine auto-generate the essay than going off and reading around the answers the essay gives to understand the adjacent space to field the upcoming question. Good; that means they just used the machine as a "cliff's notes on the topic" generator, they still did some research).
That's also just testing memory. I had a teacher do that to me in high school where they graded papers during winter break and when they asked me about it in January I could barely remember that I wrote the document let alone the specifics of the topic.
Of course, I have a exceptionally bad (37th percentile) long-term memory so maybe that's just me. Schooling in general seems to be geared towards tests of memory at the expense of understanding.
High school me: "Well I understood it at the time" hehe
IIUC, testing memory is half of pedagogy. Memory (or the student growing the necessary techniques and discipline to supplement an internal lack of it) is pretty key to (at least US) pedagogy in the primary / intermediate levels.
A student that has done the underpinning research to generate the essay should be able to field a closely-related question, offered in realtime. If they can't, that's concerning.
(To be sure, some students will address this approach by having the machine auto-generate the essay than going off and reading around the answers the essay gives to understand the adjacent space to field the upcoming question. Good; that means they just used the machine as a "cliff's notes on the topic" generator, they still did some research).