Identifying ChatGPT Text-based Plagiarism
General plagiarism identification advice applies to ChatGPT text-based plagiarism, too.
You may have heard about detection tools but they are not reliable, recommended or approved for use.
- Read this post for more on current limits of these tools.
- Read this article about how ChatGPT can be prompted to beat the detection tools
- Read this post for more on how our academic misconduct regulations may apply.
ChatGPT is less about accuracy and more about generating responses indistinguishable, linguistically, to what humans would produce. This means much of what ChatGPT produces seems like work we would attribute to our students. There are some things to watch for.
Possible flags (in 2023, anyway):
- Tendency toward generalities may be one of the biggest “tells” as the tools are trained for most likely responses not the most specific or detailed responses. However, this problem can be counteracted to some extent by prompting for more specificity or detail.
- Missing the point of the assessment especially when the topic is too specialized for ChatGPT. A topic may be too specific for ChatGPT because there was less training data related to it and as a result ChatGPT produces more general or less relevant responses.
- Missing context from class discussions or other specific experiences that are meant to be reflected in the assignment. However, by inputting detail to a ChatGPT prompt and requesting a more personalized and reflective writing style, the class context can be incorporated into ChatGPT outputs.
- Repetitiveness of content where it seems the same idea is stated in multiple ways, sometimes without much variation
- Overuse of lists because ChatGPT is really good at producing lists
- Limited citing and referencing in what is produced because ChatGPT has less of this kind of content in its training data. But, ChatGPT is getting better at this, especially the paid version of ChatGPT 4.
- Fabrication of sources was a larger problem early in 2023 than it is now, but they can still occur with ChatGPT results.
- Other so-called “hallucinations” where the GenAI presents content convincingly as being accurate or truthful when it is not. In addition to fabrication of sources, ChatGPT may fabricate details about an event, use incorrect dates, get distances between towns wrong, give wrong advice, and so on.
It is possible to finetune ChatGPT responses to avoid the above problems produced by ChatGPT. And, it is important to note that ChatGPT is rapidly improving and more specialized GenAI tools are being developed.
Read this post for ideas about how to talk to your students about your concerns.