Reimagining Assignments: Assessing ChatGPT’s (or other AI platform) Factual Accuracy, Biases and Artificial Empathy
This is a possible reimagined approach to an assignment.
This assignment comprises two parts (as seen in the diagram below). Read on for more details regarding each.
Examples of tasks:
The reflection part of these tasks can be done with or without ChatGPT having to connect to the internet. These tasks should be adapted to the course you’re teaching.
Blue colour is the actual prompt. Red is an aspect to be changed by the teacher.
Prompt: Provide a brief explanation of [teacher-chosen concept]. (philosophy (e.g., existentialism), economics (e.g., supply and demand), psychology (e.g., cognitive dissonance), literature (e.g., metaphor), and political science (e.g., democracy)
In this task you need to include the section of the text to which you’re referring in your answer. Also include the full original output as an appendix.
- How accurately does ChatGPT explain this concept?
- Does it capture the complexities and nuances of the concept effectively?
Please note that you should inform your students that AIs in rare situations can “hallucinate” or make things up. So students should take every piece of advice or explanation critically and evaluate that advice.
Prompt: [teacher chooses scenario e.g.] I just lost my job and I'm feeling really upset.
In this task you need to include the section of the text to which you’re referring in your answer. Also include the full original output as an appendix.
- Does ChatGPT's response show understanding and acknowledgment of the emotional state described?
- Does it adapt its language tone according to the emotional content of the scenario?
Please note that you should inform your students that AIs in rare situations can “hallucinate” or make things up. So students should take every piece of advice or explanation critically and evaluate that advice.
Prompt: What is/are the main argument(s) of [teacher-chosen topic]. (E.g. climate change, gun control, universal healthcare, vegetarianism/veganism, and feminism.)
In this task you need to include the section of the text to which you’re referring in your answer. Also include the full original output as an appendix.
- Does ChatGPT provide a balanced overview of each topic, representing various perspectives accurately and fairly?
- Does ChatGPT's language remain neutral and objective, or does it unknowingly use charged or biased language?
Please note that you should inform your students that AIs in rare situations can “hallucinate” or make things up. So students should take every piece of advice or explanation critically and evaluate that advice.
Rubric
As an example, we have created the following rubric that a teacher can use to grade the tasks/assignments we created.
| Grade | Criteria |
|---|---|
| 5 - Excellent | Demonstrates an excellent understanding of the selected concept. The analysis is thorough, detailed, and accurate. The student uses strong evidence from ChatGPT's responses to support their analysis. |
| 4 - Very Good | Demonstrates a good understanding of the selected concept. The analysis is mostly accurate and detailed, with minor omissions or errors. The student uses evidence from ChatGPT's responses to support their analysis. |
| 3 - Satisfactory | Demonstrates an acceptable understanding of the selected concept. The analysis could be more detailed or precise. The student uses some evidence from ChatGPT's responses to support their analysis, but it may not be thoroughly integrated. |
| 2 - Fair | Demonstrates a limited understanding of the selected concept. The analysis is superficial or contains significant errors. The student does not adequately use evidence from ChatGPT's responses to support their analysis. |
| 1 - Poor | Fails to demonstrate an understanding of the selected concept. The analysis is incorrect, highly incomplete, or missing. The student does not use evidence from ChatGPT's responses to support their analysis. |
