Experiment Description

1. ChatGPT was told to write a comparative essay about "Mall Shopping and Online Shopping".
2. We generated a B1 English level option. (B2 of the Common European
   Framework of Reference for Languages (CEFR))
3. We then compared them to a real student assignment (of B1 level) on the same topic.

Examples (ChatGPT)

ChatGPT generated output for the task above:

The proliferation of the internet and digital technology has dramatically shifted the landscape of retail, spawning two distinct methods of consumerism – the time-honored tradition of shopping at a brick-and-mortar mall and the modern convenience of online shopping. Each of these modes offers unique experiences that cater to the versatile preferences and demands of contemporary consumers.

The experience of shopping, one steeped in human tradition, stands out as a key differentiation between the two modes. For some, the act of visiting a mall transcends a mere commercial transaction, transforming into a sensory, and even social, adventure. The tactile experience of feeling products, the immediate gratification of a purchase, the festive ambiance of a bustling mall – these elements provide a certain palpable satisfaction that online shopping cannot replicate.

We asked ChatGPT to generate at B1 level - The text in red highlights unrealistic writing.

In the age of digital technology, the practice of shopping has transformed dramatically. Where shopping traditionally was associated with visiting brick-and-mortar stores or shopping malls, the advent of e-commerce has given birth to the convenient alternative of online shopping. This essay will compare and contrast these two shopping methods - shopping at a mall and shopping online - using the point-by-point method to illustrate the key differences and similarities.

Examples (Student)

B1 - The text in red highlights realistic writing (such as mistakes, human-writing etc.).

Shopping is part of our life and just like we need other necessity stuff to stay alive we also need to shop as well. It doesn’t really matter the amount or the quantity of products you are buying but it’s all about the time you spend to doing it. Over past years people have been shopping at malls for most of the household stuff including food etc. Therefore, it is a fact that some people still prefer doing shopping at the mall with various personal issues and preferences. Some might argue that they get to enjoy a day off by shopping at the mall with friends and family otherwise good source of place where you can pass the time. To some the mall is their number one place to buy merchandise. To others the mall is the place where they can go and get their appetite satisfied or unwind through various entertainment activities. On Other hand with the coming of the internet online shopping has been revolutionized.

Many people are going for online shopping as compared to going to the local mall. All this is true about shopping at the mall more fun to most people but the fact is that the hassle of looking to what you want is undeniably tiring. In other words, as the internet constantly gained popularity majority of people are going for shopping online. Shopping online has taken preference over the conventional shopping at the local mall since it has been made for the benefit of consumers offering many advantages.

Our Impressions:

After comparing more outputs to real student work (other than the examples above) we have the following observations to make.

• There are no mistakes in the AI text.
• Vocabulary used in the AI text is more sophisticated
• The essays generated by the AI aren't that creative, they're rigid and mechanical lists of data. Whereas people tend find more creative relationships to outline when writing comparative essays.
• The AI text has no individuality.
• The AI text is quite dry and boring to read.

"Would a student in my class write like that?"

• The human text has low-level vocabulary
• It showcases some original thinking, like mentioning history and business management themes to illustrate a point, whereas the AI just followed instructions flawlessly.
• Human text also tends to have easy-to-miss mistakes such as punctuation marks. This is true for all writers from A1 to world-renowned authors.

Please note that you should inform your students that if they are going to use AI to generate content, they are responsible for their own work. The AI in rare situations can “hallucinate” or make things up. So students should take every piece of advice or explanation critically and evaluate that advice.