Chatbots are computer programs that attempt to engage a human in a dialogue. Chatbots
originated in the 1960s with MITs Eliza. Advances in language technology have produced
chatbots with increasing linguistic quality and sophistication, leading to their potential to serve as
a tool for CALL. With the recent breakthroughs, they have the potential to become ubiquitous in
society.
This project aims to assess the effectiveness of chatbots as a pedagogical tool for learning
English as a foreign language. Try out the following chatbots, and select two of these chatbots
for evaluation in the rest of your project:
Mitsuku: https://www.pandorabots.com/mitsuku/
CleverBot: https://www.cleverbot.com/
Bing: Go to bing.com and click Chat to sign up
ChatGPT, Claude, Dragonfly, or Sage: accessible at poe.com
Chatbot session as NGSL Level 2 user
You will simulate an English learner at New General Service List (NGSL) Level 2. In other
words, you can understand and write words listed up to Level 2 in the NGSL. You may use the
Online Graded Text Editor (https://www.er-central.com/ogte/) to ensure that your input matches
your expected vocabulary proficiency.
You will have a chat session with each of the two selected chatbots. Each chat session should
contain a total of at least 40 turns, i.e., there should be at least 20 user turns. Use text input only;
do not use speech input, to avoid speech recognition errors in the chat session.
Chatbot session as NGSL Level 5 user
Proceed in the same way as above, except you will simulate an English learner at NGSL Level 5.
To submit
Report (max 1000 words): Assess and compare the utility of the selected chatbots for learning
English as a foreign language. Your discussion may comment on the quality and complexity of
the chatbot replies; whether the user would benefit more from interacting with chatbots or
reading materials in textbooks; and any other relevant considerations. Your assessment should
be based on statistical evidence and illustrated with examples.
Data: There should be a total of four chat sessions. The content of these sessions, including both
the user input and chatbot replies, are to be saved in the template Excel file.
