The commitment of Mega Italia Media in the study and application of Artificial Intelligence (A.I.) in online training through the DynDevice LMS eLearning platform has led to the publication of the paper "Evaluating different Natural Language Understanding services in a real business case for the Italian language" in the international journal Procedia Computer Science.
The paper was presented on September 16-18, 2020 at the "24th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems".
The experience, developed thanks to the close collaboration between the Department of Computer Engineering of the University of Brescia (UNIBS) and Mega Italia Media Spa, eLearning company of Castel Mella (BS), is linked to the creation of a chatbot for DynDevice LMS: DyBot. DyBot has been conceived to give an automatic response to the users during the delivery of eLearning courses.
What is a Chatbot?
The term chatbot comes from the union of two words: "chat" (instant messaging system) and "bot" (diminutive of the term "robot"); the latter term is generally used to define a machine, or in the case of computer science a software, able to automate one or more tasks.
So a chatbot is a software that allows you to talk autonomously with a person through a text system. Generally this type of software relies on complex algorithms of Machine learning and Artificial Intelligence in order to interpret the sentences that the user submits to it in natural language (human language).
The core of a chatbot are the complex algorithms of Natural Language Understanding (NLU) that allow the chatbot to interpret the human language and capture the essential information. The information that an NLU engine can extract from a sentence are mainly two: the intent and the entities. The intent is the general meaning of the sentence, what the user wants to say to the chatbot while the entities do not consider the general meaning, but look for specific information to extract (for example, a date or the name of a city). The incredible power of these technologies is that they allow, with a certain degree of error, to correctly interpret a sentence written in any way even with spelling errors.
Here is an example to better clarify this concept: the user wants to know if he can stop using the course and resume it later or if he has to watch all the lessons in a single session; in this case the chatbot must be able to correctly classify all the following statements:
- Good morning, can I resume the course later?
- If I interrupt a video, when I reconnect to the platform will I have to start from the beginning again?
- I have little time in the office, can I watch them at home?
For the three previous sentences the NLU engine must recognize the same intent, i.e. "Resume course later" (note that the sentences are written completely differently and the third has obvious spelling errors). So, as we can understand from the example, the strength of chatbots is the potential ability to correctly interpret endless combinations of different sentences with the same purpose and to be robust to spelling and syntactic errors.
Train a Chatbot
Many technologies studied in modern computer science involve the use of algorithms that are capable of self-learning through experience. To understand this complex concept in an intuitive way, we can think of algorithms with self-learning as very young children who learn through direct experience and the gratification they receive from their parents or educators. For example, a small child learns to say mom or other words because the parents encourage him to say the same word dozens of times and, following numerous attempts, when the child is able to say the word, he receives from his parents gratifications of affection that allow the child to understand that he has done a correct action. This method of learning works well, so much so that, during the first years of life, even though they do not know grammar, children are able to learn to speak correctly thanks to the examples given to them by the surrounding world and the figures of reference that help them in this process.
The technologies used by NLU engines are very similar to the child in our example, because to learn to correctly interpret the intent underlying a sentence they need to receive several tens or hundreds of examples of sentences (properly classified) having the same intent, but different wording. In technical jargon this phase is called "training". The developer, like the parent of the example above, must supervise this process and understand if the algorithms learn correctly. If they do not, they must provide additional examples to help the algorithm learn further. This process is repeated until an adequate performance level is reached (this last step is called testing).
The development of Dybot
The chatbot developed by Mega Italia Media (MIM) is called DyBot and in its first version allows you to interpret and respond autonomously to a specific class of requests from learners. In the future, it will allow you to respond to an increasing number of requests from users. In fact, requests from learners may evolve over time (for example, the introduction of a new feature in DynDevice LMS could lead to the emergence of a new class of requests from learners). Dybot is a technology designed to meet these needs, both short and long term. DyBot doesn't replace the human tutors of Mega Italia Media, but it works alongside them, answering frequently asked questions with standardized answers. However, when it is not able to provide the best answer, it automatically forwards the question to the human tutor so that the user can be assisted in the best way.
The development of Dybot has been developed following a solid and rigorous scientific and experimental process (Analysis, Classification, Implementation, Training and Testing). In particular:
- in the first instance we proceeded to analyze the history of requests received by tutors through the chat implemented in the DynDevice LMS eLearning platform. The analysis phase allowed to establish which classes of questions were the most frequent and then focus the attention of the chatbot to the resolution of those requests.
- In the classification phase the requests received were "labeled", i.e. each request was associated with a specific intent.
- Consequently, the code useful to integrate the chatbot into the DynDevice LMS messaging system was developed.
- Finally, the NLU engine has been trained to recognize the most common intent underlying the requests that DynDevice LMS users normally submit to tutors.
The content of the international paper on Dybot
The contribution starts from the assumption that, in the last decade, leading private companies operating in the IT sector have developed cloud-based platforms with natural language understanding (NLU) functions, widely used for research and commercial purposes. One of the main reasons for the success of NLU platforms is that they allow to simplify the development of Chatbots or Spoken Dialogue Systems (SDS), in many cases, without the knowledge of programming languages.
The article presents a general description and a taxonomy that brings together the features and constraints of the different cloud-based NLU services available on the market. In addition, it provides an assessment and comparison on the ability to recognize the underlying intent of the different phrases. The sentences used are a collection of requests made by users in Italian on the DynDevice LMS eLearning platform.
The analysis is very useful to help the company, owner of the eLearning platform, to understand in which NLU platform is best to build a chatbot for the Italian language with the aim of answering the most frequently asked questions. Previously, solid studies on the topic have been carried out only on English language platforms
The article can be downloaded here.
Translated with www.DeepL.com/Translator