Echo Dot (3rd Gen) - Smart speaker with Alexa - Charcoal

Use your voice to play a song, artist, or genre through Amazon Music, Apple Music, Spotify, Pandora, and others. With compatible Echo devices in different rooms, you can fill your whole home with music.

Buy Now

Wireless Rechargeable Battery Powered WiFi Camera.

Wireless Rechargeable Battery Powered WiFi Camera is home security camera system lets you listen in and talk back through the built in speaker and microphone that work directly through your iPhone or Android Mic.

Buy Now

How to make your first chat bot in 50 lines of code — theory and practice


Most of modern chat bot platforms consist of 3 main things — intent recognition , slot filling and dialog graph.

1.1 Intent recognition

Intent recognition is a text classification task which goal is to capture specific intent behind a user query. This is motivated by the fact that users tend to formulate their request in a lot of different ways so we need to have a system that is able to tell if those messages relate to the same thing or not. Let’s illustrate this with an example of a bank chat bot, where users can ask the bot to withdraw money:

Visualization of intent detection system

You can see that although the requests are formulated in a lot of different ways and in different styles they all mean basically the same thing and chat bot should react in the same way. Therefore we need text classification model that captures the semantics behind user sentences and assigns them to the specific predefined class.

1.2 Slot filling

Once we know what action the user wants to take we need to capture specific parameters of those actions. For example, if you want Alexa to play your favorite song, you want her to play this specific song not just any song, so besides detecting intent chat bots also need to perform a task that is called slot-filling.

1.3 Dialog graph

Visualization of dialog graph

Another requirement for chat bot functionality is dialog graph. It’s goal is to steer conversation in the right direction. For example when you say “Check the weather” the chat bot could then ask “What day should I check the weather for?” and next it will be looking for intents like ‘tomorrow’ or ‘today’. The important part here is, there would be no point in asking the second question without the first one, so there is a need for a system that stores the information of the point in the conversation where we are and what are the possible next states.

1.4 Our chat bot

In this tutorial our goal is to create a simple chat bot, so we are going to focus only on intent detection task and simple dialog graph model. This is enough to make a chat bot that is able to answer FAQ and conduct as simple conversation.

Our goal in designing an intent detection is to create a system that, given a few examples for intent, can detect that a sentence given by the user is similar to these examples and therefore should have the same intent.

The problem behind this system is that we have to design a system for checking if 2 sentences are similar. This could be achieved by eg. counting how many overlapping words are in the new sentence and the sentences in training data set. This is however a naive approach because a user can use a word that has similar meaning, but is different from the ones in the train examples.

2.1 Word embedding

A solution here is to use word embedding.

Word vectors ref :

Word embeddings are mathematical representations of words encoded as vectors in n-dimentional space. Similar (used in the same context) words are close to each other in this space. This means that we can compare 2 or more words to each other not by e.g. the number of overlapping characters but by how close they are to each other in they embedded form.

2.2 Sentence embeddings

From word embeddings we can construct embeddings for the whole sentence. This can be done in a variety of ways, we can simply take the average of the word vectors, use weighted average to check how important the words are by e.g tf-idf coefficient or even use more advanced methods like transformer neural networks.

2.3 Similarity

Once we have prepared embeddings for the sentences we have to design a way for comparing them. A simple widely used method here is cosine similarity that measures similarity between two vectors as the angle between them.

Cosine similarity ref:

To create the sentence embedding we are going to use flair library. This library is based not only on static word embeddings but also analyses the words character by character which helps in dealing with out-of-vocabulary words.

In our model we are going to embed the examples for each intent and then, while processing the users message, find the most similar one. This approach is mainly taken as fast and simple one, illustrating how embedding work. Most of modern systems use neural networks (link to related articles can be found at the end), however this approach can still be used if you want to design a system that is fast and and doesn’t use a lot of resources.

We begin our program with creating the outline of the model.

outline of the program


1–9 : importing necessary libraries
11 : initialization of the flair model for creating embeddings of sentences. We are using English word embeddings and mean polling method for creating sentence embeddings from word embeddings.
13–20 : chatbot class, this class has two static methods one for creating embeddings and one for processing user message and answering it.

3.1 Preparing embeddings

Firstly we need to prepare a file containing our intents and their examples. This is a json dictionary that uses intents as keys and tables of examples as values.

intents.json file

Next we need to to create a function that constructs embeddings for the examples.

Function for crating embeddings


4 : Creating new python dictionary for the embeddings
5–6 : Opening the input file and loading it to python dictionary
7–8 : For each intent we create a table in the embeddings dictionary
9–12 For each example in the intent, we create a Flair sentence object that we can later embed using the model specified earlier. Finally we add the embedded sentence to the table
13–14: If the file doesn’t exist, we create it
15: We save the embedded dict. We use pickle instead of json to store the numpy arrays

3.2 Answering the message

answers.json file
function for for answering messages


3 : We use the embeddings model
4 -5 : We load load the embeddings file created earlier
6–8 : Embedding of user message
9–10 : Initializing best intent and best sore variables
11–16 For each intent we loop through it’s embedded examples and check the cosine similarity between users message and those examples.
We chose the intent, which example has the highest similarity with the new message
17–18 : Loading the answers dict
19 : Checking if intent chosen by the system is in the answers dict
20 : Return random answer from the ones assigned to the chosen intent

In this format the chat bot has to choose one of the intents provided. This means we have no way of detecting if user said something that doesn’t belong to any of the intents. A possible solution is to check the numerical values of the cosine similarity and based on those observation assign a threshold value below which the bot will classify the message as the one it doesn’t know how to answer.

Read More


Please enter your comment!
Please enter your name here