It is very crucial for any company to create positive and engaging customer experience to make end user feel that they are important. Today many businesses are making use of conversational chat bots to deliver answers faster to the end consumers, and this is in fact very much important because in this age of instant gratification, consumers expect too much in a very little time. Most of the companies today engage with their end users to provide customer support, flight details, product inquiries, etc. There are even numerous conversational AI applications including Siri, Google Assistant, personal travel assistant, and others which personalizes user experience.
There are mainly two forms of conversational chat bots: Rule Based and Self Learning.
- Rule Based Approach: Here the bot answers query based on certain rules defined. This is a very simple and nice approach to start with, but it can fail to handle complex questions.
- Self Learning Approach: This uses Machine Learning/Deep Learning techniques to answer questions, and this is definitely efficient in contrast to the Rule Based Approach.
In this article, I will demonstrate to you on how to build basic chatbot in Python using Rule Based Approach with the use of regular expression. We will begin by understanding the essential components of Natural Language Understanding namely intents and entities; we shall then use the power of regular expression to create rule based chat bot. Although this is a tedious approach, this is a good starting point to understand.
Natural Language Understanding
Natural Language Understanding (NLU) is an art of extracting the purpose or intent of the text, which in our case would be question. Also, we need to pull out right piece of information from the text.
Example Query: I would like to book my air ticket from A to C.
Following are the two important components of NLU:
Intents: In this case, objective of the end user is to reserve a flight from one location ( A ) to the another location ( C ). Therefore, the intent of this question would be: reserve ticket . Once we are able to find the intent of this question, we can then use it to map it to our answer.
Entities: There are two entities in this query which is important in finding the answer; Departure City (A) and the destination city (B). We can also include date and time here.
To summarize, intent “reserve ticket” has the following entities:
After we know all this, we can then use it to search from our database to retrieve the information for our end user.
We will now build a very simple bot using the power of regular expression for the website specializing in gifts and packaging.
An important step here is to to classify user’s question into an intent to identify the purpose of the question. For example, the intent of these questions, “describe yourself”, “explain yourself”, “identify you”, would be “about chatbot”. End user here is interested in finding who this person is.
Step1: You can start by building dictionary containing synonyms of related words. This is very useful step because the questions can be addressed in many different ways. For example: describe can be replaced by its synonyms such as explain, identify etc.
from nltk.corpus import wordnet
for word in list_words:
for syn in wordnet.synsets(word):
for lem in syn.lemmas():
Step 2: Build the keywords dictionary in Python to identify intent of the question. Here we will make use of regular expression in order to match the pattern of question. For example, user can greet the chatbot by saying “hello!”, “hi”, “hello, hope you are good” ; these all variants will come under the intent of “greet”.
Note: .* means “0 or more of any character”. Any number of characters could follow and precede an important keyword, so we can make use of regular expression (.*) to match question. To match whole keyword use b (https://stackoverflow.com/questions/15863066/python-regular-expression-match-whole-word)
for synonym in list(dict_syn['hello']):keywords['about_chatbot']=
for synonym in list(dict_syn['describe']):
keywords['about_chatbot'].append('.*\b'+synonym+'\b.*'+'\byou\b'+'.*')..........CREATE FURTHER RULES................
Values for an intent “greet” could either contain “hi”, “hello” etc so now we need to compile regular expression pattern into regular expression object using re.compile() (https://docs.python.org/3/library/re.html), which can be used for matching using its match(), search() methods. Different values can be combined using the bitwise OR (the | operator).
for intent, keys in keywords.items():
Step 3: Now that the system has capability of identifying intent of the question, we require another dictionary consisting of responses to the intent matched.
'greet':'Hello! How can I help you?',
'about_chatbot':'Hi, My name is Sam. I am here to help you out',
'role': 'I help people in understanding functionality of our product website. I also assist the end user in purchasing and refunding our product',
'about_site':'We help people around the world to celebrate important occassions with a special gift.',
'site_functionality':'Please use the menu on the top to navigate and explore different gift categories. We specialize in anniversary gifts, birthday gifts, and other types of gifts',
'refund':'Well, we value our customers. You can refund the amount if you return it back within a day. For more information explore our refund section on this link: ____',
'located_in':'We are based in ______ near _____',
Step 4: We just need to define three methods to match intent, respond and send message.
matched_intent = None
for intent,pattern in patterns.items():
if intent in responses:
Step 5: Test it ( try different variants)
send_message(“hello, hope you are good”)
send_message(“hello hi, “)
send_message(“please tell me how to operate this website”)
send_message(“please tell me how this website function”)
send_message(“please tell me how to operate this site “)
send_message(“I need to know refund policy”)
send_message(“please tell me return policy”)
send_message(“Well, where are you located?”)
send_message(“you are situated where??”)
send_message(“who are you????”)
send_message(“your role please”)
send_message(“what is your purpose here???”)
send_message(“please tell me about this website”)
In this article, we started by highlighting the importance of chatbot, and the two approaches to create conversational agent namely Rule based and Self Learning. We then went ahead to discuss an important concept of Natural Language Understanding, which includes intents and entities. We then discussed the rule based approach with the use of regular expression; this is a tedious approach, and managing rules here could be complex. Machine Learning along with the word vectors to perform intent classification would be a good approach (you can try nearest neighbour/cosine approach for simplicity).
We, however, didn’t see on how to use regular expression to extract entities from the text; this I leave up to you.
There are also many advanced approaches available including Sequence Modelling to add memory element into chatbot.