Let us take an example of Chatbot which is support to handle 15 languages. Following are the challenges —
- A Language expert — To properly train a Chatbot for perfect response, one has to engage a Language expert and train the LUIS and QnA maker to understand and provide best fit response. Further, there is a need to create a resource file to provide user with a response in the language of their preference. For 15 languages, you would need 15 experts, unless someone knows multiple of those in the list 😋
- Multiple LUIS Services — While LUIS supports more than 30+ Languages and allow a trainer to provide utterances for all those languages, one has to create 15 different LUIS Services to handle each language and this could be too costly. Further, Chatbot has to redirect to right LUIS Service based on the localization of user. So you need 15 LUIS Services running😳
- Multiple QnA maker Services — Similar to LUIS, QnA maker need to have separate Knowledge base Services for each language. Costly again 😳
- Resource Files — Create 15 different resource files for your Bot to handle conversation workflow. 😪
- Proper Testing and Active Learning — It’s difficult to train above services with perfection and test them. This could lead to issues in response and intent detection.
Your approach to solve the requirements depends on what you have in terms of language expertise, no. of languages to serve, cost implications and performance. Following are two approaches possible —
1. How Businesses are Winning with Chatbots & Ai
2. 21 Best Telegram Bots That Everyone Should Know
Given you have following challenges and requirements –
- Less Budget for Azure Services and maintenance
- No language expert readily available
- 15 Languages to handle
For above scenario, one can use Microsoft’s Text Translator Service which support text translation for more than 30 languages (Refer official docs for more details). This approach follows simple steps –
if language = English:
then{
a. GET intent from LUIS OR GET response from QnA
b. Display response to user
} if language != English:
then{
a. Convert the text to English using Microsoft Translator
b. GET intent from LUIS OR GET response from QnA
c. Convert the response back to preferred language using Microsoft Translator
d. Display response to user on chat
}
Advantages of using Microsoft Translator Service
- None or less dependency on Language expert
- No need to create multiple services for LUIS and QnA
- Reduced cost due to single LUIS and QnA maker service
- No Resource File — No need to create Resource Files for each language
- Quicker development — As there is no need to train LUIS and QnA for each language. That head-ache is not given to Microsoft Text Translator Service
Disadvantages of using Microsoft Translator Service
- Lack of Perfection — Text translator does not have complete human parity on language translation and hence can jumble words and can kill the intent or meaning out of the sentence which can embarrass the bot 🤕 and impact user experience
- Performance Latency — Added latency in response time as translator has to convert each and every text to and fro.
Given you have following requirements –
- Lack of confidence on Text Translator service to translate text with perfection and without loosing or jumbling intent and entities
- Performance to ensure user get rapid response and does not get delayed with translation service
- Possibilities of adding more languages to a Bot in future
For above scenario, one case use the approach of training LUIS and QnA service for each language of preference. The approach follows the sample steps –
if language = English:
then{
a. GET intent from English LUIS or English QnA Service
b. Display response to user
}if language = Spanish:
then{
a. GET intent from Spanish LUIS or Spanish QnA Service
b. Display response to user
}.
.
.
And so on......
Advantages of using Language specific LUIS and QnA Approach
- Better performance and reduced latency by removing extra processing time required in language translation
- Perfect response and properly trained model for LUIS and QnA in each language ensuring best user experience
- Easy to add more languages by plugging-in extra component set for newly required language.
Disadvantages of using Microsoft Translator Service
- Cost — Having multiple services for LUIS and QnA would cost extra
- Complexity — Added complexity to handle and manage multiple set of QnA and LUIS Services
- Increased development time — Increased development time due to training and testing required for LUIS and QnA
