Echo Dot (3rd Gen) - Smart speaker with Alexa - Charcoal

Use your voice to play a song, artist, or genre through Amazon Music, Apple Music, Spotify, Pandora, and others. With compatible Echo devices in different rooms, you can fill your whole home with music.

Buy Now

Wireless Rechargeable Battery Powered WiFi Camera.

Wireless Rechargeable Battery Powered WiFi Camera is home security camera system lets you listen in and talk back through the built in speaker and microphone that work directly through your iPhone or Android Mic.

Buy Now

How to create a Vocal Assistant (or chatbot) with Ionic Angular and Google Dialogflow

0
113


Alexminichino

The voice assistant market is also recently affecting the app market, people want smarter apps that do things for them.

Thanks to Dialogflow service we can create a bot in no time, defining training phrases, answers and parameters will be a breeze.
We can also connect webhooks to our backend to better manage requests and data.
By combining communication with the bot with speech to text and text to speach we obtain a real vocal assistant like Google Assistant or Alexa.

Ionic is a very powerful tool, if we want to create cross platform mobile applications without wasting time.

The first step is the creation of new bot (or agent) on Dialogflow.
We should go to
Dialogflow console and log in with your google account.

Click on “Create Agent” and fill out the fields:

then, click on “CREATE”.

Now, you can edit other things, like avatar, description or additional languages but go on.

Now, it’s time to create the intents !
To create a new intent, we need to specify a list of training phrases, and a list of responses, and so doing we define a Presentation intent:

When user says “ Who speech?”, “ What are you?”, or “ Who are you?”, the bot responds with “Hi, i’m your personal assistant”, “I’m an artificial intelligence” or “I’m a bot at your service”.

Note: More training phrases make your bot better thanks to machine learning.

Now, you can already test!
To do it, you just need to go on the right side of console and ask a question at your bot:

Well, it was kind of easy, right?

Now, let’s go on to the ionic side, the first step is a creation of new project:

ionic start vocalAssistant blank

and install the (DialogFlow library) API-AI javascript library:

npm install api-ai-javascript@2.0.0-beta.21ionic cordova plugin add cordova-plugin-apiai@2.0.0 => to test

Note: We use a cordova plugin, then the app can be only deployed on real device.

Now, we need to retrieve an API key in settings page:

and save it in environments/environments.ts:

The next step is a genereation of “Chat Service”:

ionic generate service sevice/chat

First of all, we need to import environment, ApiAiClient, and BehaviorSubject, the latter allow to share values between componets.

Then, we define a Messageclass to handle users phrases and bot responses.

Following, we define the ChatService that allows to send requests to DialogFlow and manage responses.
Also, we put a profile_picproperty in the Message objects to differentiate user and bot.

Note: We have to remember to put 2 images in the asset folder, one for user and one for bot.

The following code shows what is described:

Now, it needs a page to display “conversation” flow, about that, we will create a SpeechPage.

ionic generate page speech

An example of UI could be the following:

In this example we have an Button to activate the speech recognition and a binding of messages, all with an activant UI thanks to this CSS:

Now, we need to add the following libraries for speech to text and text to speech:

ionic cordova plugin add cordova-plugin-speechrecognition
npm install @ionic-native/speech-recognition
ionic cordova plugin add cordova-plugin-tts
npm install @ionic-native/text-to-speech

We have to remember to add the imports in app.module.ts and add it in providersarray under @NgModule

At this point, let’s move on Logic of the page:

in this file we have the core of our assistant, the startListeningfunction provides to activate microphone and sends detected sentence of user.
Instead, the ResponseHandler function takes care of responses of bot, adding the response in an array and activating the text to speech function, allowing to bring to life at out assistant.

In this example, I used the actions of DialogFlow intents to open a pages in Ionic, the doActionfunction takes care of this, but this aspect can be changed and used for different purpose.

Other functions need to handle permission, navigation etc.

The last step is a creation of assistant button that can be placed in all pages of our app:

ionic generate component assistant

Define the markup:

the style:

and the component class:

Ok, now we can use the Assistant component in every page, for example the home page:

Let’s add the selector in home HTML:

Let’s add import in home.module.tsand put in entryComponentsand declarationsArray:

The game is done !
Our assistant is complete, we just have to do the build and run.

You can run app on web browser but only to get UI preview, I suggest you to “mock” a conversation to edit your UI.
To run on web browser you can execute:

ionic serve

To running app you just need to execute:

ionic cordova run android 
//OR
ionic cordova run ios

After clicking the assistant button, the app asks microphone permission for working, then we can ask something and the app will speech with bot response.

Note: We have to remember that question and answers dependes of intents on Dialogflow portal.

You can find the project on my github: https://github.com/alexminichino/vocalAssistant

I hope I’ve been of help, thank you for reading.

Enjoy! 😎





Read More

LEAVE A REPLY

Please enter your comment!
Please enter your name here