Overview

This module implements the ReactNative library named "Voice", which supports both online and offline on the device for iOS and Android. https://github.com/react-native-voice/voice

The module includes the widget, the datamodel and nanoflows to handle the widget outcomes, and it also includes a sample implementation.

Finally an optional module template is provided in the resource folder to manage intents based on keywords, so the voice can trigger actions in the app.

Documentation

This speech to text module is the implementation of the "voice" library : https://github.com/react-native-voice/voice

it works online and offline. It supports iOS and Android. It provides a module template for intent management.

==================

WIDGET

==================

Widget Pre-requisites

IMPORTANT : This module can't work with the generic "Make It Native" app. You have to build your custom app with the correct permissions.

- Android : 'RECORD_AUDIO',

- iOS : 'SpeechRecognition'

Widget Sample

To try the sample, just call from a native page the nanoflow "SpeechToTextSession_InitAndLoadSample_USE_ME" in the folder "Sample".

Widget implementation

Clone the snippet “SpeechToText_WidgetSnippet” or the whole sample sample

You may have an error on the widget because the event “onResult” doesn’t find its nanoflow. You can solve this error with two approaches :

- If you don’t want to implement “intent management”

o Duplicate the nanoflow "SpeechToTextSession_onResultSample" in the folder "CLONE_ME", and move the clone in your own module.

o Edit the widget and associate the "onResult" event with the nanoflow that you have just created

- If you want to implement “intent management”

o importing the addon module will solve the error. See below instructions.

Widget “SpeechToText_WidgetSnippet” how it works

The snippets “SpeechToText_WidgetSnippet” contains the widget and gives you an example of implementation. The widget starts listening as soon it’s loaded, and automatically stops when the recognition ends. It automatically updates the attribute “status” of the “SpeechToTextSession” entity. To restart the recognition, you need to hide and show the widget again. The snippets “SpeechToText_WidgetSnippet” implements the condition visibility allowing you to control the widget.

=======================

INTENT ADDON

=======================

Do you want your speech to trigger actions by keywords ?

Intent management addon - Installation

1- In studio pro, import the module *.mpk file located in the “resources”:

" \resources\SpeechToTextNative_NBD_IntentMgmt__vX.X.X.mpk"

This is provided in a separated module because it's more a template to be customized. It expect you to customize the domain model (an enumeration) and it's not easily updatable.

2- install dependencies

- CommunityCommons

- DataWidgets (Datagrid2)

3- On Startup of the application, add the "onStart" microflow to generate the IntentRules

4- Add a link to the responsive admin page to manage keywords.

5- Check or change the widget "onResult" event to trigger the nanoflow “SpeechToTextSession_onResult” in folder "CUSTOMIZE_ME"

Intent management addon - Implementation

This module is a template, you can customize it and make it yours.

Please change the "IntentEnum" and define corresponding actions in the "SpeechToTextSession_onResult " nanoflow.

Each intent of the enumeration will be associated to keywords. Those keywords can be defined with 2 methods

- hardcoded in the model as enumeration captions and separated with semi-column";".

- managed at run-time with the Admin page.

The “SpeechToTextSession_onResult” nanoflow includes a call to the nanoflow "SpeechToText_DetectIntent" that search for keywords in the text suggestions.

Note : This nanoflow running on the device requires the IntentRules to be generated (triggered at application start) and these intentRules to be synchronized on the device.