Overview

This module implements the ReactNative library named "Voice", which supports both online and offline on the device for iOS and Android. https://github.com/react-native-voice/voice

The module includes the widget, the datamodel and nanoflows to handle the widget outcomes, and it also includes a sample implementation.

Finally an optional module addon is available to manage intents based on keywords, so the voice can trigger actions in the app. See Speech To Text Native Offline - Intent Addon

Documentation

This speech to text module is the implementation of the "voice" library : https://github.com/react-native-voice/voice

it works online and offline. It supports iOS and Android. It provides a module template for intent management.

IMPORTANT : This module can't work with the generic "Make It Native" app. See "Widget Permissions and Dependencies" chapter below.

Widget Sample

To try the sample, just call from a native page the nanoflow "SpeechToTextSession_InitAndLoadSample_USE_ME" in the folder "Sample".

Widget implementation

If you want to implement "itent management" where keywords can trigger actions, directly go to below chapter.

1. Clone the "1_CLONE_ME" folder, it contains the snippet “SpeechToText_WidgetSnippet” which includes the widget and its required visibility conditions
2. Clone or inspire from the "3_sample" folder
3. Edit the widget and associate the "onResult" event with your own copy of the nanoflow "SpeechToTextSession_onResultSample" that was cloned in previous step

Widget “SpeechToText_WidgetSnippet” how it works

The snippets “SpeechToText_WidgetSnippet” contains the widget and gives you an example of implementation. The widget starts listening as soon it’s loaded, and automatically stops when the recognition ends. It automatically updates the attribute “status” of the “SpeechToTextSession” entity. To restart the recognition, you need to hide and show the widget again. The snippets “SpeechToText_WidgetSnippet” implements the visibility conditions allowing you to control the widget.

Widget Permissions and Dependencies

This module can't work with the generic "Make It Native" app. You have to build your custom app with the correct permissions after you have implemented the module (used the widget in a visible page) so the dependencies are implemented.

Permissions to be added in the "build app" wizard

- Android : 'RECORD_AUDIO',

- iOS : 'SpeechRecognition'

Dependencies visible in the GIT repo created by Mendix "Build app" wizard, check "YourProjectRepo/.dependencies". The build is correct if it contains

"@react-native-voice/voice": "3.2.4",

"@expo/config-plugins": "2.0.4"

Intent management addon

Do you want your speech to trigger actions by keywords ?

Download from the marketplace the addon "Speech To Text Native Offline - Intent Addon"

This is provided in a separated module because it's more a template to be customized. It expects you to customize the domain model (an enumeration) and it's not easily updatable.

NOTE : Since this module is meant to be customize, you may want to import this module as a "custom module" instead of a "marketplace module", to do so, download it in your file system first from a web browser in the "release" tab, and import it with "right clic on project > import module package".

Troubleshooting

If react native app debugging shows the warning "possible unhandled promise rejection", your app has probably been not correctly built. Verify Permissions and dependencies (see above). If you miss the dependencies, make sure the widget is used in a visible page and rebuild the app.