How To Replay A Node

[drod7425]

My application is a mobile VR application. The conversation format is linear and reads like an interview (no player decision affects what the NPCs say). However, player input must be recorded and logged. Because the phone will be strapped to the user's head, we have decided to not use a UI with Dialogue System. Instead, all lines will be communicated via voiceover and all player voice input will use IBM Watson's Speech-To-Text service to record their answer and return a transcript.

Now that you know the general project background, here is the issue: Sometimes the microphone won't pick up the user's voice or they just don't say anything and Watson returns a speech error. In that case, I'd like the speaker to say some pre-recorded line about how they couldn't hear you, then have the current node's voiceover replay, including the sequence (I made a custom "Listen" sequence command that runs the speech-to-text). I've tried using:

Code: Select all

DialogueSystemController controller;
controller.ConversationController.GotoState(controller.CurrentConversationState);

It replays the voiceover, but it doesn't seem to replay the sequence. Am I on the right track? Is there a better way?

[drod7425]

Ah, you know what - Tony has linked the Backtracker to me before and I think that example could be helpful in this case as well. I will try ending the conversation and starting it back up at that entry.

[Tony Li]

Hi,

When devs switch to voice recognition and text-to-speech, they usually make a custom dialogue UI -- sometimes as a subclass of StandardDialogueUI, sometimes from scratch as an implementation of the IDialogueUI interface.

In either case, I recommend overriding/implementing the dialogue UI's ShowResponses method. ShowResponses receives the most recent subtitle and a list of possible responses.

In this method, kick off a coroutine that waits for the user's voice command. If it recognizes the command as a valid response, call OnClick(response).

If it doesn't recognize the command, call ShowSubtitle(), passing it the most recent subtitle, and then resume listening for the user's voice command.

This way there's no need to backtrack.

[drod7425]

I decided to use the interface route, because I will not need any UI implemented by StandarDialogueUI. Unfortunately, this is not working. Because all my implementation of ShowSubtitle does is cache the subtitle, I'm guessing that there are things I need to further implement, but I'm unsure of what's required. Here is my code:

Code: Select all

using UnityEngine;
using PixelCrushers.DialogueSystem;
using System;

public class CustomUI : MonoBehaviour, IDialogueUI
{
    IDIA.HealthcareDisasterResponse.SpeechToText speechToText;
    Subtitle currentSubTitle;
    Response currentResponse;

    public event EventHandler<SelectedResponseEventArgs> SelectedResponseHandler;

    public void OnEnable()
    {
        IDIA.HealthcareDisasterResponse.SpeechToText.OnSpeechRecognized += this.SpeechToText_OnSpeechRecognized;
        IDIA.HealthcareDisasterResponse.SpeechToText.OnSpeechError += this.SpeechToText_OnSpeechError;
    }

    public void Start()
    {
        speechToText = FindObjectOfType<IDIA.HealthcareDisasterResponse.SpeechToText>();
    }

    private void SpeechToText_OnSpeechError(string error)
    {
        speechToText.Active = false; //Turns off speech to text
        ShowSubtitle(currentSubTitle); //Followed instructions, but unsure of how to tell DialogueSystem to replay the node
    }

    private void SpeechToText_OnSpeechRecognized(string transcript)
    {
        speechToText.Active = false; //Turns off speech to text
        SelectedResponseHandler?.Invoke(this, new SelectedResponseEventArgs(currentResponse)); //Invokes event with reponse
    }

    public void ShowSubtitle(Subtitle subtitle)
    {
        currentSubTitle = subtitle; //Caches the current subtitle so we can return
    }

    public void ShowResponses(Subtitle subtitle, Response[] responses, float timeout)
    {
        Debug.Log(responses.Length);
        if (responses.Length > 0)
        {
            speechToText.Active = true; //Starts a coroutine in another script that handles the speech to text (This is working)
        }
    }

    public void OnDisable()
    {
        IDIA.HealthcareDisasterResponse.SpeechToText.OnSpeechRecognized -= this.SpeechToText_OnSpeechRecognized;
        IDIA.HealthcareDisasterResponse.SpeechToText.OnSpeechError -= this.SpeechToText_OnSpeechError;
    }

    public void Open()
    {
    }

    public void Close()
    {
    }

    public void HideSubtitle(Subtitle subtitle)
    {
    }

    public void HideResponses()
    {
    }

    public void ShowQTEIndicator(int index)
    {
    }

    public void HideQTEIndicator(int index)
    {
    }

    public void ShowAlert(string message, float duration)
    {
    }

    public void HideAlert()
    {
    }
}

EDIT: All I really need to do is replay the (default sequence + any node-specific) sequence for the previous node and then play the response sequence to start listening. Looking through the base functionality of ShowSubtitle, I'm not seeing where this is done.

[Tony Li]

If I understand you correctly, you want to go back to the conversation state that calls ShowSubtitle().

In your dialogue UI, this should just cache the subtitle again and then proceed to ShowResponses() again.

To go back to the subtitle's conversation state, add this to your dialogue UI script:

Code: Select all

ConversationState currentConversationState;

public void ShowSubtitle(Subtitle subtitle)
{
    // MAYBE UNNECESSARY: currentSubTitle = subtitle; //Caches the current subtitle so we can return
    currentConversationState = DialogueManager.currentConversationState;
}

Then to go back to that state:

Code: Select all

DialogueManager.conversationController.GotoState(currentConversationState);