Pixel Crushers Forum

Hey there everyone! Thanks for all of your help so far. I've mentioned before that I'm working on a game that you control by using a microphone with very little input beyond speaking into it. Because of this, I've run into some interesting roadblocks that I was hoping I could get some help with.

Below is a small chunk of the introductory conversation the player has with the character. What I would like to happen is that when the scene starts, the dialogue begins playing in the form of sequential barks over the character's head, up until the third node, where the character asks "Can you hear me?". The player will be prompted to give a verbal answer into the microphone and the next bit of dialogue will change depending on whether the player gives an affirmative answer, a negative answer, or an answer that is neither affirmative or negative. (I've already handled parsing the text to look for words like "yes" and "no" and translating that result into a variable that can be checked for, so that bit is all taken care of!) I have two questions regarding this. First, is there a way I can make it so that each node in this conversation plays in sequence? So, after the first bubble that says "...Huh?" fades away, a second one with "Hey! I'm over here!" fades in sequentially for the whole thing. I currently have a trigger that makes the caracter bark the first node on start, but it stops after the first node.

Secondly (and more importantly!) Is there a way I can stop the sequence at the third node to allow the player to record their response? If it's any help, in order to record voice, the player needs to press and hold the spacebar down to activate the recording and when it's let go, the recording is processed through an API to return the result. I'm maybe thinking there's a way to pause the sequence until the player lets go of the space bar, signaling that they have given a response.

Sorry for all the questions! The design behind this game has been a huge hit so far, but I'd be lying if I said this was a simple process!

Thank you so much for your time as always!

Hi,

Run a regular conversation, but use overhead bubble subtitle panels. (See: How To: Show Overhead Conversation Bubble Text)

Set up the "waiting for input" parts as response menus (i.e., player-assigned blue nodes), as in this example from the Recogmissimo example scene:


That example, and other integrations for other speech recognition systems, are here: [HOWTO] How To Use Wit.ai, Windows Speech Recognition, Google Cloud Voice Recognition, or Recognissimo in Conversations

Hey there Tony! Thank you for the speedy reply as always! Been busy this week so I'm only just now getting around to working on this again haha.

I got the conversation playing over the character's head, which is great! I do have just a few questions about the voice integration I was hoping you could maybe help me out with.

So I set up the conversation as you had shown, with the available options as their own response menus. When I do this, the response menu actually does show up and the player is able to click on the option they want. This probably sounds crazy and backward, but this isn't quite what I'm looking for. I now realize I probably could have been more thorough when explaining my goal here, I apologize!
So, after the third node "Can you hear me?" fades away, a little microphone icon will appear on screen, prompting the player to record a response into the mic. (I'll be able to code in the prompt myself!). What I was hoping to do was instead of having the response menu pop up with pre selected options to click on, the player would verbally say "yes" or "no" or "yep" or something into the mic to advance the conversation.

If it's any help, I already have a variable within the dialogue system that saves the player's input and one that returns a value depending on whether the player's answer is affirmative (yes, yep, yeah, ETC = 1) negative (no, nah, I don't ETC = 2) or other (Literally anything else = 3). Both of these variables work as intended, I'm just curious as to how we can pause at the end of the third node to wait for the player's input before advancing based on what the Y/N/E variable returns as.
My apologies if I misinterpreted any of your instructions and just explained something you already understood haha. I tried to look at the Recogmissimo example, but it kept throwing me errors when I tried to run it so I couldn't quite tell whether or not it was the solution I was looking for. Thanks again for all your help so far!

Hi,

The easiest way -- which doesn't require any scripting -- is to keep your response menu buttons invisible by putting them in a subpanel with a Canvas Group with Alpha set to 0, and add a microphone icon to the response menu panel itself:


So the response menu buttons will technically be there, but they won't be visible or clickable.

Hey again! Thanks for the canvas tip, Thanks to that I was able to get the mic graphic to show up, which is Great!

The last issue I'm facing is that even though the dialogue options are invisible, when the player gives a voice command the conversation does not seem to progress after the dialogue is processed as intended. While the voice command is processed as normal and the variables update properly, the game continues to wait for the player to click the invisible button.

This is super close to what I'm looking for, but not quite there yet! Ideally, I'd like the conversation to continue the moment the player's verbal command is processed and the proper variables are updated instead of the player having to click on the invisible button. Is there a way we can make it so that there's no need for nay sort of input beyond recoding the voice command? Or is that something I'd have to do in code? Figured I'd see if you had any pointers on how to do this in the dialogue system, if that's something we can do. Let me know if a video or more pictures would be helpful, i'd be happy to provide whatever you need to come to a solution!

As always, thank you so much for your time, and I hope to hear from you soon!

Hi,

What speech recognition service/plugin are you using? And have you replaced your dialogue UI's StandardUIMenuPanel component with a subclass that uses this service/plugin? (See any of the examples in the link in my first reply.)

Hey again, I'm using the HuggingFace API for my speech to text as seen here:

https://huggingface.co/blog/unity-asr

I've been using it outside of the dialogue system to let the player give commands. the command is ran through a set of functions that check for objects in the room that match certain keywords that correspond to different objects within the scene and makes the character move over to said object (I.E. the command "Go to the painting" is parsed into a string and the string is checked for an object with the attached keyword, in this case "painting"). As far as it's used in my game, it's a fancy way to accept text input.

I haven't replaced or messed with anything in regards to the dialogue system, and thus far the two have existed separately aside from the two LUA variables that store the parsed string from a given voice command and the "Yes/No/Else" variable that tracks whether or not the player has given an command with an affirmative, a negative, or something else entirely which I have already implemented.

Having taken a closer look at the examples, I'm now seeing that this is probably something I'll have to do in code instead of the dialogue system itself. Something like the code below where I'll have to simulate a click on the response button through with Instance.OnClick. The API specifics are different, but the general idea is the same, I'd just have to re-write it so that it works with hugging face. When I get this taken care of, I'd be more than happy to share my results with you and maybe add it to the big thread? Maybe someone out there would find it useful down the line haha.

public static void getIntentFromWit3d(string Wit3Dintent) {
if (!string.IsNullOrEmpty(Wit3Dintent)) {
foreach (var response in Instance.responses) {
if (string.Equals(Wit3Dintent, response.formattedText.text)) {
// We have a match, select the choice:
Instance.OnClick(response); // Simulate a click on a response button.
}
}
}
}
}

Look's like its back to the lab for me! Thanks again for all your help!

Sounds good!

You really don't need to do much. The bare minimum would be to pass the corresponding Response object to DialogueManager.standardDialogueUI.OnClick(Response). The example StandardUIMenuPanel subclasses above just wrap it up in a comprehensive way.

hey again, sorry to bother ya!

Wanted to just clarify the use of the DialogueManager.standardDialogueUI.OnClick(Response) line you shared with me the other day. I tried to implement it as shown below, but I might be doing it wrong since When i give the command, the response menu with the mic does disappear, but nothing else happens. I feel like I'm missing something silly here again haha. For reference, "input" with the lowercase "i" is the name of the node that acts as the player input (The invisible button), so I'm trying to invoke the onclick of that, but I might be going about it wrong haha.
Thanks, as always!

Hi,

Please pass a Response object to DialogueManager.standardDialogueUI.OnClick(Response), such as: