Entrytag VO lines with FMOD

[fkkcloud]

Hello,

I have 2 questions regarding the entry tag for dialogue lines for VO.

1. I am using FMOD for all of your sounds. Do I need to do any additional setup to make the entrytag work as expected in this video?

2. If I use

Code: Select all

AudioWait(entrytag);

as a default sequence, and the entrytag does not exist, what exactly happens? and is this an ideal thing to do if I have some nodes that are there for non-dialogue actions? (see the attached img)

화면 캡처 2022-02-15 173913.png (70.28 KiB) Viewed 1599 times

Thank you

[Tony Li]

Hi,

AudioWait() does not use FMOD.

If you're using FMOD, you can make an audio event for each entrytag value. Then you can use a custom FMOD sequencer command such as the one in this forum post.

In the custom sequencer command, you may want to check if the event exists. Then only call FMOD's RuntimeManager.PlayOneShot() if the event exists.

[fkkcloud]

What if I still want to use the benefit of keyword "entrytag"?

There are going to be 2500 lines recorded and I hope not to manually bind them 1 by 1.

Say, I created a custom fmod sequencer command called "FMODWait(x)".

If I want to use it as "AudioWait(entrytag)", I guess I will have to use the entrytag as a event's pathname like:

event:/VO/actor_conversationID_lineID

Code: Select all

"event:/" + entrytag

Edit:
I found this link : https://alessandrofama.com/tutorials/fm ... mer-sounds
With this setup, would AudioWait(entrytag); actually work?

[Tony Li]

Hi,

You can still use "entrytag". Entrytags are not specific to AudioWait(). They just represent a unique string for each dialogue entry. You can use them with any sequencer command.

Here's a rough sketch of an FMODWait() sequencer command. I'm just typing it in here; there may be typos. I'm also omitting using statements for brevity.

Code: Select all

public class SequencerCommandFMODWait : SequencerCommand
{
    EventInstance eventInstance;
    
    IEnumerator Start()
    {
        string event = GetParameter(0);
        if (DialogueDebug.logInfo) Debug.Log($"Dialogue System: Sequencer: FMODWait({event})");
        eventInstance = CreateInstance(PathToGUID(event));
        // Don't use 3D position in this example: instance.set3DAttributes(RuntimeUtils.To3DAttributes(position));
        if (eventInstance != null)
        {
            while (IsPlaying())
            {
                yield return null;
            }
        }
        Stop();        
    }
    
    bool IsPlaying()
    {
        if (eventInstance == null) return false;
	FMOD.Studio.PLAYBACK_STATE state;   
	eventInstance.getPlaybackState(out state);
	return state != FMOD.Studio.PLAYBACK_STATE.STOPPED;
    }
    
    void OnDestroy()
    {
        if (eventInstance != null) 
        {
            if (IsEventPlaying()) eventInstance.stop();
            eventInstance.release();
        }
    }
}

[Mihail]

it worked for me. I'm not sure if this is the best code, because I'm not a programmer.

Code: Select all

        EventInstance eventInstance;
        PLAYBACK_STATE state;
        EventDescription eventCheck;

        public IEnumerator Start()
        {
            string FMODEvent = "event:/"+GetParameter(0); //the path to the event in FMOD
            if (DialogueDebug.logInfo) Debug.Log($"Dialogue System: Sequencer: FMODWait({FMODEvent})");
          
            RuntimeManager.StudioSystem.getEvent(FMODEvent, out eventCheck); //checking if an event exists
            if (eventCheck.isValid()) 
            {
                Debug.Log("Event found");
              
            }
            else
            {
                Stop();
                Debug.Log("Event not found");
            }
           
            eventInstance = RuntimeManager.CreateInstance(FMODEvent);
            eventInstance.start();

            if (state != PLAYBACK_STATE.STOPPED)
            {
                while (IsPlaying())
                {
                    yield return null;
                }
            }
            Stop();
        }

        bool IsPlaying()
        {
            if (state == PLAYBACK_STATE.STOPPED) return false;
            eventInstance.getPlaybackState(out state);
            return state != PLAYBACK_STATE.STOPPED;
        }

        public void Awake()
        {
      
        }

        public void Update()
        {
           
        }

        public void OnDestroy()
        {
           
            if (state != PLAYBACK_STATE.STOPPED)
            {
                if (IsPlaying()) eventInstance.stop(STOP_MODE.IMMEDIATE);
                eventInstance.release();
            }
            Debug.Log("Fmod voice event destroyed");
        }

[Tony Li]

That looks fine to me. You remembered to include an OnDestroy() method, which is important in case the player skips ahead early before the VO has finished. You can remove the empty Awake() and Update() methods.

[fkkcloud]

Thanks for sharing an example.

I think I will be using "Table" in FMOD which fits better to my production.
It is for quite large list of VO lines (which in my case it will exceed around 10500 lines).

https://fmod.com/docs/2.02/unity/exampl ... ounds.html

Once I get onto it and implement this FMOD Table with DialogueSystem, I will share my code here as well

[Tony Li]

Full SequencerCommandFMODWait.cs script:

Code: Select all

using UnityEngine;
using FMODUnity;
using FMOD.Studio;
using PixelCrushers.DialogueSystem;

namespace PixelCrushers.DialogueSystem.SequencerCommands
{
    public class SequencerCommandFMODWait : SequencerCommand
    {
        EventInstance eventInstance;
        PLAYBACK_STATE state;
        EventDescription eventCheck;

        public IEnumerator Start()
        {
            string FMODEvent = "event:/"+GetParameter(0); //the path to the event in FMOD
            if (DialogueDebug.logInfo) Debug.Log($"Dialogue System: Sequencer: FMODWait({FMODEvent})");
          
            RuntimeManager.StudioSystem.getEvent(FMODEvent, out eventCheck); //checking if an event exists
            if (!eventCheck.isValid()) 
            {
                Debug.LogWarning("Event not found");
                Stop();
                yield break;
            }
           
            eventInstance = RuntimeManager.CreateInstance(FMODEvent);
            eventInstance.start();

            if (state != PLAYBACK_STATE.STOPPED)
            {
                while (IsPlaying())
                {
                    yield return null;
                }
            }
            Stop();
        }

        bool IsPlaying()
        {
            if (state == PLAYBACK_STATE.STOPPED) return false;
            eventInstance.getPlaybackState(out state);
            return state != PLAYBACK_STATE.STOPPED;
        }

        public void OnDestroy()
        {
           
            if (state != PLAYBACK_STATE.STOPPED)
            {
                if (IsPlaying()) eventInstance.stop(STOP_MODE.IMMEDIATE);
                eventInstance.release();
            }
            //Debug.Log("Fmod voice event destroyed");
        }
    }
}

Example:

• Dialogue Text: "Hello world."
• Sequence: FMODWait(hello_world)

(where "hello_world" is an FMOD event)

[fkkcloud]

Here is revised script to use FMOD's Programmer Sound.

Code: Select all

public class SequencerCommandFMODWait : SequencerCommand
    {
        EventInstance eventInstance;
        PLAYBACK_STATE state;
        EventDescription eventCheck;
        FMOD.Studio.EVENT_CALLBACK dialogueCallback;

        public IEnumerator Start()
        {
            dialogueCallback = new FMOD.Studio.EVENT_CALLBACK(DialogueEventCallback);

            PlayDialogue(key); // key will be the entryTag e.g. [CharacterName]_[SceneID]_[EntryID] e.g  "Tifa_6_22" or do "entrytag". -> PlayDialogue(GetParamgeter(0)); and in the sequence, you would put just --> FMODWait(entrytag);
            
            if (state != PLAYBACK_STATE.STOPPED)
            {
                while (IsPlaying())
                {
                    yield return null;
                }
            }
            Stop();
        }

        bool IsPlaying()
        {
            if (state == PLAYBACK_STATE.STOPPED) return false;
            eventInstance.getPlaybackState(out state);
            return state != PLAYBACK_STATE.STOPPED;
        }

        public void OnDestroy()
        {
            if (state != PLAYBACK_STATE.STOPPED)
            {
                if (IsPlaying()) eventInstance.stop(STOP_MODE.IMMEDIATE);
                eventInstance.release();
            }
        }
        
        void PlayDialogue(string key)
        {
            // TODO: check localization and set the bank to EN JP or KR
            eventInstance = FMODUnity.RuntimeManager.CreateInstance("event:/Dialogue/Dialogue_EN"); // this event will be the part where I will be swapping only for localization. So, "event:/Dialogue/Dialogue_EN" contains, say 4000 english lines, "event:/Dialogue/Dialogue_JA" will contain 4000 japanese lines.

            // Pin the key string in memory and pass a pointer through the user data
            GCHandle stringHandle = GCHandle.Alloc(key);
            eventInstance.setUserData(GCHandle.ToIntPtr(stringHandle));

            eventInstance.setCallback(dialogueCallback);
            eventInstance.start();
        }
        
        [AOT.MonoPInvokeCallback(typeof(FMOD.Studio.EVENT_CALLBACK))]
        static FMOD.RESULT DialogueEventCallback(FMOD.Studio.EVENT_CALLBACK_TYPE type, IntPtr instancePtr, IntPtr parameterPtr)
        {
            FMOD.Studio.EventInstance instance = new FMOD.Studio.EventInstance(instancePtr);

            // Retrieve the user data
            IntPtr stringPtr;
            instance.getUserData(out stringPtr);

            // Get the string object
            GCHandle stringHandle = GCHandle.FromIntPtr(stringPtr);
            String key = stringHandle.Target as String;

            switch (type)
            {
                case FMOD.Studio.EVENT_CALLBACK_TYPE.CREATE_PROGRAMMER_SOUND:
                {
                    FMOD.MODE soundMode = FMOD.MODE.LOOP_NORMAL | FMOD.MODE.CREATECOMPRESSEDSAMPLE | FMOD.MODE.NONBLOCKING;
                    var parameter = (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));

                    if (key.Contains("."))
                    {
                        FMOD.Sound dialogueSound;
                        var soundResult = FMODUnity.RuntimeManager.CoreSystem.createSound(Application.streamingAssetsPath + "/" + key, soundMode, out dialogueSound);
                        if (soundResult == FMOD.RESULT.OK)
                        {
                            parameter.sound = dialogueSound.handle;
                            parameter.subsoundIndex = -1;
                            Marshal.StructureToPtr(parameter, parameterPtr, false);
                        }
                    }
                    else
                    {
                        FMOD.Studio.SOUND_INFO dialogueSoundInfo;
                        var keyResult = FMODUnity.RuntimeManager.StudioSystem.getSoundInfo(key, out dialogueSoundInfo);
                        if (keyResult != FMOD.RESULT.OK)
                        {
                            break;
                        }
                        FMOD.Sound dialogueSound;
                        var soundResult = FMODUnity.RuntimeManager.CoreSystem.createSound(dialogueSoundInfo.name_or_data, soundMode | dialogueSoundInfo.mode, ref dialogueSoundInfo.exinfo, out dialogueSound);
                        if (soundResult == FMOD.RESULT.OK)
                        {
                            parameter.sound = dialogueSound.handle;
                            parameter.subsoundIndex = dialogueSoundInfo.subsoundindex;
                            Marshal.StructureToPtr(parameter, parameterPtr, false);
                        }
                    }
                    break;
                }
                case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROY_PROGRAMMER_SOUND:
                {
                    var parameter = (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));
                    var sound = new FMOD.Sound(parameter.sound);
                    sound.release();

                    break;
                }
                case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROYED:
                {
                    // Now the event has been destroyed, unpin the string memory so it can be garbage collected
                    stringHandle.Free();

                    break;
                }
            }
            return FMOD.RESULT.OK;
        }

    }

Let me know if this looks weird in terms of Dialogue System wise. (or even FMOD wise - I am new to this)
At least it seems to work as intended so far.

Mixed with Tony's with FMOD tutorial's code here: https://www.fmod.com/docs/2.02/unity/ex ... ounds.html

[fkkcloud]

Above bring up 1 more question- is it possible to know when the voice sound ended (if player didnt make any interation yet)?
So I can sync mouth animation keep rolling while there is sound and stop when there isn't.