Saving additional data from dialog entry field

[mrcsbmr]

Hi,

in early development we added a field to each dialogue entry that stores an int value which is very important for us. For examination texts, we have loops and sequences and this value dictates which dialogue entry of a conversation has to be displayed. We did it this way because we didn't want to set up variables for aaaalll of these texts (it is a lot) in Articy.

Now we discovered that the fields are not being saved (makes sense).

Is there an easy way to save this data without getting a huge overload?
Could we for example somehow write this int data into the Sim Status field which we aren't using in our game, but can be saved by design?

(edit) oh yes, and we're using AdventureCreator, to make it even more complicated.

[mrcsbmr]

Here's an update. After realizing the Sim Status field is being written to by DialogueSystem and not wanting to hack deep into the system, I started writing a RememberScript for AdventureCreator that is supposed to save the current state of the so called "Current Value" fields in all my dialogue entries.

In the save method of that remember script, I coded:

Code: Select all

foreach (var conversation in DialogueManager.MasterDatabase.conversations)
            {
                var conversationTable = Lua.Run("return Conversation[" + conversation.id + "]").AsTable;
                var dialogTable = conversationTable.luaTable.GetValue("Dialog") as Language.Lua.LuaTable;

                for (int i = 0; i < conversation.dialogueEntries.Count; i++)
                {
                    var entryID = conversation.dialogueEntries[i].id;
                    var dialogFields = dialogTable.GetValue(entryID) as Language.Lua.LuaTable; // ERROR is always null :/
                    if (dialogFields != null)
                    {
                        var articyID = dialogFields.GetValue("articyID");
                        var currentValue = dialogFields.GetValue("CurrentValue");

                        Debug.Log(entryID + " : " + articyID);

                        SavedData.SavedFields.Add(articyID.ToString(), currentValue .ToString());
                    }
                }
            }
            

I copied and modified some code of the PersistentDataManager. However, I'm not good with the Lua stuff and am not sure what's happening there. The idea is to save a Dictionary that maps all "Current Value"s to their Articy IDs.

The code is taken from PersistentDataManager.cs, starting line 427, where DialogueSystem saves the Sim Status field.

Problem 1: dialogFields always stays null and I cannot save anything.

Problem 2: I don't know yet how to map the loaded Dictionary back to the Dialogue Database. I'm easily able to save and load that Dictionary object, but how do I get the data back into the MasterDatabase?

Thanks for the support!

[Tony Li]

Hi,

When you start a new game, are the values of this int field all the same (e.g., zero)? That'll make it much easier. But if that's not the case, and each field could start with a different value, we can still come up with a solution.

[Tony Li]

Here's an example:

SaveExtraDialogField_2017-12-23.unitypackage

In the assigned database, in conversation 1, I set the CurrentValue values for entries 1, 2, and 3. It doesn't show anything onscreen. Press Escape to open the menu to save and load. It uses a custom script, which I'll also include below. It hooks into PersistentDataManager, allowing it to append extra save information to the saved game string. In this case, it appends Lua code that, when reloaded, will set up the conversations' Dialog tables with the saved values of CurrentValue.

SaveExtraField.cs

Code: Select all

using UnityEngine;
using System.Collections;
using System.Text;
using PixelCrushers.DialogueSystem;

public class SaveExtraField : MonoBehaviour
{

    private void Awake()
    {
        // Tell PersistentDataManager to add our custom save data:
        PersistentDataManager.GetCustomSaveData = GetMyCustomSaveData;
    }

    private IEnumerator Start()
    {
        // Let Dialogue Manager load database first:
        yield return new WaitForEndOfFrame();

        // Then add in the custom Dialog values:
        foreach (var conversation in DialogueManager.MasterDatabase.conversations)
        {
            var sb = new StringBuilder();
            sb.AppendFormat("Conversation[{0}].Dialog = {{", conversation.id);
            foreach (var entry in conversation.dialogueEntries)
            {
                var articyId = Field.LookupValue(entry.fields, "ArticyId");
                var currentValue = Field.LookupInt(entry.fields, "CurrentValue");
                sb.AppendFormat("[{0}] = {{ ArticyId = '{1}', CurrentValue = {2} }}, ", entry.id, articyId, currentValue);
            }
            sb.Append("}; ");
            Lua.Run(sb.ToString(), true);
        }
    }

    private string GetMyCustomSaveData()
    {
        var sb = new StringBuilder();
        foreach (var conversation in DialogueManager.MasterDatabase.conversations)
        {
            sb.AppendFormat("Conversation[{0}].Dialog = {{ ", conversation.id);

            var dialogTable = Lua.Run("return Conversation[" + conversation.id + "].Dialog").AsTable;
            foreach (var key in dialogTable.Keys)
            {
                var dialogValueTable = dialogTable[key] as LuaTableWrapper;
                var currentValueNumber = (System.Single)dialogValueTable["CurrentValue"];
                var currentValue = (int)currentValueNumber;
                sb.AppendFormat(" [{0}] = {{ CurrentValue = {1} }}, ", key, currentValue);                
            }

            sb.Append("}; ");
        }
        return sb.ToString();
    }

}

[mrcsbmr]

Thank you so much, I will look into it!
And yes, in a new game, all values are always zero.

[mrcsbmr]

I tried it, but unfortunately it isn't working in our project. I tried your original script, which causes some errors. I then did the following:

Firstly, I commented out the Articy ID part, because that doesn't really need to be saved. I just used it as a key previously. Also, the name of the varibable is not ArticyId but 'Articy Id' which caused issues in the Lua script. But since it's not needed:

Code: Select all

    private IEnumerator Start()
    {
        // Let Dialogue Manager load database first:
        yield return new WaitForEndOfFrame();

        // Then add in the custom Dialog values:
        foreach (var conversation in DialogueManager.MasterDatabase.conversations)
        {
            var sb = new StringBuilder();
            sb.AppendFormat("Conversation[{0}].Dialog = {{", conversation.id);
            foreach (var entry in conversation.dialogueEntries)
            {
                //var articyId = Field.LookupValue(entry.fields, "ArticyId");
                var currentValue = Field.LookupInt(entry.fields, "CurrentValue");
                //sb.AppendFormat("[{0}] = {{ ArticyId = '{1}', CurrentValue = {2} }}, ", entry.id, articyId, currentValue);
                sb.AppendFormat("[{0}] = {{ CurrentValue = '{1}' }}, ", entry.id, currentValue);
            }
            sb.Append("}; ");
            Lua.Run(sb.ToString(), true);
        }
    }

Am I right that the whole part in Start() is only needed when the values in *new* games are not all 0? If so, we wouldn't even need this whole method at all.


In GetMyCustomSaveData, I tried both the code you wrote for this and when it wasn't working properly in the project, also the stolen code from the Sim Status stuff in the PersistantData manager which looks like it's doing the same thing in a slightly different way.
The problem with the original code from this forum post was that I got an error saying it couldn't cast the variable in this line:
var currentValueNumber = (System.Single)dialogValueTable["CurrentValue"];
I then changed that into casting that into a string and then just use that string in the Lua script (see below).

When that didn't work out, I tried the stolen code from the PersistantDataManager you use for storing the Sim Status. It's commented out in the code snippet below.

Code: Select all

    private string GetMyCustomSaveData()
    {
        
        var sb = new StringBuilder();
        foreach (var conversation in DialogueManager.MasterDatabase.conversations)
        {
            //Debug.Log("<color=red>Conv: " + conversation.id + "</color>");

            sb.AppendFormat("Conversation[{0}].Dialog = {{ ", conversation.id);
            
            // From forum:
            var dialogTable = Lua.Run("return Conversation[" + conversation.id + "].Dialog").AsTable;
            foreach (var key in dialogTable.Keys)
            {
                var dialogValueTable = dialogTable[key] as LuaTableWrapper;
                //var currentValueNumber = (System.Single)dialogValueTable["CurrentValue"];
                var currentValueNumber = dialogValueTable["CurrentValue"].ToString();
                //var currentValue = (int)currentValueNumber;
                var currentValue = currentValueNumber;
                sb.AppendFormat(" [{0}] = {{ CurrentValue = {1} }}, ", key, currentValue);                
            }
            
            /*
            // From sim status code:
            var conversationTable = Lua.Run("return Conversation[" + conversation.id + "]").AsTable;
            var dialogTable = conversationTable.luaTable.GetValue("Dialog") as Language.Lua.LuaTable;
            for (int i = 0; i < conversation.dialogueEntries.Count; i++)
            {
                var entryID = conversation.dialogueEntries[i].id;
                var dialogFields = dialogTable.GetValue(entryID) as Language.Lua.LuaTable;
                if (dialogFields != null)
                {
                    var currentValue = dialogFields.GetValue("CurrentValue");
                    sb.AppendFormat("[{0}]={{CurrentValue=\"{1}\"}},", new System.Object[] { entryID, currentValue });
                }
            }*/
            
            sb.Append("}; ");
        }

        //Debug.Log("<color=red>" + sb.ToString() + "</color>");
        return sb.ToString();
    }

Both variations didn't really work.
I now suspect AdventureCreator to be the problem. The string which is supposed to be stored in an AC variable becomes SO very long. We have 947 conversations and 591 variables in the database. In these 947 conversations, we have 7435 lines which all have a CurrentValue. It might just be too much / the string getting too long?

[mrcsbmr]

I just realized that our CurrentValue field only needs to be saved/loaded when at the same time, a field called "IsSequence" or a field called "IsLoop" in the same dialogue entry is set to True. Reducing saving/loading to these would probably help performance and reduce the string length significantly. I tried adding it, but didn't get behind it yet.

[Tony Li]

Because of the amount of your data, I'm going to put together an example that's a tiny bit more complicated but will perform much better. I'll post back here soon.

[Tony Li]

Here's an updated example:

SaveExtraDialogField_2017-12-28.unitypackage

It uses the script below, which saves the CurrentValue fields in a much more compact state that also runs faster.

In the example, I set IsSequence or IsLoop true in two entries of the "Private Hart" conversation and set their CurrentValues to 42 and 7. I also set IsSequence true in one entry of the "Terminal" conversation and set its CurrentValue to 99. The script generates this saved game data for it:

Code: Select all

RestoreCurrentValues("0;1;42;2;7;-1;3;2;99;-1;")

The script has a Debug checkbox that will print out what it's doing, in case that's helpful.

SaveExtraField.cs

Code: Select all

using UnityEngine;
using System.Collections;
using System.Text;
using PixelCrushers.DialogueSystem;

public class SaveExtraField : MonoBehaviour
{

    public bool debug;

    private IEnumerator Start()
    {
        // Let Dialogue Manager load database first:
        yield return new WaitForEndOfFrame();

        // Then add in the custom CurrentValue fields to the Lua environment:
        AddCurrentValueFieldsToLua();

        // Register a Lua function to parse a string that compactly
        // records CurrentValue values:
        Lua.RegisterFunction("RestoreCurrentValues", this, SymbolExtensions.GetMethodInfo(() => RestoreCurrentValues(string.Empty)));

        // Tell PersistentDataManager to add our custom save data,
        // which will be a call to our custom Lua function RestoreCurrentValues,
        // passing in a compact record of CurrentValue values.
        PersistentDataManager.GetCustomSaveData = GetMyCustomSaveData;
    }

    private void AddCurrentValueFieldsToLua()
    {
        // Add CurrentValue fields to the runtime Lua environment in the form
        // Conversation[#].Dialog[#].CurrentValue.

        foreach (var conversation in DialogueManager.MasterDatabase.conversations)
        {
            var sb = new StringBuilder();
            sb.AppendFormat("Conversation[{0}].Dialog = {{", conversation.id);
            foreach (var dialogueEntry in conversation.dialogueEntries)
            {
                if (Field.LookupBool(dialogueEntry.fields, "IsSequence") || Field.LookupBool(dialogueEntry.fields, "IsLoop"))
                {
                    var currentValue = Field.LookupInt(dialogueEntry.fields, "CurrentValue");
                    sb.AppendFormat("[{0}] = {{ CurrentValue = {1} }}, ", dialogueEntry.id, currentValue);
                }
            }
            sb.Append("}; ");
            Lua.Run(sb.ToString(), debug);
        }
    }

    private string GetMyCustomSaveData()
    {
        // Return a string that calls our custom Lua function, passing it the data
        // necessary to set the CurrentValue fields of specific dialogue entries.

        var sb = new StringBuilder();
        sb.Append("RestoreCurrentValues(\"");
        var conversations = DialogueManager.MasterDatabase.conversations;
        for (int c = 0; c < conversations.Count; c++)
        {
            var conversation = conversations[c];
            var first = true; // We haven't found a dialogue entry with CurrentValue yet.
            for (int d = 0; d < conversation.dialogueEntries.Count; d++)
            {
                var dialogueEntry = conversation.dialogueEntries[d];
                if (Field.LookupBool(dialogueEntry.fields, "IsSequence") || Field.LookupBool(dialogueEntry.fields, "IsLoop"))
                {
                    if (first) 
                    {
                        // If we found the first dialogue entry with CurrentValue in this conversation, record the conversation index:
                        sb.Append(c);
                        sb.Append(';');
                        first = false;
                    }
                    // Then record the dialogue entry index and its CurrentValue (assumed to be int):
                    sb.AppendFormat("{0};{1};", d, Field.LookupInt(dialogueEntry.fields, "CurrentValue"));
                }
            }
            if (!first)
            {
                // If we found a dialogue entry with CurrentValue, append -1 to signify the end of this conversation:
                sb.Append("-1;");
            }
        }
        sb.Append("\")");
        if (debug) Debug.Log("GetMyCustomSaveData: " + sb);
        return sb.ToString();
    }

    private void RestoreCurrentValues(string data)
    {
        if (string.IsNullOrEmpty(data)) return;
        var conversations = DialogueManager.MasterDatabase.conversations;
        string[] s = data.Split(';');
        var i = 0;
        while (i < s.Length)
        {
            // Get the conversation index in the dialogue database:
            var c = Tools.StringToInt(s[i++]);
            if (!(0 <= c && c < conversations.Count)) return; // Sanity check.
            var conversation = conversations[c];

            // Start a Lua command that will set the conversation's CurrentValue fields:
            var sb = new StringBuilder();
            sb.AppendFormat("Conversation[{0}].Dialog = {{", conversation.id);

            // Process all recorded dialogue entries' CurrentValues:
            var done = false;
            while (!done && i < s.Length)
            {
                var d = Tools.StringToInt(s[i++]);
                if (d == -1) // Our special signifier that this conversation's list of values is done.
                {
                    done = true;
                }
                else
                {
                    var dialogueEntry = conversation.dialogueEntries[d];
                    var currentValue = Tools.StringToInt(s[i++]);
                    sb.AppendFormat("[{0}] = {{ CurrentValue = {1} }}, ", dialogueEntry.id, currentValue);
                }
            }

            // Run the Lua command to set the conversation's CurrentValue fields:
            sb.Append("}; ");
            Lua.Run(sb.ToString(), debug);
        }
    }

}

[mrcsbmr]

Thank you so much! After a few adjustments to our own scripts (OK, that took a good while) and understanding the Lua functions better, it is finally working for our project as well. I'll attach the code we have now working here just for future references, if anybody has a problem with this one day and looks it up in the forum's archieve.

I'm not sure what's now different to Tony's previous version of it anymore. It's definitely that we need to store the IsLoop and IsSequence data in the Lua environment too and that we need to make all changes using Lua commands and never access the database with the direct functions from DialogueSystem because that causes issues.
So, all in all this is what is working for us now:

Code: Select all

using UnityEngine;
using System.Collections;
using System.Text;
using PixelCrushers.DialogueSystem;

public class SaveExtraField : MonoBehaviour
{

    public bool debug;

    private IEnumerator Start()
    {
        // Let Dialogue Manager load database first:
        yield return new WaitForEndOfFrame();

        // Then add in the custom CurrentValue fields to the Lua environment:
        AddCurrentValueFieldsToLua();

        // Register a Lua function to parse a string that compactly
        // records CurrentValue values:
        Lua.RegisterFunction("RestoreCurrentValues", this, SymbolExtensions.GetMethodInfo(() => RestoreCurrentValues(string.Empty)));

        // Tell PersistentDataManager to add our custom save data,
        // which will be a call to our custom Lua function RestoreCurrentValues,
        // passing in a compact record of CurrentValue values.
        PersistentDataManager.GetCustomSaveData = GetMyCustomSaveData;
    }

    private void AddCurrentValueFieldsToLua()
    {
        // Add CurrentValue fields to the runtime Lua environment in the form
        // Conversation[#].Dialog[#].CurrentValue.

        foreach (var conversation in DialogueManager.MasterDatabase.conversations)
        {
            var sb = new StringBuilder();
            sb.AppendFormat("Conversation[{0}].Dialog = {{", conversation.id);
            foreach (var dialogueEntry in conversation.dialogueEntries)
            {
                if (Field.LookupBool(dialogueEntry.fields, "IsSequence") || Field.LookupBool(dialogueEntry.fields, "IsLoop"))
                {
                    var currentValue = Field.LookupInt(dialogueEntry.fields, "CurrentValue");
                    bool isLoop = Field.LookupBool(dialogueEntry.fields, "IsLoop");
                    bool isSequence = Field.LookupBool(dialogueEntry.fields, "IsSequence");
                    sb.AppendFormat("[{0}] = {{ CurrentValue = {1} ,  IsLoop = '{2}' ,  IsSequence = '{3}' }}, ", dialogueEntry.id, currentValue, isLoop, isSequence);
                }
            }
            sb.Append("}; ");
            Lua.Run(sb.ToString(), debug);
        }
    }

    private string GetMyCustomSaveData()
    {
        // Return a string that calls our custom Lua function, passing it the data
        // necessary to set the CurrentValue fields of specific dialogue entries.

        var sb = new StringBuilder();
        sb.Append("RestoreCurrentValues(\"");
        var conversations = DialogueManager.MasterDatabase.conversations;
        for (int c = 0; c < conversations.Count; c++)
        {
            var conversation = conversations[c];
            var first = true; // We haven't found a dialogue entry with CurrentValue yet.
            for (int d = 0; d < conversation.dialogueEntries.Count; d++)
            {
                var dialogueEntry = conversation.dialogueEntries[d];
                if (Field.LookupBool(dialogueEntry.fields, "IsSequence") || Field.LookupBool(dialogueEntry.fields, "IsLoop"))
                {
                    if (first)
                    {
                        // If we found the first dialogue entry with CurrentValue in this conversation, record the conversation index:
                        sb.Append(c);
                        sb.Append(';');
                        first = false;
                    }
                    // Then record the dialogue entry index and its CurrentValue (assumed to be int):
                    sb.AppendFormat(d + ";" + Lua.Run("return Conversation[" + dialogueEntry.conversationID + "].Dialog[" + dialogueEntry.id + "].CurrentValue").AsString + ";");
                    //sb.AppendFormat("{0};{1};", d, Field.LookupInt(dialogueEntry.fields, "CurrentValue"));
                }
            }
            if (!first)
            {
                // If we found a dialogue entry with CurrentValue, append -1 to signify the end of this conversation:
                sb.Append("-1;");
            }
        }
        sb.Append("\")");
        if (debug) Debug.Log("GetMyCustomSaveData: " + sb);
        return sb.ToString();
    }

    private void RestoreCurrentValues(string data)
    {
        if (string.IsNullOrEmpty(data)) return;
        var conversations = DialogueManager.MasterDatabase.conversations;
        string[] s = data.Split(';');
        var i = 0;
        while (i < s.Length)
        {
            // Get the conversation index in the dialogue database:
            var c = Tools.StringToInt(s[i++]);
            if (!(0 <= c && c < conversations.Count)) return; // Sanity check.
            var conversation = conversations[c];
            
            // Start a Lua command that will set the conversation's CurrentValue fields:
            var sb = new StringBuilder();
            sb.AppendFormat("Conversation[{0}].Dialog = {{", conversation.id);

            // Process all recorded dialogue entries' CurrentValues:
            var done = false;
            while (!done && i < s.Length)
            {
                var d = Tools.StringToInt(s[i++]);
                if (d == -1) // Our special signifier that this conversation's list of values is done.
                {
                    done = true;
                }
                else
                {
                    var dialogueEntry = conversation.dialogueEntries[d];
                    var currentValue = Tools.StringToInt(s[i++]);
                    
                    sb.AppendFormat("[{0}] = {{ CurrentValue = {1},  IsLoop = '{2}' ,  IsSequence = '{3}'  }}, ", dialogueEntry.id, currentValue, Field.LookupBool(dialogueEntry.fields, "IsLoop"), Field.LookupBool(dialogueEntry.fields, "IsSequence"));
                }
            }

            // Run the Lua command to set the conversation's CurrentValue fields:
            sb.Append("}; ");
            Lua.Run(sb.ToString(), debug);
        }
    }

}

Thanks for the amazing support!