Provide proper way to migrate asset data changes for save data.

[creta5164]

Hello there.

I am a developer currently organizing a migration infrastructure for quest data stored in an older version in my organization.

The version of the asset I am currently using is 1.2.34, and I am experiencing the following issue.

• Setting 4 conditions on a set of conditions specified in a quest and creating a save
• Reducing the number of condition sets in the same quest to 2 and importing the save
• Throws 'IndexOutOfRange' exception on 'QuestStateSerializer.ReadConditionSetDataFromStream'

I solved the problem by first reading the saved value and then comparing it to the data in the static Quest Asset to handle the exception, but I came to the conclusion that this is not the only issue, as the serialized save affects the schema of the resource nature of the data, and it is hard to tell if there are other issues.

This is a maintenance nightmare for version control, so I searched the forums and found the following responses.
https://www.pixelcrushers.com/phpbb/vie ... 872#p28872

It would probably be more robust to remove the quest from your quest database and add a new quest with a new quest ID. Otherwise you might not know if the version in the player's journal is the old version or the new version of the quest. For example, if the old quest's ID is “bakePie”, you could replace it with a new quest with ID “bakePie-v2”.

If we still have the old version of the quest asset, it's not hard to map it, but I think it's more of a challenge to correctly map the data in a migration environment where we're dealing with low-level data, because we don't even know the schema layout when we only have serialized binary data.

So, at the very least, I would like to see an option to save this as JSON so that it can be easily migrated via external infrastructure, and additionally, I think there should be some guidance on versioning, even in the official documentation.

[Tony Li]

Hi,

For general save version handling, please see: How To: Handle Saved Game Versions

However, that by itself won't solve everything A few more notes:

1. Quest Machine saves handwritten quests (as opposed to generated quests) using the QuestStateSerializer class, in that compact binary format that's causing your headaches. When it saves each quest, it saves the current QuestStateSerializer.version value in the quest. This way if the format that QuestStateSerializer uses ever changes, it can handle older format. This still doesn't address your main concern, but I wanted to make you aware of it.

2. When Quest Machine saves generated quests, it serializes the entire quest to JSON format using the QuestProxy class.

In a pinch, you can make subclasses that override the QuestJournal/QuestGiver RecordData() and ApplyData() methods to save handwritten quests using QuestProxy, like JsonUtility.ToJson(new QuestProxy(quest)).

However, if you can wait until after GDC next week, I plan to add an option to save handwritten quests using JSON instead of a compact byte array. This would be in QM version 2.2.52, since 2.2.51 is coming out before GDC.

[creta5164]

Hi Tony, thanks to quick response.

Your answer seems like it could be helpful to some extent, but in my case I have the challenge of doing this reliably on top of software already deployed in production.

I can't rush the asset version, so I'm going to have to choose between 1 and 2, but

However, if you can wait until after GDC next week, I plan to add an option to save handwritten quests using JSON instead of a compact byte array. This would be in QM version 2.2.52, since 2.2.51 is coming out before GDC.

I was wondering if there is any chance that the feature you mentioned at the end will be backported to 1.2.

Also, if the asset is fundamentally having these issues with schema, then it seems like the enumerated data will have a significantly different schema than the stored data when reordered.

I wonder if there might be a stable feature coming in the future to address this, such as a migration script in EF Core that tracks up/down schemas.

In my opinion, I think this should be complemented with an explicit non-sequential ID, such as a GUID, instead of indexed data, so that we can have a migration-safe organization of the data.

Lastly, this is also a challenge because the directors/writters/producers who typically use Quest Machines to work with the data don't have the technical skills to note every change to the content.

In fact, when we've identified these issues while explained these, we've gotten responses like, “The instructions are vague on how to fix it, and it's dangerous.”

I think this is even more critical for non-technical users, so I hope you'll consider this carefully.

[Tony Li]

Hi,

Yes, as soon as I get back from GDC this week I'll be evaluating some options to retain backward compatibility while still introducing nontransient GUIDs.