Skyrim save issue + fix

[ITotalJustice]

Hi. I tried to restore my skyrim save from switch A onto Switch B. The restore was successful and Skyrim loaded the save just fine.

However, i was unable to create a new save in-game, instead it would result in a crash. Due to messing with save files when making ftpsrv, i immediately knew what was wrong, the data_size was incorrect.

I logged the save extra data from Switch A and got:
data_size: 176160768 journal_size: 25165824
and from Switch B:
data_size: 100203061 journal_size: 25165824

using fsExtendSaveDataFileSystem() to correct the data size to match Switch A fixed the issue, and now able to create saves in-game.

---

Does JKSV not output the save extra data when creating a backup? If not, then i propose it creates a file within the save folder with that data, ie .nx_save_data_extra.bin or .nx_save_data_meta.ini. This would allow JKSV be able to correctly restore saves such as skyrim which auto-extend the save size every X blocks.

This can either be a raw dump of the save extra data, or an ini with data_size and journal_size + whatever else may be useful to display to the user when restoring.

related issues: #189 #184 #123

[ITotalJustice]

If having an extra file within the save folder would be causes issues in some games, then the only alternative is to embed the data within the zip. Theres 2 places where it can go, the comment field and the extra field. See https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html for more info and here for a list of reserved id's for the extra field https://libzip.org/specifications/extrafld.txt.

[J-D-K]

Right now, the most the master branch will do is grab the total size of all the files in the folder or zip and extend it to fit them. There is an option in the one menu that allows you to extend the containers manually too. People asked for it years ago for Minecraft.

[ITotalJustice]

Ah, I didn't see the option for that, where abouts is it? I can see that can be useful for importing pc saves yeah, Skyrim is another example of that.

Although ideally jksv should dump enough info along side the save to be able to restore a save from another switch.

[J-D-K]

When you're in the icon grid, highlight the game and press X. I think it's one of the last ones. I remember their size is 4GB max including the journaling space.

I'll figure it out. I'm trying not to rush the rewrite too much. I have a list of stuff I need to straighten out in it and I still need to rework the drive code. I hate the master branch's code so much now. It works, but the rewrite is a completely different beast and isn't as touchy and difficult to work with. I don't know what I was thinking with all the function pointers. Even if I rewrote the thing entirely in C, it would be a lot different.

[ITotalJustice]

Yeah no worries mate. I would personally recommend embedding the data within the zip, rather than having an extra file in the save folder. You could add a struct with a u32 version as the first entry, then whatever else after. That allows you to add things later on by incrementing the version. Up to you however.

Yeah I share your pain about poor code quality. Tbh these days I no longer care how it looks, as long as it's performant and it gets the job done, that's all that matters (that's how I justify it to myself anyway 😄)

[ITotalJustice]

Here is a quick draft of how it could look, mostly pseudo code.

struct JKSaveMeta {
    u32 magic; // JKSV (0x4A4B5356)
    u32 version; // 1
    u64 timestamp; // FsSaveDataExtraData::timestamp
    u64 data_size; // FsSaveDataExtraData::data_size
    u64 journal_size; // FsSaveDataExtraData::journal_size
};

constexpr auto SAVE_META_VERSION1_SIZE = 32;

void CreateSave() {
    FsSaveDataExtraData extra{};
    fsReadSaveDataFileSystemExtraDataBySaveDataSpaceId(&extra, sizeof(extra));

    const JKSaveMeta meta{
        .magic = 0x4A4B5356,
        .version = 1,
        .timestamp = extra.timestamp,
        .data_size = extra.data_size,
        .journal_size = extra.journal_size
    };

    char buf[sizeof(meta) + 1]{};
    std::memcpy(buf, &meta, sizeof(meta));
    zipOpen2(path, 1, buf, NULL);
}

void LoadSave() {
    char buf[sizeof(meta) + 1]{};
    const auto comment_len = unzGetGlobalComment(zfile, buf, sizeof(buf));

    if (comment_len >= SAVE_META_VERSION1_SIZE) {
        JKSaveMeta meta{};
        std::memcpy(&meta, buf, std::min<u32>(comment_len, sizeof(meta)));
        if (meta.magic == 0x4A4B5356) {
            fsExtendSaveDataFileSystem(save_data_space_id, saveID, meta.data_size, meta.journal_size);
        }
    }
}

for zips, the only option seems to be to embed this data as a comment. The extra field is per-file, whereas the comment is global. I've attached code to add a global comment as well as retrieving the global comment.

[J-D-K]

Honestly, I have no issue just making it a file within the zip itself. JKSV only uses unzipping code for restoring saves anyway. if(filename == ".save_info.bin") continue; is all that needs to be done to skip it really. Same with folders. The real challenge is making it not match some obscure game I've never heard of so I don't get an issue a week later that Suki-Chan's iPhone Slumber Party Port 3: No Pantsu Remix isn't restoring right because the one file is named .save_info.bin. I can't believe people buy that stuff and have no problem admitting they play it too.

[ITotalJustice]

For the naming, calling it ".jksv_save_meta.bin" (or anything with jksv within the name) should be good enough. I can't imagine that clashing with any file names out there, unless some rouge inde dev wants to give you a headache.

[J-D-K]

You'd be surprised at some of the stuff I've encountered. There's one in particular that would serve no purpose other than to prevent JKSV (or whatever else) from working correctly. There also some weird Japanese only 3DS game that would detect if JKSM deleted the secure value and just refuse to work. It does happen. But, yeah. Consider it done.

[J-D-K]

This is what I'm rolling with at the moment: https://gist.github.com/J-D-K/87e43a19ac702ef8a8fc1490feeb7627. I think that should have everything covered. I'd push it to the rewrite branch, but I'm still trying to get some stuff sorted before that happens.