Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement memory-based caching #139

Open
kevmo314 opened this issue Feb 27, 2024 · 1 comment
Open

Implement memory-based caching #139

kevmo314 opened this issue Feb 27, 2024 · 1 comment
Assignees

Comments

@kevmo314
Copy link
Owner

Right now, we still write to disk instead of committing to memory with mmap. We should explicitly not write to disk until the user requests. This is a bit more complicated but turns out disk is indeed a pretty big bottleneck :(

@kevmo314 kevmo314 self-assigned this Mar 5, 2024
@kevmo314
Copy link
Owner Author

The key idea here is we want to add an interface that accepts an offset and an BinaryMarshaller, and don't write it to disk but instead store it in a map[uint64]any. Then when the file is closed, write those all to disk. For reads, check if it exists in the cache already and if it does, return the untyped struct. If it doesn't, do the BinaryUnmarshalling

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant