Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: migration to leveldb #1401

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
2c5fb8a
feat: adding initial leveldb configuration
thegrannychaseroperation Jan 15, 2025
08bcf09
feat: adding initial leveldb configuration
thegrannychaseroperation Jan 15, 2025
c59b039
fix: removing unused navigate
thegrannychaseroperation Jan 15, 2025
8b47082
fix: removing unused navigate
thegrannychaseroperation Jan 15, 2025
2c881a6
fix: fixing duplicate export
thegrannychaseroperation Jan 15, 2025
a23106b
feat: migrating achievements to level
thegrannychaseroperation Jan 16, 2025
c115040
fix: fixing sonar issues
thegrannychaseroperation Jan 16, 2025
1f0e195
feat: migrating games to leveldb
thegrannychaseroperation Jan 19, 2025
d760d01
feat: migrating games to level
thegrannychaseroperation Jan 20, 2025
f1e0ba4
feat: migrating user preferences
thegrannychaseroperation Jan 21, 2025
facca3e
chore: merge with main
thegrannychaseroperation Jan 21, 2025
eb11eb2
chore: merge with main
thegrannychaseroperation Jan 21, 2025
0f0a67b
chore: merge with main
thegrannychaseroperation Jan 21, 2025
71cb4cd
chore: level as external dep
thegrannychaseroperation Jan 22, 2025
2aff983
fix: fixing sonar issues
thegrannychaseroperation Jan 22, 2025
f387560
fix: migrating level to classic-level
thegrannychaseroperation Jan 22, 2025
549481f
chore: improving download queue
thegrannychaseroperation Jan 22, 2025
f5532fa
chore: encrypting real-debrid
thegrannychaseroperation Jan 22, 2025
4c5c602
fix: adding bit validation to is deleted
thegrannychaseroperation Jan 22, 2025
bfd54d5
fix: fixing sonar issues
thegrannychaseroperation Jan 22, 2025
dcd1634
fix: fixing sonar issues
thegrannychaseroperation Jan 22, 2025
93fc486
fix: fixing sonar issues
thegrannychaseroperation Jan 22, 2025
a839e51
chore: changing boolean strategy on migration
thegrannychaseroperation Jan 22, 2025
f81e4ac
fix: adding catch to tables
thegrannychaseroperation Jan 22, 2025
3335e98
Merge branch 'main' into feat/migration-to-leveldb
zamitto Jan 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions electron.vite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@ export default defineConfig(({ mode }) => {
build: {
sourcemap: true,
},
css: {
preprocessorOptions: {
scss: {
api: "modern",
},
},
},
resolve: {
alias: {
"@renderer": resolve("src/renderer/src"),
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@
"jsdom": "^24.0.0",
"jsonwebtoken": "^9.0.2",
"knex": "^3.1.0",
"level": "^9.0.0",
"lodash-es": "^4.17.21",
"parse-torrent": "^11.0.17",
"piscina": "^4.7.0",
Expand Down
7 changes: 6 additions & 1 deletion src/main/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,18 @@ export const defaultDownloadsPath = app.getPath("downloads");

export const isStaging = import.meta.env.MAIN_VITE_API_URL.includes("staging");

export const levelDatabasePath = path.join(
app.getPath("userData"),
`hydra-db${isStaging ? "-staging" : ""}`
);

export const databaseDirectory = path.join(app.getPath("appData"), "hydra");
export const databasePath = path.join(
databaseDirectory,
isStaging ? "hydra_test.db" : "hydra.db"
);
thegrannychaseroperation marked this conversation as resolved.
Show resolved Hide resolved

export const logsPath = path.join(app.getPath("appData"), "hydra", "logs");
export const logsPath = path.join(app.getPath("userData"), "hydra", "logs");

export const seedsPath = app.isPackaged
? path.join(process.resourcesPath, "seeds")
Expand Down
4 changes: 0 additions & 4 deletions src/main/data-source.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,7 @@ import {
Game,
GameShopCache,
UserPreferences,
UserAuth,
GameAchievement,
UserSubscription,
} from "@main/entity";

import { databasePath } from "./constants";
Expand All @@ -15,9 +13,7 @@ export const dataSource = new DataSource({
type: "better-sqlite3",
entities: [
Game,
UserAuth,
UserPreferences,
UserSubscription,
GameShopCache,
DownloadQueue,
GameAchievement,
Expand Down
2 changes: 0 additions & 2 deletions src/main/entity/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
export * from "./game.entity";
export * from "./user-auth.entity";
export * from "./user-preferences.entity";
export * from "./user-subscription.entity";
export * from "./game-shop-cache.entity";
export * from "./game.entity";
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: duplicate export of game.entity on line 1 - remove this line

export * from "./game-achievements.entity";
Expand Down
45 changes: 0 additions & 45 deletions src/main/entity/user-auth.entity.ts

This file was deleted.

42 changes: 0 additions & 42 deletions src/main/entity/user-subscription.entity.ts
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: The one-to-one relationship with UserAuth needs to be maintained in the new LevelDB structure to prevent data inconsistency

This file was deleted.

13 changes: 10 additions & 3 deletions src/main/events/auth/get-session-hash.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,20 @@
import jwt from "jsonwebtoken";

import { userAuthRepository } from "@main/repository";
import { registerEvent } from "../register-event";
import { db } from "@main/level";
import type { Auth } from "@types";
import { levelKeys } from "@main/level/sublevels/keys";
import { Crypto } from "@main/services";

const getSessionHash = async (_event: Electron.IpcMainInvokeEvent) => {
const auth = await userAuthRepository.findOne({ where: { id: 1 } });
const auth = await db.get<string, Auth>(levelKeys.auth, {
valueEncoding: "json",
});
Comment on lines +9 to +11
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: db.get() can throw - needs try/catch to handle LevelDB errors gracefully


if (!auth) return null;
const payload = jwt.decode(auth.accessToken) as jwt.JwtPayload;
const payload = jwt.decode(
Crypto.decrypt(auth.accessToken)
) as jwt.JwtPayload;
Comment on lines +14 to +16
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Crypto.decrypt() can throw - needs try/catch to handle decryption failures


if (!payload) return null;

Expand Down
21 changes: 13 additions & 8 deletions src/main/events/auth/sign-out.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import { registerEvent } from "../register-event";
import { DownloadManager, HydraApi, gamesPlaytime } from "@main/services";
import { dataSource } from "@main/data-source";
import { DownloadQueue, Game, UserAuth, UserSubscription } from "@main/entity";
import { DownloadQueue, Game } from "@main/entity";
import { PythonRPC } from "@main/services/python-rpc";
import { db } from "@main/level";
import { levelKeys } from "@main/level/sublevels/keys";

const signOut = async (_event: Electron.IpcMainInvokeEvent) => {
const databaseOperations = dataSource
Expand All @@ -11,13 +13,16 @@ const signOut = async (_event: Electron.IpcMainInvokeEvent) => {

await transactionalEntityManager.getRepository(Game).delete({});

await transactionalEntityManager
.getRepository(UserAuth)
.delete({ id: 1 });

await transactionalEntityManager
.getRepository(UserSubscription)
.delete({ id: 1 });
await db.batch([
{
type: "del",
key: levelKeys.auth,
},
{
type: "del",
key: levelKeys.user,
},
]);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: LevelDB batch operation inside SQLite transaction could leave data in inconsistent state if SQLite transaction fails but LevelDB operations complete

})
.then(() => {
/* Removes all games being played */
Expand Down
14 changes: 9 additions & 5 deletions src/main/events/misc/open-checkout.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
import { shell } from "electron";
import { registerEvent } from "../register-event";
import { userAuthRepository } from "@main/repository";
import { HydraApi } from "@main/services";
import { Crypto, HydraApi } from "@main/services";
import { db } from "@main/level";
import type { Auth } from "@types";
import { levelKeys } from "@main/level/sublevels/keys";

const openCheckout = async (_event: Electron.IpcMainInvokeEvent) => {
const userAuth = await userAuthRepository.findOne({ where: { id: 1 } });
const auth = await db.get<string, Auth>(levelKeys.auth, {
valueEncoding: "json",
});
Comment on lines +8 to +10
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: db.get() can throw if database is corrupted or inaccessible - needs error handling


if (!userAuth) {
if (!auth) {
return;
}
Comment on lines +12 to 14
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: silently returning on missing auth could lead to confusing UX - consider throwing UserNotLoggedInError instead


const paymentToken = await HydraApi.post("/auth/payment", {
refreshToken: userAuth.refreshToken,
refreshToken: Crypto.decrypt(auth.refreshToken),
}).then((response) => response.accessToken);
Comment on lines 16 to 18
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: no error handling for failed API requests or decryption failures - should wrap in try/catch


const params = new URLSearchParams({
Expand Down
11 changes: 7 additions & 4 deletions src/main/events/user/get-user-friends.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,19 @@
import { userAuthRepository } from "@main/repository";
import { db } from "@main/level";
import { registerEvent } from "../register-event";
import { HydraApi } from "@main/services";
import type { UserFriends } from "@types";
import type { User, UserFriends } from "@types";
import { levelKeys } from "@main/level/sublevels/keys";

export const getUserFriends = async (
userId: string,
take: number,
skip: number
): Promise<UserFriends> => {
const loggedUser = await userAuthRepository.findOne({ where: { id: 1 } });
const user = await db.get<string, User>(levelKeys.user, {
valueEncoding: "json",
});

Comment on lines +12 to +14
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: no error handling for failed db.get() operations - could throw unhandled exceptions

if (loggedUser?.userId === userId) {
if (user?.id === userId) {
return HydraApi.get(`/profile/friends`, { take, skip });
}

Expand Down
3 changes: 3 additions & 0 deletions src/main/level/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
export { db } from "./level";

export * from "./sublevels";
4 changes: 4 additions & 0 deletions src/main/level/level.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
import { levelDatabasePath } from "@main/constants";
import { Level } from "level";

export const db = new Level(levelDatabasePath, { valueEncoding: "json" });
7 changes: 7 additions & 0 deletions src/main/level/sublevels/games.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import { Game } from "@types";
import { db } from "../level";
import { levelKeys } from "./keys";

export const gamesSublevel = db.sublevel<string, Game>(levelKeys.games, {
valueEncoding: "json",
});
1 change: 1 addition & 0 deletions src/main/level/sublevels/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from "./games";
8 changes: 8 additions & 0 deletions src/main/level/sublevels/keys.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import type { GameShop } from "@types";

export const levelKeys = {
games: "games",
game: (shop: GameShop, objectId: string) => `${shop}:${objectId}`,
user: "user",
auth: "auth",
};
7 changes: 0 additions & 7 deletions src/main/repository.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,7 @@ import {
Game,
GameShopCache,
UserPreferences,
UserAuth,
GameAchievement,
UserSubscription,
} from "@main/entity";

export const gameRepository = dataSource.getRepository(Game);
Expand All @@ -18,10 +16,5 @@ export const gameShopCacheRepository = dataSource.getRepository(GameShopCache);

export const downloadQueueRepository = dataSource.getRepository(DownloadQueue);

export const userAuthRepository = dataSource.getRepository(UserAuth);

export const userSubscriptionRepository =
dataSource.getRepository(UserSubscription);

export const gameAchievementRepository =
dataSource.getRepository(GameAchievement);
28 changes: 28 additions & 0 deletions src/main/services/crypto.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import { safeStorage } from "electron";
import { logger } from "./logger";

export class Crypto {
public static encrypt(str: string) {
if (safeStorage.isEncryptionAvailable()) {
return safeStorage.encryptString(str).toString("base64");
} else {
logger.warn(
"Encrypt method returned raw string because encryption is not available"
);

return str;
}
}

public static decrypt(b64: string) {
if (safeStorage.isEncryptionAvailable()) {
return safeStorage.decryptString(Buffer.from(b64, "base64"));
} else {
logger.warn(
"Decrypt method returned raw string because encryption is not available"
);

return b64;
}
}
}
3 changes: 2 additions & 1 deletion src/main/services/hosters/datanodes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ export class DatanodesApi {
"User-Agent":
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36",
},
maxRedirects: 0, validateStatus: (status: number) => status === 302 || status < 400,
maxRedirects: 0,
validateStatus: (status: number) => status === 302 || status < 400,
}
);

Expand Down
Loading
Loading