Importing library takes ages for no apparent reason

[titivermeesch]

Hello everyone,

I'm currently trying to setup photoprism (docker-compose with mariadb) as my main photo/video storage but it seems impossible to import my library smoothly.
I started by upload some photos and videos (around 50GB) in the import folder (which I configured through the docker compose volumes).
A few pictures went through but now it seems to be stuck at videos. Here are the logs I got from the container:

time="2025-02-09T11:12:24Z" level=info msg="moving files from import"
time="2025-02-09T11:12:24Z" level=info msg="classify: loading nasnet"
time="2025-02-09T11:12:40Z" level=info msg="classify: loading labels from labels.txt"
time="2025-02-09T11:12:41Z" level=info msg="media: DJI_20250207142243_0001_D.MP4 was taken at 2025-02-07 13:22:44 +0000 UTC (meta)"
time="2025-02-09T11:12:41Z" level=info msg="import: moving main mp4 file DJI_20250207142243_0001_D.MP4 to 2025/02/20250207_132244_55C11117.mp4"
time="2025-02-09T11:12:42Z" level=info msg="convert: converting 20250207_132244_55C11117.mp4 to 20250207_132244_55C11117.mp4.jpg (ffmpeg)"
time="2025-02-09T11:13:14Z" level=info msg="convert: 20250207_132244_55C11117.mp4.jpg created in 32.118816316s (ffmpeg)"
time="2025-02-09T11:13:18Z" level=info msg="media: generated 10 thumbnails for 2025/02/20250207_132244_55C11117.mp4.jpg [4.275034827s]"
time="2025-02-09T11:13:19Z" level=info msg="import: added main mp4 file 2025/02/20250207_132244_55C11117.mp4"
time="2025-02-09T11:14:00Z" level=info msg="index: matched 2 labels with 20250207_132244_55C11117.mp4.jpg [15.164560181s]"
time="2025-02-09T11:14:01Z" level=info msg="photo: created sidecar file 2025/02/20250207_132244_55C11117.yml"
time="2025-02-09T11:14:01Z" level=info msg="import: stacked related jpg file 2025/02/20250207_132244_55C11117.mp4.jpg"
time="2025-02-09T11:42:31Z" level=warning msg="metadata: unknown json in f78d96beb6a2fa63bdda6aaf5667bc8fd9acc124_exiftool.json"
time="2025-02-09T11:42:31Z" level=info msg="media: DJI_20250208132248_0002_D.MP4 was taken at 2025-02-08 12:42:41.87 +0000 UTC (file mod time)"

As you see between 11:14 and 11:42 there are no logs, even though the videos were in the import folder. So it took around 30 minutes to start processing the next video, I'm not entirely sure what is going on.
The photoprism instance run on a Raspberry PI 4 B with 8GB of RAM along with 16GB swap and has a total of 4 cores (which don't seem to be used that much, the ram is also not above 1.5GB/8GB at any time).
Photoprism itself is running on a 500Gb SDD and the storage volumes are on a 5TB modern HDD.

I understand that since I'm running my storage on a HDD that reading or copying a 16Gb file may be a bit slow, but I would like to understand while it took 30 minutes between the 2 video files while that video file 0001 was 18Mb. 0002 on the other hand is a bit short of 18Gb

Anyone knows what the issue could be? Or is it just expected to have these kind of delays with video files?

[lastzero]

You should use a single volume for storage and originals in this case (eg mount /photoprism), so that files can be moved/renamed when importing them. This will be much much faster.

[titivermeesch]

I have originals mounted to /photos and storage to /photoprism/storage, both are on the same physical storage device. Is that not how it's supposed to be?

[lastzero]

Sorry, I meant import and originals folder (not storage). Using separate mounts for both is working, but will be slow when importing many files to originals since for the photoprism container, it looks like separate drives.

[titivermeesch]

And that would then mean I should upload my pictures in the originals folder somewhere?