Skip to content

Commit

Permalink
Added option to remove files instant
Browse files Browse the repository at this point in the history
  • Loading branch information
madslundt committed Sep 18, 2017
1 parent d5826b5 commit 6bfdf3e
Show file tree
Hide file tree
Showing 4 changed files with 45 additions and 7 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,10 @@ Environment variables:
* `-e CLEAR_CHUNK_AGE` - Plexdrive: The maximum age of a cached chunk file (default **24h**) - this is ignored if `CLEAR_CHUNK_MAX_SIZE` is set
* `-e MONGO_DATABASE` - Mongo database used for Plexdrive (default **plexdrive**)
* `-e DATE_FORMAT` - Date format for loggin (default **+%F@%T**)
* `-e REMOVE_LOCAL_FILES_BASED_ON` - Remove local files based on `space` or `time` (default **space**)
* `-e REMOVE_LOCAL_FILES_WHEN_SPACE_EXCEEDS_GB` - Remove local files when local storage exceeds this value in GB (default **100**) - this is ignored if `REMOVE_LOCAL_FILES_BASED_ON` is set to time
* `-e FREEUP_ATLEAST_GB` - Remove atleast this value in GB on removal (default **80**) - this is ignored if `REMOVE_LOCAL_FILES_BASED_ON` is set to time
* `-e REMOVE_LOCAL_FILES_AFTER_DAYS` Remove local files older than this value in days (default **10**) - this is ignored if `REMOVE_LOCAL_FILES_BASED_ON` is set to space
* `-e REMOVE_LOCAL_FILES_BASED_ON` - Remove local files based on `space`, `time` or `instant` (default **space**)
* `-e REMOVE_LOCAL_FILES_WHEN_SPACE_EXCEEDS_GB` - Remove local files when local storage exceeds this value in GB (default **100**) - this is ignored if `REMOVE_LOCAL_FILES_BASED_ON` is set to time or instant
* `-e FREEUP_ATLEAST_GB` - Remove atleast this value in GB on removal (default **80**) - this is ignored if `REMOVE_LOCAL_FILES_BASED_ON` is set to time or instant
* `-e REMOVE_LOCAL_FILES_AFTER_DAYS` Remove local files older than this value in days (default **10**) - this is ignored if `REMOVE_LOCAL_FILES_BASED_ON` is set to space or instant
* `-e READ_ONLY` If Rclone and Plexdrive should be read only or not. 0 means writeable and 1 means read only (default **1**)
* `-e PGID` Group id
* `-e PUID` User id
Expand Down Expand Up @@ -174,7 +174,7 @@ Everytime new media is retrieved it should be added to `/local-media`. By adding

By having a cronjob to rmlocal it will sooner or later move media from `/local-decrypt` depending on the `REMOVE_LOCAL_FILES_BASED_ON` setting. Media is only removed from `/local-decrypt` and still appears in `/local-media` because it is still be accessable from the cloud.

If `REMOVE_LOCAL_FILES_BASED_ON` is set to **space** it will only remove content (if local media size has exceeded `REMOVE_LOCAL_FILES_WHEN_SPACE_EXCEEDS_GB`) starting from the oldest accessed file and will only free up atleast `FREEUP_ATLEAST_GB`. If **time** is set it will only remove files older than `REMOVE_LOCAL_FILES_AFTER_DAYS`.
If `REMOVE_LOCAL_FILES_BASED_ON` is set to **space** it will only remove content (if local media size has exceeded `REMOVE_LOCAL_FILES_WHEN_SPACE_EXCEEDS_GB`) starting from the oldest accessed file and will only free up atleast `FREEUP_ATLEAST_GB`. If **time** is set it will only remove files older than `REMOVE_LOCAL_FILES_AFTER_DAYS`. If **instant** is set it will remove all files when running.

*Media is never deleted locally before being uploaded successful to the cloud.*

Expand Down
2 changes: 1 addition & 1 deletion root/etc/services.d/mongodb/run
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ umask 022

mongodb_command="mongod --logpath ${log_dir}/mongod.log"

echo "Executing ${mongodb_command}"
echo "Started ${mongodb_command}"
exec s6-setuidgid abc $mongodb_command
2 changes: 1 addition & 1 deletion root/etc/services.d/mount/run
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ umask 022

mount_command="mount"

echo "Executing ${mount_command}"
echo "Started ${mount_command}"
exec $mount_command
38 changes: 38 additions & 0 deletions scripts/rmlocal.script
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,40 @@ rm_time () {
find "${local_decrypt_dir}" -mindepth 1 -type d -empty -delete
}

rm_instant () {
# Generate filelist and iterate through it...
find "${local_decrypt_dir}" -type f |
while read -r n; do

# Find the pathname relative to the root of your remote and store filename
filename="$(echo "$n" | sed -e s@"${local_decrypt_dir}"@@)"
destpath="$(dirname "$n" | sed -e s@"${local_decrypt_dir}"@@)"

# Skip hidden or partial files.
case "$n" in
(*.partial~) continue ;;
(*_HIDDEN~) continue ;;
(*.QTFS) continue ;;
(*.unionfs-fuse*) continue ;;
(*.DS_STORE) continue ;;
esac

# If file is opened by another process, wait until it isn't.
while [ "$(lsof "$n" >/dev/null 2>&1)" ] || \
[ "$(lsof "${local_decrypt_dir}/${n}" >/dev/null 2>&1)" ] || \
[ "$(lsof "${local_media_dir}/${n}" >/dev/null 2>&1)" ]; do
echo "[ $(date $(printenv DATE_FORMAT)) ] File -> ${n} in use. Retrying in 10 seconds."
sleep 10
done

# Move file to remote destination[s], retaining path
echo "[ $(date $(printenv DATE_FORMAT)) ] Moving file -> ${n} to Google Drive."
rclone move $rclone_options "$@" "$n" "$(printenv RCLONE_CLOUD_ENDPOINT)${destpath}" >/dev/null 2>&1
done

find "${local_decrypt_dir}" -mindepth 1 -type d -empty -delete
}

rm_space () {
maxSize=$(($REMOVE_LOCAL_FILES_WHEN_SPACE_EXCEEDS_GB * 1000 * 1000 * 1000))
currentSize="$(du -sb "$local_decrypt_dir" | awk '{print $1}')"
Expand Down Expand Up @@ -103,10 +137,14 @@ fi

check_rclone_cloud

echo "Removing files based on ${REMOVE_LOCAL_FILES_BASED_ON}"

if [ "$REMOVE_LOCAL_FILES_BASED_ON" = "space" ]; then
rm_space
elif [ "$REMOVE_LOCAL_FILES_BASED_ON" = "time" ]; then
rm_time
elif [ "$REMOVE_LOCAL_FILES_BASED_ON" = "instant" ]; then
rm_instant
else
echo "[ $(date $(printenv DATE_FORMAT)) ] no option to remove old files"
exit 02
Expand Down

0 comments on commit 6bfdf3e

Please sign in to comment.