v1.6.0: Add periodic cleaning for `concat`

Changes

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Caution

This release contains a critical bug in the HLS downloader. Some streams may get corrupted (10x size).
Please use version >= v1.6.2.

• Added deletion of old .ts that has an existing .combined.* file:

  defaultParams:
    ## Keep the raw .ts recordings after it has been remuxed. (default: false)
    ##
    ## If this option is set to false and concat is true, before every "waiting for stream to be online", a scan will be executed to detect *.combined.* files.
    ## The scan will be done on the directory of `scanDirectory`.
    ## If a non-corrupted .combined. file is detected, it will remove .ts older than `eligibleForCleaningAge`.
    ## After the cleaning, the .combined files will be renamed without the ".combined" part (if a file already exists due to remux, it won't be renamed).
    keepIntermediates: false
    ## Directory to be scanned for .ts files to be deleted after concatenation. (default: '')
    ##
    ## Scan is recursive.
    ##
    ## Empty value means no scanning.
    scanDirectory: ''
    ## Minimum age of .combined files to be eligible for cleaning. (default: 48h)
    ##
    ## The minimum should be the expected duration of a stream to avoid any race condition.
    eligibleForCleaningAge: '48h'

  The cleaner is ran every hour.

• Added corrupted file deletion feature.

  defaultParams:
    ## Delete corrupted .ts recordings. (default: true)
    deleteCorrupted: true

• Fix: Fixed various memory leaks with concat.

• Added clean command

• Fix: Instead of checking if a file exists to run remux, probe the file to see if the file is a video (equivalent to ffprobe).

• Fix: Concat has a better ordering with numbers.

SHA256 Checksums

2d8aaeeda04f1c11c4eef46a980354e510676d37658fc4f3b6460d8f5946cc84 *fc2-live-dl-go-darwin-amd64
94428d66c1882f64d05ab5c5624ceec2f7ae0655ee310e3063edd37580494573 *fc2-live-dl-go-darwin-arm64
35e11e080617be2844be10986188d6c77e6465fa738d63ed2356b3c87d09c219 *fc2-live-dl-go-linux-amd64
00909c99652607afac6699e502797db857f508d798f6c11e06cbee570123609b *fc2-live-dl-go-linux-arm64
5aec6e5fd143834b641b2b1cf8831864d16dc8daf064887ba464b5a63814291e *fc2-live-dl-go-linux-riscv64
e20dfa65119e8dd038cc7141c0b92b2aa22efc490d3337d35b1f315498dfdb84 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.5.1...v1.6.0

v1.6.0-rc.4

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Fixes

• Run cleaner periodically.
• Fix memory leaks (one related to args, the other related the a missing av_packet_unref.

SHA256 Checksums

26294d3839ca8967e543c46b2205d6ad6b7ee368f5c834e69003c7f16568a6ba *fc2-live-dl-go-darwin-amd64
855d7082a7ee08df33fd049516d64865a88804b80a46d57fad46d9bb00350c6e *fc2-live-dl-go-darwin-arm64
7dd9192433e682b78850b1f841aefafe492ec7ec59d2640dbe3c34cc480ac96a *fc2-live-dl-go-linux-amd64
6f1a0ab38dd00bfb1300734112748175f4f8ed48af1c80c6a36c7390f675f3cc *fc2-live-dl-go-linux-arm64
19b1be764cea2b9ef2037af793c0467c9bcd6cc9455fade123bf1f9052c71e51 *fc2-live-dl-go-linux-riscv64
3112488783dfdb06178a3e682b8c77b18928d29487450544e0a43c49b69d5941 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.0-rc.2...v1.6.0-rc.4

v1.6.0-rc.2: Delete old intermediate files after concat

Changes

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Caution

An issue has been found when concatenating: possible memory leak when concatenating audio (concatenating video is fine).

• Added deletion of old .ts that has an existing .combined file:

  defaultParams:
    ## Keep the raw .ts recordings after it has been remuxed. (default: false)
    ##
    ## If this option is set to false and concat is true, before every "waiting for stream to be online", a scan will be executed to detect *.combined.* files.
    ## The scan will be done on the directory of `scanDirectory`.
    ## If a non-corrupted .combined. file is detected, it will remove .ts older than `eligibleForCleaningAge`.
    ## After the cleaning, the .combined files will be renamed without the ".combined" part (if a file already exists due to remux, it won't be renamed).
    keepIntermediates: false
    ## Directory to be scanned for .ts files to be deleted after concatenation. (default: '')
    ##
    ## Scan is recursive.
    ##
    ## Empty value means no scanning.
    scanDirectory: ''
    ## Minimum age of .combined files to be eligible for cleaning. (default: 48h)
    ##
    ## The minimum should be the expected duration of a stream to avoid any race condition.
    eligibleForCleaningAge: '48h'

• Added corrupted file deletion feature.

  defaultParams:
    ## Delete corrupted .ts recordings. (default: true)
    deleteCorrupted: true

• Added clean command

• Fix: Instead of checking if a file exists to run remux, probe the file to see if the file is a video (equivalent to ffprobe).

• Fix: Concat has a better ordering with numbers.

SHA256 Checksums

c0baa26f13a49d43db52020b519e8eee99a10dff844a2dcbbed0daa6f6b7e352 *fc2-live-dl-go-darwin-amd64
866faf2d6e2db5bf732e6096f24d57044d8fcf690b01e38826a9b0974b837c61 *fc2-live-dl-go-darwin-arm64
2c45f594c4756444e2484cca3374c01bbf6e935aec081645884fb168e6e9f825 *fc2-live-dl-go-linux-amd64
4c76b22fb8aad1465530cc76055a821804fe3350f5eedf9987c09cf1d0dc39e3 *fc2-live-dl-go-linux-arm64
b020ee0cb2c123aee788a02593e5cf1755827743009964b62df7e7c5551b7b82 *fc2-live-dl-go-linux-riscv64
ce2e3a7296056c392cc65637a9dae8cb5e1a30daaa491d367074de9edb3438a7 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.5.1...v1.6.0-rc.2

v1.6.0-alpha.3: Fix critical bug where .ts gets deleted before concatenation

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Fixes

• Keep intermediates when remux is disabled.
• Fix wrong probe during intermediates cleaning.
• Make probe quiet
• Check during cleaning if the file is actually a video.

SHA256 Checksums

0d31316ac64a716477efcfa8a809f542c49bec8abf36657acec1ed2035ae0953 *fc2-live-dl-go-darwin-amd64
fb8bdce665c110900f7042ed6ba6a6ed9b2abac1c6cf15de63f5ca2e8fe1ff6b *fc2-live-dl-go-darwin-arm64
248af05e4e6e960eb8055e1fbc79bb36342de7473c6af129167ece70d2d4784f *fc2-live-dl-go-linux-amd64
0e92c8d62dda466dbb547653959d987ac4db77853fb4c8d3089c127e73d863fe *fc2-live-dl-go-linux-arm64
54060f194fd7f2dfbe3568f944db258734c21782f5c732c66d92f4ede5b356f1 *fc2-live-dl-go-linux-riscv64
e121fb468e835bd17d7717cfe31238439be41e1cec108c58987d983b91ed93c2 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.6.0-alpha.1...v1.6.0-alpha.3

v1.6.0-alpha.1: Delete old ts file when concat

Changes

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Caution

This release contain a bug in the cleaner where .combined.m4a is considered as valid for cleaning. Audio files should not force the removal of intermediates file.
This version deletes .ts when remux is disabled, resulting in the loss of the stream, making concatenation impossible.
Please use version >= v1.6.0-alpha.3.

• Added deletion of old .ts that has an existing .combined file:

  defaultParams:
    ## Keep the raw .ts recordings after it has been remuxed. (default: false)
    ##
    ## If this option is set to false and concat is true, before every "waiting for stream to be online", a scan will be executed to detect *.combined.* files.
    ## The scan will be done on the directory of `outFormat`.
    ## If a non-corrupted .combined. file is detected, it will remove .ts older than 48h.
    keepIntermediates: false
    ## Directory to be scanned for .ts files to be deleted after concatenation. (default: '')
    ##
    ## Scan is recursive.
    ##
    ## Empty value means no scanning.
    scanDirectory: ''

• Added corrupted file deletion feature.

  defaultParams:
    ## Delete corrupted .ts recordings. (default: true)
    ##
    ## By corrupted, we mean that it's not even openable by ffprobe, i.e, not recoverable.
    deleteCorrupted: true

• Fix: Instead of checking if a file exists to run remux, probe the file to see if the file is a video (equivalent to ffprobe).

• Fix: Concat has a better ordering with numbers.

Full Changelog: v1.5.1...v1.6.0-alpha.1

v1.5.1: when concatenating, skip corrupted input files

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Caution

This version contains a critical bug where it deletes .ts when remux is disabled, resulting in the loss of the stream, making concatenation impossible.
Please use version >= v1.6.0-alpha.3, or rollback to v1.4.1.

Caution

This release contains a critical bug in the HLS downloader. Some streams may get corrupted (10x size).
Please use version >= v1.6.2.

Fixes

• When concatenating, skip corrupted input files.

Full Changelog: v1.5.0...v1.5.1

v1.5.0: add concatenation feature

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Caution

This version contains a critical bug where it deletes .ts when remux is disabled, resulting in the loss of the stream, making concatenation impossible.
Please use version >= v1.6.0-alpha.3, or rollback to v1.4.1.

Caution

This release contains a critical bug in the HLS downloader. Some streams may get corrupted (10x size).
Please use version >= v1.6.2.

concat feature added

defaultParams:
  ## Concatenate and remux with previous recordings after it is finished. (default: false)
  ##
  ## WARNING: We recommend to DISABLE remux since concat also remux.
  ##
  ## Files must be named <name>.<n>.<ts/mp4/mkv...>. If n=0, n is optional.
  ## Output will be named: "<name>.combined.<remuxFormat>".
  ##
  ## n is only used to determine the order. If there are missing fragments,
  ## the concatenation will still be executed.
  ##
  ## The extensions do not matter. A 1.ts and a 2.mp4 will still be concatenated together.
  ## TS files will be prioritized over anything else.
  ##
  ## If remux is enabled, remux will be executed first, then the concatenation will be executed.
  ##
  ## If extractAudio is true, the m4a will be concatenated separatly.
  ##
  ## The previous files won't be deleted even if keepIntermediates is false. In
  ## the future, they will be deleted by scheduling a deletion based on the
  ## modification time.
  ##
  ## TL;DR: This is to concatenate if there is a crash.
  concat: false

I recommend disabling remux when using concat. concat uses the same logic as remux, but applied to multiple input files. keepIntermediates has no impact on concat for now. In the future, I will implement a garbage cleaner that will delete .ts files after a certain time and if a .combined.<ext> file is present.

You can still use both remux and concat, if you want to.

Fixes

• Fixed DoWithContextTimeoutWithResult where it didn't exit when there is no error.

Full Changelog: v1.4.1...v1.5.0

v1.5.0-alpha.1: add concatenation feature

concat feature added

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Caution

This version contains a critical bug where it deletes .ts when remux is disabled, resulting in the loss of the stream, making concatenation impossible.
Please use version >= v1.6.0-alpha.3, or rollback to v1.4.1.

defaultParams:
  ## Concatenate and remux with previous recordings after it is finished. (default: false)
  ##
  ## WARNING: We recommend to DISABLE remux since concat also remux.
  ##
  ## Files must be named <name>.<n>.<ts/mp4/mkv...>. If n=0, n is optional.
  ## Output will be named: "<name>.combined.<remuxFormat>".
  ##
  ## n is only used to determine the order. If there are missing fragments,
  ## the concatenation will still be executed.
  ##
  ## The extensions do not matter. A 1.ts and a 2.mp4 will still be concatenated together.
  ## TS files will be prioritized over anything else.
  ##
  ## If remux is enabled, remux will be executed first, then the concatenation will be executed.
  ##
  ## If extractAudio is true, the m4a will be concatenated separatly.
  ##
  ## The previous files won't be deleted even if keepIntermediates is false. In
  ## the future, they will be deleted by scheduling a deletion based on the
  ## modification time.
  ##
  ## TL;DR: This is to concatenate if there is a crash.
  concat: false

I recommend disabling remux when using concat. concat uses the same logic as remux, but applied to multiple input files. keepIntermediates has no impact on concat for now. In the future, I will implement a garbage cleaner that will delete .ts files after a certain time and if a .combined.<ext> file is present.

Fixes

• Fixed DoWithContextTimeoutWithResult where it didn't exit when there is no error.

Full Changelog: v1.4.1...v1.5.0-alpha.1

v1.4.1: `priority` can be disabled for notifications

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Caution

This release contains a critical bug in the HLS downloader. Some streams may get corrupted (10x size).
Please use version >= v1.6.2.

priority can be disabled for unsupported notification transports

Some notification services throw an error if a priority key is sent in the notification request. Priorities can now be disabled by adding:

 notifier:
+  noPriority: true

SHA256 Checksums

4eb683e9c0087ac95baa3c2b9356007e809f067c896fafdcab91d02872385f33 *fc2-live-dl-go-darwin-amd64
67ca80743d76e151f67db50e3bab98ef79e981db74ad62b5e496b0398325dafd *fc2-live-dl-go-darwin-arm64
3347f298d56f69c16f7ad11a908918904c90590a24e18d6ba73278b90528e077 *fc2-live-dl-go-linux-amd64
165b5b5b577b3eda6537573ae8f856b1bb7f5d7c3d6e13451b4dc011ffaab5fb *fc2-live-dl-go-linux-arm64
7d6f10564b86d66fa6e083f859419ba61d3d439fed1eb260f4f2745dc4148df8 *fc2-live-dl-go-linux-riscv64
a9c1f2e13ea342ef6320bd8f6635d8da6284bfe6edf4256b8c0bf27565d711b3 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.4.0...v1.4.1

v1.4.0: Customizable remux format

Caution

Windows binaries are not statically compiled since v1.1.3+3. This is obviously a bug that need to be fixed.

Caution

This release contains a critical bug in the HLS downloader. Some streams may get corrupted (10x size).
Please use version >= v1.6.2.

Remux format can now be changed.

The option remuxFormat has been added to the config.yaml file:

   cookiesRefreshDuration: '24h'
   ## Remux recordings into mp4/m4a after it is finished. (default: true)
   remux: true
+  ## Remux format (default: mp4)
+  remuxFormat: 'mp4'
   ## Keep the raw .ts recordings after it has been remuxed. (default: false)
   keepIntermediates: false
   ## Generate an audio-only copy of the stream. (default: false)

Note that remuxing does not include re-encoding, the video will still be in the original stream format (mpeg), as well with the audio (aac). Only the container can be changed (mkv, mp4, ...).

SHA256 Checksums

3f2e2e36263a87226ca8a6ce85320db33c689ffb26d5e923eccc02559cfabc0f *fc2-live-dl-go-darwin-amd64
e0fb1fe94bea491c1c9ba3038a00c3bcf1b0641700dff979c260051a684c5f0a *fc2-live-dl-go-darwin-arm64
c47ebedd8fb7a9fb4fc78479b269565182baf6ece21872a82b2b8843a2427d61 *fc2-live-dl-go-linux-amd64
fb6d6e77cc03290d9cf7963fd5e9ad8434938bfce7e3f161d2453be8e9e700d6 *fc2-live-dl-go-linux-arm64
1ac306a30e4de74153cde656163f3bc5c0b0543e185c983168a6e6f29e0230a5 *fc2-live-dl-go-linux-riscv64
bf3a9b939b400a04db37e4ef3b8f8277d303a2b28dfec762231201dfc3b76432 *fc2-live-dl-go-windows-amd64.exe

Full Changelog: v1.3.2...v1.4.0