Using compression algorithms to improve file transfer speeds in GS Connect #1650
Replies: 4 comments 7 replies
-
|
It's a good idea, but would definitely have to be implemented upstream first. There would have to be a way for clients to negotiate compression formats, although I'm not sure a user will be able to guess the best algorithm. You can open a bug at https://bugs.kde.org to start a conversation with the upstream team. |
Beta Was this translation helpful? Give feedback.
-
|
The list of benefits should also be caveat-ed (? that can't be a word) with "some". "Some" files will see speedups, "some" large files will transfer faster, etc. If you're transferring an already-compressed file format, the benefits of applying further transfer compression may be negligible or even negated (resulting in slower transfers of accidentally-larger amounts of data). Ideally a wire compression algorithm will recognize when it's doing more harm than good and auto-disable itself, but that requires even more logic in the implementation, and more complex endpoint negotiations. I'm not saying it's a bad idea, at all! Just that it's a bad idea to do it wrong. 😉 |
Beta Was this translation helpful? Give feedback.
-
|
(You can even replace "some" with "many", if preferred. I can get behind "many".) |
Beta Was this translation helpful? Give feedback.
-
|
In my opinion, compression can increase CPU usage and memory usage during compression and decompression, especially on devices with limited resources. But it can increase transfer speed significantly in modern devices. Therefore, I think we should give the user the option to choose whether or not to use the compression algorithm. We would also need to test different compression algorithms to see which ones perform both faster compression and decompression while also significantly reducing file size. Please feel free to recommend some compression algorithms. I have also created a discussion in the KDE community about this. I am interested in working on this project and would really appreciate any help in this area. Any suggestions or help would be greatly appreciated. 😃 |
Beta Was this translation helpful? Give feedback.
-
|
I don't see much point in us testing compression algorithms as we don't know what users are transferring. I for example rarely use GSConnect to transfer files, but when I do it's always images that are already compressed as much as they're going to compress. That said, I'm under the impression that a modern good choice for "do some compression without too much CPU effort" is zstd at like level 3. |
Beta Was this translation helpful? Give feedback.
-
|
Something like zstd was my thought as well, general purpose and low compression rate seems most sensible. There's an old zlib converter you can plug into streams in GIO, but other algorithms might be an issue unless they're available in libarchive or something. |
Beta Was this translation helpful? Give feedback.
-
|
Interestingly, it appears that's one of the ways people have hooked up zstd: Nestled inside an emulation wrapper, so it implements the zlib API and can be used as a drop-in replacement in code that's written for zlib. That was 5 years ago (when zstd was a lot less mature/complex), and the impact on performance due to the wrapper was... not insignificant. (Relative to code that's written to take full advantage of the zstd APIs, that is. Relative to linking with the actual-zlib using its own APIs, a zlib-emulating zstd was faster even with the extra overhead!) |
Beta Was this translation helpful? Give feedback.
-
Yeah, IMHO having to choose an algorithm is a losing proposition — there's no right answer, but there are dozens of wrong ones. Whether it's us testing various implementations against payloads that will never hope to represent most users' typical needs; the code trying to knee-jerk guess how to handle things on the fly; or (*shudder*) asking the poor users to make a choice they couldn't possibly get right (and shouldn't ever have to try)... nuts to all of that. We're better off finding one reasonable algorithm and just going with that. The possible exception being files over a certain minimum length. Since any file could potentially be a compressed file (like one of your images), and failed compression gets more wasteful and counterproductive the more you do it, running the first 1-2 MB of bigger files through the algorithm locally might be a good pre-check step. When it fails to come out smaller-enough (or, worse, comes out bigger) we're in that more-harm-than-good scenario. Then the only right answer is to forcibly "negotiate" an uncompressed transfer for that file. (...This is all assuming we're the sending party, of course.) |
Beta Was this translation helpful? Give feedback.
-
|
If user really worry about this kind of performance so they should compres it by themself and share it to another device. In general it doesn't nacssaary |
Beta Was this translation helpful? Give feedback.
-
|
Actually as linux community is growing outside of developers to normal users most of the common people don't really want to do extra work, they just want a system that will do necessary work with in a less time. |
Beta Was this translation helpful? Give feedback.
-
|
Also, when the device on the other end is running Android, decompression tools are somewhat limited. So, saying "just deal with it yourself" isn't entirely realistic. (Though, broadly and in general, I'm on board with that sort of argument.) The fact is, users DON'T "really worry about this kind of performance". And they shouldn't have to worry about it. |
Beta Was this translation helpful? Give feedback.
-
Description:
GS Connect is a great tool for sharing files between devices. However, the file transfer speeds can be slow for large files. This is because GS Connect does not currently support compression algorithms.
Compression algorithms can significantly improve file transfer speeds by reducing the size of the files that are transferred. This is because compression algorithms remove redundant data from the files, which can significantly reduce the amount of data that needs to be transferred.
I propose that GS Connect should be updated to support compression algorithms. This would allow users to transfer large files much faster.
Benefits:
Drawbacks:
Implementation:
The following steps would be required to implement this idea:
Discussion:
I would like to start a discussion about this idea on the GS Connect GitHub repository. I would like to hear from other users about their thoughts on this idea and their experiences with file transfer speeds in GS Connect.
I believe that this is a valuable improvement that would make GS Connect even more useful. I hope that you will consider supporting this idea.
Thank you for your time.
Beta Was this translation helpful? Give feedback.
All reactions