common: general improvements #695
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reviewer's Guide by SourceryThis pull request introduces several improvements to the common module, including retry logic for downloads, enhanced error handling, and logging. It also adds an MIT license header. Sequence diagram for improved file download with retry logicsequenceDiagram
participant C as Client
participant D as download_file()
participant H as HttpClient
participant S as Server
C->>D: download_file(url, dest_path)
activate D
rect rgb(200, 255, 200)
note over D: Retry loop (max 5 attempts)
D->>H: init(url, headers, output_file)
activate H
H->>S: HTTP GET request
alt Success
S-->>H: File data
H-->>D: Success
D-->>C: Return
else Timeout/Error
S--xH: Timeout/Error
H--xD: Error
note over D: Log error & wait
note over D: Exponential backoff
D->>H: Retry request
end
deactivate H
end
alt Max retries reached
D--xC: Exit with error
end
deactivate D
Class diagram showing download_file function changesclassDiagram
class download_file {
+download_file(url: str, dest_path: str, headers: dict, show_progress: bool)
}
class HttpClient {
+init(url: str, headers: dict, output_file: str, progress: bool)
}
class Logger {
+info(message: str)
+warning(message: str)
+error(message: str)
+debug(message: str)
}
download_file ..> HttpClient : uses
download_file ..> Logger : uses
note for download_file "Added retry mechanism\nImproved error handling\nAdded logging"
State diagram for download retry mechanismstateDiagram-v2
[*] --> Downloading
Downloading --> Success: Download complete
Downloading --> RetryWait: Error/Timeout
RetryWait --> Downloading: Retry attempt
RetryWait --> Failed: Max retries reached
Success --> [*]
Failed --> [*]
note right of RetryWait
Exponential backoff:
0.1s, 0.2s, 0.4s...
end note
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Tested on macOS, I will give a try on Linux, please wait. |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @dougsland - I've reviewed your changes - here's some feedback:
Overall Comments:
- The download_file() function should raise RuntimeError after max retries instead of calling sys.exit(1). This allows callers to handle the error appropriately.
Here's what I looked at during the review
- 🟡 General issues: 2 issues found
- 🟢 Security: all looks good
- 🟢 Testing: all looks good
- 🟢 Complexity: all looks good
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
On macOS: |
|
On Linux output seems good too: |
|
Holy moly these AI bots on github are getting good, let implement the suggestions. |
dougsland
force-pushed
the
timeout-handle
branch
3 times, most recently
from
4ce2fee to
6ebcbde
Compare
|
I believe it's ready for review :) |
|
CI/CD error not related to the patch. |
|
@rhatdan PTAL |
ericcurtin
reviewed
Feb 1, 2025
ericcurtin
reviewed
Feb 1, 2025
|
We should check if the terminal actually supports emojis before printing them, this is how systemd does it: |
I will skip emoji for now I guess to make it less complicated. :( |
It would be something like this FWIW, if we get around to it: |
|
The emojis are fun, but want to make sure they work on Windows, Linux and Mac terminals. |
Thanks tonight I will double check and test it. |
Already tested on mac and linux, work great. Windows to be tested soon. |
Added a condition to check if the source model already exists, otherwise pull it down for the convert or push (convert and push were the only cli functions that used _get_source as far as I could tell) Signed-off-by: Kush Gupta <[email protected]>
Give the user a more descriptive error if getting a HTTP Error 404: Not Found Signed-off-by: Kush Gupta <[email protected]>
Update new error message for not finding an Ollama model in the convert test Signed-off-by: Kush Gupta <[email protected]>
Gives the user a clearer message on HTTP 404 Not Found Errors when trying and failing to pull from Ollama Signed-off-by: Kush Gupta <[email protected]>
Signed-off-by: Kush Gupta <[email protected]>
Signed-off-by: Kush Gupta <[email protected]>
dougsland
force-pushed
the
timeout-handle
branch
2 times, most recently
from
9a9dab5 to
2f44162
Compare
I think just copy the systemd technique, we won't be able to test every terminal, platform, combination, etc. systemd is deployed at crazy scale, they probably have it done as correctly as it can be within reason |
ericcurtin
reviewed
Feb 2, 2025
|
If we create a new file, we must add to install.sh , we have broken installs already because of this with gpu_detector |
ericcurtin
reviewed
Feb 2, 2025
ericcurtin
reviewed
Feb 2, 2025
ericcurtin
reviewed
Feb 2, 2025
- if timeout happens we try 5 times before sending Timeout error to users. - improve user experience when timeout occurs - Added console source tree for handling messages Resolves: containers#693 Signed-off-by: Douglas Schilling Landgraf <[email protected]>
dougsland
force-pushed
the
timeout-handle
branch
from
2f44162 to
fce28b6
Compare
|
macOS: |
same behaviour on Fedora 41 |
|
@ericcurtin ready for review, i just didn't follow why I need to change install.sh. Make install seems to work just nice. |
ericcurtin
requested changes
Feb 2, 2025
|
"make install" requires a "git clone", that's not a reasonable thing to expect a user to do, a developer yes, but not a user. The install.sh also handles brew, dnf, apt, etc. People are relying on this, especially macOS users. |
|
Bug reported and bug fix from lack of updating the install.sh file: |
rhatdan
reviewed
Feb 2, 2025
install.sh
Outdated
| local branch="${BRANCH:-s}" | ||
| local url="${host}/containers/ramalama/${branch}/bin/${from_file}" | ||
| local branch="main" | ||
| local url="${host}/kush-gupt/ramalama/${branch}/bin/${from_file}" |
|
This is all we need to do for emoji-support detection: I didn't invent this technique or anything, I just ported this from systemd, it's more tried and tested than anything any of us would handcraft. |
|
But the FORCE or emoji_enabled() code with the cached variable looks fine. |
ericcurtin
reviewed
Feb 2, 2025
|
|
||
| def is_locale_utf8(): | ||
| """Check if the system locale is UTF-8.""" | ||
| return 'UTF-8' in os.getenv('LC_CTYPE', '') or 'UTF-8' in os.getenv('LANG', '') |
There was a problem hiding this comment.
This code looks great, LC_ALL is the other or you could add if you wanted
@ericcurtin install.sh should be improved to avoid breaking things (automatic detect new files or other mechanism). and also added to CI/CD @ericcurtin. Regarding the windows code, everything has a start, we don't need to start big. I just added as mr. @rhatdan requested. If he request to added Windows support, we will make it fly. |
|
Let me close this one, I mess with my sync branch request, let me do a fresh PR and link this one. |
|
@dougsland our Windows solution is not native Windows it's WSL2, I understand make sure it works on Windows is open to interpretation, I was trying to clarify that, it's documented here: https://github.com/containers/ramalama/blob/main/docs/readme/wsl2-podman-cuda.md |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
if timeout happens we try 5 times before sending Timeout error to users.
improve user experience when timeout occurs
Add logging to handle message info/warn/error
Add MIT header
Resolves: #693
Summary by Sourcery
Enhancements: