Skip to content

Commit

Permalink
Improve error message
Browse files Browse the repository at this point in the history
Summary:

Example error message:
```
couldn't load runner/third-party/tokenizers/test/resources/test_sentencepiece.model.
Error message:
model file path should not be empty.
It is likely that the tokenizer artifact is broken or of a different format.
```

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
  • Loading branch information
larryliu0820 committed Dec 27, 2024
1 parent 08e3cfa commit 479a90e
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions src/sentencepiece.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,10 @@ Error SPTokenizer::load(const std::string &tokenizer_path) {
const auto status = _processor->Load(tokenizer_path);
if (!status.ok()) {
fprintf(stderr,
"couldn't load %s\n. It is likely that the tokenizer artifact is "
"couldn't load %s. \nError message: \n%s\n"
"It is likely that the tokenizer artifact is "
"broken or of a different format.",
tokenizer_path.c_str());
tokenizer_path.c_str(), status.error_message());
return Error::LoadFailure;
}
// load vocab_size, bos_tok, eos_tok
Expand Down

0 comments on commit 479a90e

Please sign in to comment.