Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do I know which ONNX transformation models are available? (Errors when loading models with CDN) #1194

Open
mz-imhj opened this issue Feb 15, 2025 · 1 comment
Labels
question Further information is requested

Comments

@mz-imhj
Copy link

mz-imhj commented Feb 15, 2025

Question

I am using a CDN to load the models, as shown in the code below.
I filtered the models in HuggingFace the way you recommend (text-generation, transformers.js) and put the id of the model I looked up. As I understand it, to change the model, I only need to change the model id.
However, I get an error for each of the below models.

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'model')

  • HuggingFaceTB/SmolLM2-135M-Instruct
  • Xenova/codegen-350M-mono
    ...

Uncaught (in promise) Error: Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: Deserialize tensor model.layers.4.mlp.gate_proj.MatMul.weight_Q4 failed.Failed to load external data file ""model_q4f16.onnx_data"", error: Module.MountedFiles is not available.

  • onnx-community/Phi-3.5-mini-instruct-onnx-web
    ...

I'm ultimately saying that I don't know what model will be available.
Additionally, I was wondering if there is a way to know 'in advance' which 'dtype' and 'device' can be supported.

  import { pipeline } from 'https://cdn.jsdelivr.net/npm/@huggingface/[email protected]';

    generator = await pipeline('text-generation', 'onnx-community/DeepSeek-R1-Distill-Qwen-1.5B-ONNX', {
      dtype: "auto",
      device: "auto",
    });
@mz-imhj mz-imhj added the question Further information is requested label Feb 15, 2025
@benc-uk
Copy link

benc-uk commented Feb 16, 2025

Duplicate of this issue #963 which I've had open since Oct last last year

AFAIK there's a bug in how transformers locates or fetches these external data files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants