Error in vectorized calls to OpenAI API

[v4nn4]

Describe the issue as clearly as possible:

Vectorized calls to OpenAI API are throwing an error.

Steps/code to reproduce the bug:

import outlines
from pydantic import BaseModel


class Model(BaseModel):
    name: str

    class Config:
        extra = "forbid"

model = models.openai("gpt-4o-mini")
generator = outlines.generate.json(model, Model)
results = generator(["This is a first test", "This is a second test"])  # fails
print(len(results))

Expected result:

2

Error message:

Traceback (most recent call last):
  File "/Users/xxx/Documents/Repos/xxx/main.py", line 183, in <module>
    fire.Fire(DuneApp)
  File "/Users/xxx/Library/Caches/pypoetry/virtualenvs/xxx-DgNc0zAM-py3.11/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Library/Caches/pypoetry/virtualenvs/xxx-DgNc0zAM-py3.11/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Library/Caches/pypoetry/virtualenvs/xxx-DgNc0zAM-py3.11/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Documents/Repos/xxx/main.py", line 76, in test
    results = generator(["This is a first test", "This is a second test"])  # fails
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Library/Caches/pypoetry/virtualenvs/xxx-DgNc0zAM-py3.11/lib/python3.11/site-packages/outlines/models/openai.py", line 151, in __call__
    return self.format_sequence(response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Library/Caches/pypoetry/virtualenvs/xxx-DgNc0zAM-py3.11/lib/python3.11/site-packages/outlines/generate/json.py", line 90, in <lambda>
    format_sequence = lambda x: schema_object.parse_raw(x)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Library/Caches/pypoetry/virtualenvs/xxx-DgNc0zAM-py3.11/lib/python3.11/site-packages/pydantic/main.py", line 1275, in parse_raw
    raise pydantic_core.ValidationError.from_exception_data(cls.__name__, [error])
pydantic_core._pydantic_core.ValidationError: 1 validation error for Model
__root__
  the JSON object must be str, bytes or bytearray, not ndarray [type=type_error, input_value=array(['{"name":"Test Suc...}'],
      dtype='<U31'), input_type=ndarray]

Outlines/Python version information:

0.1.13
Python 3.11.11 (main, Dec 3 2024, 17:20:40) [Clang 16.0.0 (clang-1600.0.26.4)]

accelerate==1.3.0 airportsdata==20241001 annotated-types==0.7.0 anyio==4.8.0 appnope==0.1.4 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 arrow==1.3.0 asgiref==3.8.1 asttokens==3.0.0 async-lru==2.0.4 attrs==24.3.0 autoawq==0.2.7.post3 babel==2.16.0 backoff==2.2.1 bcrypt==4.2.1 beautifulsoup4==4.12.3 bleach==6.2.0 blis==1.2.0 bokeh==3.6.2 bs4==0.0.2 build==1.2.2.post1 cachetools==5.5.1 catalogue==2.0.10 certifi==2024.12.14 cffi==1.17.1 charset-normalizer==3.4.1 chroma-hnswlib==0.7.6 chromadb==0.6.3 click==8.1.8 cloudpathlib==0.20.0 cloudpickle==3.1.1 colorcet==3.1.0 coloredlogs==15.0.1 comm==0.2.2 confection==0.1.5 contourpy==1.3.1 cycler==0.12.1 cymem==2.0.11 dask==2025.1.0 datasets==3.2.0 datashader==0.16.3 debugpy==1.8.12 decorator==5.1.1 defusedxml==0.7.1 Deprecated==1.2.15 dill==0.3.8 diskcache==5.6.3 distro==1.9.0 durationpy==0.9 EbookLib==0.18 executing==2.2.0 fastapi==0.115.6 fastjsonschema==2.21.1 filelock==3.17.0 fire==0.7.0 flatbuffers==25.1.21 fonttools==4.55.4 fqdn==1.5.1 fsspec==2024.12.0 google-auth==2.37.0 googleapis-common-protos==1.66.0 grpcio==1.69.0 h11==0.14.0 holoviews==1.20.0 httpcore==1.0.7 httptools==0.6.4 httpx==0.28.1 huggingface-hub==0.27.1 humanfriendly==10.0 idna==3.10 imageio==2.37.0 importlib_metadata==8.5.0 importlib_resources==6.5.2 iniconfig==2.0.0 interegular==0.3.3 ipykernel==6.29.5 ipython==8.31.0 ipywidgets==8.1.5 isoduration==20.11.0 jedi==0.19.2 Jinja2==3.1.5 jiter==0.8.2 joblib==1.4.2 json5==0.10.0 jsonpointer==3.0.0 jsonschema==4.23.0 jsonschema-specifications==2024.10.1 jupyter==1.1.1 jupyter-console==6.6.3 jupyter-events==0.11.0 jupyter-lsp==2.2.5 jupyter_client==8.6.3 jupyter_core==5.7.2 jupyter_server==2.15.0 jupyter_server_terminals==0.5.3 jupyterlab==4.3.4 jupyterlab_pygments==0.3.0 jupyterlab_server==2.27.3 jupyterlab_widgets==3.0.13 kiwisolver==1.4.8 kubernetes==31.0.0 kuzu==0.7.1 langcodes==3.5.0 language_data==1.3.0 lark==1.2.2 lazy_loader==0.4 linkify-it-py==2.0.3 llvmlite==0.44.0 locket==1.0.0 lxml==5.3.0 marisa-trie==1.2.1 Markdown==3.7 markdown-it-py==3.0.0 MarkupSafe==3.0.2 matplotlib==3.10.0 matplotlib-inline==0.1.7 mdit-py-plugins==0.4.2 mdurl==0.1.2 mistune==3.1.0 mmh3==5.0.1 monotonic==1.6 mpmath==1.3.0 multipledispatch==1.0.0 multiprocess==0.70.16 murmurhash==1.0.12 nbclient==0.10.2 nbconvert==7.16.5 nbformat==5.10.4 nest-asyncio==1.6.0 networkx==3.4.2 notebook==7.3.2 notebook_shim==0.2.4 numba==0.61.0 numpy==2.2.2 oauthlib==3.2.2 onnxruntime==1.20.1 openai==1.60.0 opentelemetry-api==1.29.0 opentelemetry-exporter-otlp-proto-common==1.29.0 opentelemetry-exporter-otlp-proto-grpc==1.29.0 opentelemetry-instrumentation==0.50b0 opentelemetry-instrumentation-asgi==0.50b0 opentelemetry-instrumentation-fastapi==0.50b0 opentelemetry-proto==1.29.0 opentelemetry-sdk==1.29.0 opentelemetry-semantic-conventions==0.50b0 opentelemetry-util-http==0.50b0 orjson==3.10.15 outlines==0.1.13 outlines_core==0.1.26 overrides==7.7.0 packaging==24.2 pandas==2.2.3 pandocfilters==1.5.1 panel==1.5.5 param==2.2.0 parso==0.8.4 partd==1.4.2 pexpect==4.9.0 pillow==11.1.0 platformdirs==4.3.6 pluggy==1.5.0 posthog==3.9.2 preshed==3.0.9 prometheus_client==0.21.1 prompt_toolkit==3.0.50 protobuf==5.29.3 psutil==6.1.1 ptyprocess==0.7.0 pure_eval==0.2.3 pyarrow==18.1.0 pyasn1==0.6.1 pyasn1_modules==0.4.1 pycountry==24.6.1 pycparser==2.22 pyct==0.5.0 pydantic==2.10.5 pydantic_core==2.27.2 Pygments==2.19.1 pynndescent==0.5.13 pyparsing==3.2.1 PyPika==0.48.9 pyproject_hooks==1.2.0 pytest==8.3.4 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 python-json-logger==3.2.1 pytz==2024.2 pyviz_comms==3.0.4 PyYAML==6.0.2 pyzmq==26.2.0 referencing==0.36.1 regex==2024.11.6 requests==2.32.3 requests-oauthlib==2.0.0 rfc3339-validator==0.1.4 rfc3986-validator==0.1.1 rich==13.9.4 rpds-py==0.22.3 rsa==4.9 safetensors==0.5.2 scikit-image==0.25.0 scikit-learn==1.6.1 scipy==1.15.1 seaborn==0.13.2 Send2Trash==1.8.3 shellingham==1.5.4 six==1.17.0 smart-open==7.1.0 sniffio==1.3.1 soupsieve==2.6 spacy==3.8.4 spacy-legacy==3.0.12 spacy-loggers==1.0.5 srsly==2.5.1 stack-data==0.6.3 starlette==0.41.3 sympy==1.13.1 tenacity==9.0.0 termcolor==2.5.0 terminado==0.18.1 thinc==8.3.4 threadpoolctl==3.5.0 tifffile==2025.1.10 tinycss2==1.4.0 tokenizers==0.21.0 toolz==1.0.0 torch==2.5.1 tornado==6.4.2 tqdm==4.67.1 traitlets==5.14.3 transformers==4.48.1 typer==0.15.1 types-python-dateutil==2.9.0.20241206 typing_extensions==4.12.2 tzdata==2025.1 uc-micro-py==1.0.3 umap-learn==0.5.7 uri-template==1.3.0 urllib3==2.3.0 uvicorn==0.34.0 uvloop==0.21.0 wasabi==1.1.3 watchfiles==1.0.4 wcwidth==0.2.13 weasel==0.4.1 webcolors==24.11.1 webencodings==0.5.1 websocket-client==1.8.0 websockets==14.2 widgetsnbextension==4.0.13 wrapt==1.17.2 xarray==2025.1.1 xxhash==3.5.0 xyzservices==2025.1.0 zipp==3.21.0 zstandard==0.23.0

Context for the issue:

Vectorized calls allow batching multiple prompts into a single API request, which is more cost-effective and faster than making multiple individual requests. Without working vectorized calls, users are forced to iterate through prompts sequentially, which increases API latency and usage costs.