Sherpa-Onnx xcFramework lacks support for Native visionOS Development #1946

carlosmbe · 2025-03-01T22:03:32Z

It would be really nice if Sherpa-Onnx could support native VisionOS Development with SwiftUI. At the moment, the current Sherpa-Onnx.xcframework does not contain the binaries to build a native app with the VisionOS SDK. However, it does let you build an iPad app that runs on VisionOS.

I've tried to modify the source code to implement this.

I was able to update the OnnxRuntime.xcframework by using this branch of OnnxRuntime and adding the change suggested at the very bottom of this discussion.

However, when running building Sherpa-Onnx for VisionOS, I ran into some challenges on the C side that are outside my scope. Specifically with the building.

Moreover, I had to update the toolchain to support VisionOS. This can be found in ./toolchains/ios.toolchain.cmake an updated version should be at toolchains repo here

Afterwards, I modified the build-ios.sh script to build for both the VisionOS Device and Simulator. My script kept failing at the linking stage.

Hopefully this context helps implement visionOS support later.

The text was updated successfully, but these errors were encountered:

csukuangfj · 2025-03-04T03:34:26Z

Sherpa-Onnx.xcframework does not contain the binaries to build a native app with the VisionOS SDK

We do provide vision os onnxruntime libs. Please see
https://github.com/csukuangfj/onnxruntime-libs/releases/download/v1.20.2/onnxruntime-vision_os-xros_arm64-1.20.2.zip

It should be straightforward to update

sherpa-onnx/build-ios.sh

Line 17 in c9d6859

    
           if [ ! -f $onnxruntime_dir/onnxruntime.xcframework/ios-arm64/onnxruntime.a ]; then

to support vision os.

Please describe in detail the difficulties you have.

We don't have a mac with apple chips so we cannot test sherpa-onnx with vision os support.

carlosmbe · 2025-03-04T06:11:36Z

Hello! Thanks for the response. I truly appreciate it.

I tried running my build script with the runtime you shared but still face the same issue.
Undefined symbols for architecture arm64:
....
....
ld: symbol(s) not found for architecture arm64 clang++: error: linker command failed with exit code 1 (use -v to see invocation)

I've refactored the build script to be as close to the original and still generate the error. It's in this gist

I'm using the following Toolchain

If there's any information I can share, I'd be more than happy to help.

This is a more detailed version of the error

  "_OrtGetApiBase", referenced from:
      ___cxx_global_var_init in sherpa-onnx-offline-speaker-diarization.cc.o
      ___cxx_global_var_init in libsherpa-onnx-core.a[70](online-stream.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[71](online-transducer-decoder.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[84](onnx-utils.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[89](provider.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[91](session.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[101](transpose.cc.o)
      ...
  "_OrtSessionOptionsAppendExecutionProvider_CoreML", referenced from:
      sherpa_onnx::GetSessionOptionsImpl(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, sherpa_onnx::ProviderConfig const*) in libsherpa-onnx-core.a[91](session.cc.o)
ld: symbol(s) not found for architecture arm64
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [bin/sherpa-onnx-offline-speaker-diarization.app/sherpa-onnx-offline-speaker-diarization] Error 1
make[1]: *** [sherpa-onnx/csrc/CMakeFiles/sherpa-onnx-offline-speaker-diarization.dir/all] Error 2
ld: warning: duplicate -rpath '@loader_path' ignored
ld: warning: ignoring duplicate libraries: '-lm'
Undefined symbols for architecture arm64:
  "_OrtGetApiBase", referenced from:
      ___cxx_global_var_init in sherpa-onnx-keyword-spotter.cc.o
      ___cxx_global_var_init in libsherpa-onnx-core.a[4](cat.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[11](hypothesis.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[12](keyword-spotter-impl.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[13](keyword-spotter.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[54](online-conformer-transducer-model.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[61](online-lstm-transducer-model.cc.o)
      ...
  "_OrtSessionOptionsAppendExecutionProvider_CoreML", referenced from:
      sherpa_onnx::GetSessionOptionsImpl(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, sherpa_onnx::ProviderConfig const*) in libsherpa-onnx-core.a[91](session.cc.o)
ld: symbol(s) not found for architecture arm64
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [bin/sherpa-onnx-keyword-spotter.app/sherpa-onnx-keyword-spotter] Error 1
make[1]: *** [sherpa-onnx/csrc/CMakeFiles/sherpa-onnx-keyword-spotter.dir/all] Error 2
Undefined symbols for architecture arm64:
  "_OrtGetApiBase", referenced from:
      ___cxx_global_var_init in sherpa-onnx.cc.o
      ___cxx_global_var_init in libsherpa-onnx-core.a[4](cat.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[11](hypothesis.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[54](online-conformer-transducer-model.cc.o)
      ___cxx_global_var_init.7 in libsherpa-onnx-core.a[56](online-ctc-fst-decoder.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[57](online-ctc-greedy-search-decoder.cc.o)
      ___cxx_global_var_init in libsherpa-onnx-core.a[58](online-ctc-model.cc.o)
      ...
  "_OrtSessionOptionsAppendExecutionProvider_CoreML", referenced from:
      sherpa_onnx::GetSessionOptionsImpl(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, sherpa_onnx::ProviderConfig const*) in libsherpa-onnx-core.a[91](session.cc.o)
ld: symbol(s) not found for architecture arm64
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [bin/sherpa-onnx.app/sherpa-onnx] Error 1
make[1]: *** [sherpa-onnx/csrc/CMakeFiles/sherpa-onnx.dir/all] Error 2
2 warnings generated ...

csukuangfj · 2025-03-04T06:19:49Z

This is a more detailed version of the error

Do you get the error within xcode or with build-ios.sh?

carlosmbe · 2025-03-04T06:22:13Z

With build-ios.sh

csukuangfj · 2025-03-04T06:24:27Z

I suggest that you try the toolchain file from
https://github.com/leetal/ios-cmake/blob/master/ios.toolchain.cmake

Remember to pass

-DPLATFORM=VISIONOS

or

-DPLATFORM=SIMULATOR_VISIONOS

to cmake.

Please delete the folder build-ios before you re-try.

carlosmbe · 2025-03-04T06:36:00Z

Alas, I already tried that but it still results in the same error. This is how build-ios.sh looks now https://gist.github.com/carlosmbe/9803658ff7f575a4916e56df5cbfa1e9

cmake \
  -DBUILD_PIPER_PHONMIZE_EXE=OFF \
  -DBUILD_PIPER_PHONMIZE_TESTS=OFF \
  -DBUILD_ESPEAK_NG_EXE=OFF \
  -DBUILD_ESPEAK_NG_TESTS=OFF \
  -S .. \
  -DCMAKE_TOOLCHAIN_FILE=./toolchains/ios.toolchain.cmake \
  -DPLATFORM=SIMULATOR_VISIONOS \
  -DENABLE_BITCODE=0 \
  -DENABLE_ARC=1 \
  -DENABLE_VISIBILITY=0 \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=./install \
  -DBUILD_SHARED_LIBS=OFF \
  -DSHERPA_ONNX_ENABLE_PYTHON=OFF \
  -DSHERPA_ONNX_ENABLE_TESTS=OFF \
  -DSHERPA_ONNX_ENABLE_CHECK=OFF \
  -DSHERPA_ONNX_ENABLE_PORTAUDIO=OFF \
  -DSHERPA_ONNX_ENABLE_JNI=OFF \
  -DSHERPA_ONNX_ENABLE_C_API=ON \
  -DSHERPA_ONNX_ENABLE_WEBSOCKET=OFF \
  -DDEPLOYMENT_TARGET=1.0 \
  -B build/simulator_arm64

cmake --build build/simulator_arm64 -j 14 --verbose

csukuangfj · 2025-03-04T07:08:23Z

From
https://gist.github.com/carlosmbe/9803658ff7f575a4916e56df5cbfa1e9#file-build-vision-sh-L18

Is /onnxruntime-vision_os-xros_arm64-${onnxruntime_version} not for simulator?

But you are using

-DPLATFORM=SIMULATOR_VISIONOS

Have you tried

-DPLATFORM=VISIONOS

Remember to delete build-ios before you re-try.

csukuangfj · 2025-03-04T07:12:35Z

Please see
#1957

I appreciate if you can finish it and create a new PR so that future users can build sherpa-onnx for vision os directly.

carlosmbe · 2025-03-04T23:35:41Z

Thanks for the branch. I've spent some time on it and I've identified that the linker issue above was caused by DBUILD_SHARED_LIBS=OFF so your change fixes that issue.

However, I tried building a xcFramework but got the following error:

error: unable to find any architecture information in the binary at '.../sherpa-onnx/build-vision-os/build/vision_os_arm64/lib/libsherpa-onnx.a': Unknown header: 0xb17c0de

Based on my research, this error code 0xb17c0de appears to mean "bitcode" and it appears that building for Release on visionOS/iOS/macOS results in bitcode being enabled automatically.

jrouwe/JoltPhysics#1261 (comment)
https://developer.apple.com/forums/thread/714324

Some have suggested disabling LTO should fix the issue. I tried adding the cmake flags to turn it off but that didn't work.

I was able to workaround this by building for Debug instead of Release. So I'm able to build the framework. Theoretically, this shouldn't be an issue as some claim that bitcode was depreciated in Xcode 14.

Xcode seems to have a few issues with the built framework though so I'm working on debugging that and will post an update on this Issue. I suspect that it's due to how I'm handling the .dylib files.

Here's another thread that I've read regarding the bitcode issue
facebook/facebook-ios-sdk#1582 (comment)

carlosmbe · 2025-03-05T00:37:34Z

Working build-vision-os.sh for visionOS Devices can be found on this fork https://github.com/carlosmbe/sherpa-onnx-vision-build/tree/vision-os

Here is the gist of the .sh file https://gist.github.com/carlosmbe/3328dc72ec0cc32ce9fd8ec779d543e8

csukuangfj · 2025-03-05T00:47:49Z

Thanks! By the way, libcargs can be deleted. You don't need it.

carlosmbe · 2025-03-05T01:57:34Z

Thanks for the info and all of the help!

I've also added a script for building the simulator version of the script. A few things to note for using this vision build:

It requires a plist to be in the framework
Since this is a debug build of the framework, you'll have to go to Targets->Your Project -> Build Phases ->Link Binary With Library and set the framework's status to optional. Stack overflow I referenced -> Link

https://github.com/carlosmbe/sherpa-onnx-vision-build/blob/vision-os/build-vision-os-sim.sh

csukuangfj linked a pull request Mar 4, 2025 that will close this issue

[Not for merge]: Support building for visionOS #1957

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sherpa-Onnx xcFramework lacks support for Native visionOS Development #1946

Sherpa-Onnx xcFramework lacks support for Native visionOS Development #1946

carlosmbe commented Mar 1, 2025

csukuangfj commented Mar 4, 2025

carlosmbe commented Mar 4, 2025

csukuangfj commented Mar 4, 2025

carlosmbe commented Mar 4, 2025

csukuangfj commented Mar 4, 2025

carlosmbe commented Mar 4, 2025

csukuangfj commented Mar 4, 2025

csukuangfj commented Mar 4, 2025

carlosmbe commented Mar 4, 2025

carlosmbe commented Mar 5, 2025

csukuangfj commented Mar 5, 2025

carlosmbe commented Mar 5, 2025

Sherpa-Onnx xcFramework lacks support for Native visionOS Development #1946

Sherpa-Onnx xcFramework lacks support for Native visionOS Development #1946

Comments

carlosmbe commented Mar 1, 2025

csukuangfj commented Mar 4, 2025

carlosmbe commented Mar 4, 2025

csukuangfj commented Mar 4, 2025

carlosmbe commented Mar 4, 2025

csukuangfj commented Mar 4, 2025

carlosmbe commented Mar 4, 2025

csukuangfj commented Mar 4, 2025

csukuangfj commented Mar 4, 2025

carlosmbe commented Mar 4, 2025

carlosmbe commented Mar 5, 2025

csukuangfj commented Mar 5, 2025

carlosmbe commented Mar 5, 2025