build debug tensorflow failed

[south-ocean]

Issue Type

Build/Install

Source

source

Tensorflow Version

tf2.7

Custom Code

Yes

OS Platform and Distribution

linux

Mobile device

No response

Python version

python3.7.12

Bazel version

3.7.2

GCC/Compiler version

7.3.1

CUDA/cuDNN version

rocm5.1

GPU model and memory

No response

Current Behaviour?

A error happened!

Standalone code to reproduce the issue

when i complie the tensorflow of debug version , it failed  Linking of rule '//tensorflow/python:_pywrap_tensorflow_internal.so' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
the compliled command is 
bazel build -j 128 -c dbg --copt="-O0" --cxxopt="-O0"  --experimental_repository_cache=/home/tensorflow/Depend/v1 --config=rocm //tensorflow/tools/pip_package:build_pip_package --verbose_failures

Relevant log output

bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ScaledNumber.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 188 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ScaledNumber.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ScopedPrinter.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 81 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ScopedPrinter.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Signals.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 726 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Signals.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Signposts.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 40 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Signposts.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SmallPtrSet.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 85 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SmallPtrSet.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SmallVector.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 54 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SmallVector.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SourceMgr.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 769 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SourceMgr.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SpecialCaseList.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 646 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SpecialCaseList.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Statistic.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 628 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Statistic.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/StringExtras.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 92 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/StringExtras.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/StringMap.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 74 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/StringMap.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/StringRef.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 441 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/StringRef.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/StringSaver.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 174 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/StringSaver.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SuffixTree.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 339 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SuffixTree.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SymbolRemappingReader.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 256 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SymbolRemappingReader.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SystemUtils.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 25 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/SystemUtils.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TarWriter.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 254 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TarWriter.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TargetParser.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 178 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TargetParser.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TargetRegistry.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 201 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TargetRegistry.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ThreadLocal.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 59 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ThreadLocal.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ThreadPool.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 687 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ThreadPool.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Threading.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 135 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Threading.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TimeProfiler.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 596 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TimeProfiler.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Timer.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 765 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Timer.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ToolOutputFile.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 93 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/ToolOutputFile.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TrigramIndex.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 310 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TrigramIndex.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Triple.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 370 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Triple.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Twine.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 84 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Twine.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TypeSize.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 176 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/TypeSize.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Unicode.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 59 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Unicode.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/UnicodeCaseFold.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 5 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/UnicodeCaseFold.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Valgrind.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 5 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Valgrind.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/VersionTuple.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 55 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/VersionTuple.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/VirtualFileSystem.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 2781 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/VirtualFileSystem.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Watchdog.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 5 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Watchdog.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/WithColor.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 418 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/WithColor.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/X86TargetParser.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 475 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/X86TargetParser.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/YAMLParser.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 931 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/YAMLParser.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/YAMLTraits.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 1137 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/YAMLTraits.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Z3Solver.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 50 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/Z3Solver.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/circular_raw_ostream.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 36 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/circular_raw_ostream.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/raw_os_ostream.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 29 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/raw_os_ostream.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/raw_ostream.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 640 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/raw_ostream.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regcomp.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 47 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regcomp.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regerror.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 9 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regerror.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regexec.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 17 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regexec.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regfree.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 5 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regfree.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regstrlcpy.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 5 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/regstrlcpy.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/xxhash.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 56 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Support/xxhash.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/Demangle.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 10 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/Demangle.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/ItaniumDemangle.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 1870 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/ItaniumDemangle.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/MicrosoftDemangle.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 279 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/MicrosoftDemangle.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/MicrosoftDemangleNodes.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 188 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/MicrosoftDemangleNodes.pic.o
bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/RustDemangle.pic.o(.debug_aranges+0x6): error: relocation overflow: reference to local symbol 107 in bazel-out/k8-dbg/bin/external/llvm-project/llvm/_objs/Demangle/RustDemangle.pic.o
collect2: error: ld returned 1 exit status
Target //tensorflow/tools/pip_package:build_pip_package failed to build
ERROR: /home/tensorflow/tensorflow/lite/toco/python/BUILD:92:10 Linking of rule '//tensorflow/python:_pywrap_tensorflow_internal.so' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command
  (cd /root/.cache/bazel/_bazel_root/98bd3f3dfbd0725c619a9932413c587d/execroot/org_tensorflow && \
  exec env - \
    LD_LIBRARY_PATH=/opt/dtk-22.04/hip/lib:/opt/dtk-22.04/llvm/lib:/opt/dtk-22.04/lib:/opt/dtk-22.04/lib64:/opt/rh/devtoolset-7/root/usr/lib64:/opt/rh/devtoolset-7/root/usr/lib:/opt/rh/devtoolset-7/root/usr/lib64/dyninst:/opt/rh/devtoolset-7/root/usr/lib/dyninst:/opt/rh/devtoolset-7/root/usr/lib64:/opt/rh/devtoolset-7/root/usr/lib \
    PATH=/opt/rh/devtoolset-7/root/usr/bin:/usr/local/python3.7.12/bin:/home/tensorflow/Depend/bin:/opt/dtk-22.04/bin:/opt/dtk-22.04/llvm/bin:/opt/dtk-22.04/hip/bin:/root/miniconda3/bin:/root/miniconda3/condabin:/opt/git-2.30.0/bin:/opt/cmake-3.19.3-Linux-x86_64/bin:/opt/rh/devtoolset-7/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/local/python3.7.12/bin/python3 \
    PYTHON_LIB_PATH=/usr/local/python3.7.12/lib/python3.7/site-packages \
    ROCBLAS_TENSILE_LIBPATH=/opt/dtk-22.04/lib/library \
    ROCM_PATH=/opt/dtk-22.04 \
    TF2_BEHAVIOR=1 \
  external/local_config_rocm/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc @bazel-out/k8-dbg/bin/tensorflow/python/_pywrap_tensorflow_internal.so-2.params)
Execution platform: @local_execution_config_platform//:platform
INFO: Elapsed time: 291.473s, Critical Path: 289.93s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

[ppanchad-amd]

Hi @south-ocean. Sorry for the lack of response. Can you please test if your issue still occurs with the latest ROCm? If not, please close the ticket. Thanks!