[QST] Encountering rapids_make_logger or Failed to clone repository: '[email protected]:rapidsai/cumlprims_mg.git' error during build

[mindok7520]

Hello,

I am encountering an error with rapids_make_logger during the build process.

(cuml_dev) [root@f747f7fdf4e2 cuml]# ./build.sh libcuml
Building for the architecture of the GPU in the system...
CMake Error at CMakeLists.txt:238 (rapids_make_logger):
  Unknown CMake command "rapids_make_logger".


-- Configuring incomplete, errors occurred!

Below is the script I am using to build the project:

#!/usr/bin/env bash
set -e

################################################################################
# 1. path & env settings
################################################################################

ROOT_FOLDER=$(dirname $(realpath -s $0))

CUML_SOURCE_FOLDER=${ROOT_FOLDER}/deps/cuml
CUML_BUILD_FOLDER=${ROOT_FOLDER}/deps/cuml/build/Release

CUML_CONFIG=( # branch_name
    "branch-23.06"
)

CUML_config=${CUML_CONFIG[0]}
IFS=" " read -r branch_name <<< "${CUML_config}"

echo "=================================================="
echo "[cuML] branch: $branch_name"
echo "=================================================="

################################################################################
# 2. Source clone
################################################################################

mkdir -p ${CUML_SOURCE_FOLDER}

cd ${CUML_SOURCE_FOLDER}
git clone https://github.com/rapidsai/cuml.git .
git checkout ${branch_name}

################################################################################
# 3. make build dir
################################################################################

mkdir -p "${CUML_BUILD_FOLDER}"
cd "${CUML_BUILD_FOLDER}"

################################################################################
# 4. CMake settings
################################################################################

cmake -G Ninja "${CUML_SOURCE_FOLDER}/cpp" \
    -DCMAKE_BUILD_TYPE=Release \
    -DBUILD_CUML_MPI_COMMS=ON \
    -DDETECT_CONDA_ENV=OFF \
    -DCMAKE_PREFIX_PATH=.

################################################################################
# 5. build
################################################################################

cmake --build .

# (option) install
# cmake --install .

When I follow this installation method, I encounter the following error. I have tried several branches such as 23.06, 25.04, etc., but the same error occurs. I have also tried building on Oracle Linux 9 and Rocky Linux 8, but the error persists.

Both building from source and using the conda environment result in the same error. Has anyone else encountered this issue and found a solution?

[mindok7520]

branch-25.04 shows a logger-related error, and 24.12, 23.06 show the following error.

-- CPM: raft: adding package [email protected] (branch-23.06)
-- RMM: RMM_LOGGING_LEVEL = 'INFO'
-- CPM: raft: rmm: adding package [email protected] (9.1.0)
-- Module support is disabled.
-- Version: 9.1.0
-- Build type: Release
-- CXX_STANDARD: 11
-- Performing Test has_std_11_flag
-- Performing Test has_std_11_flag - Success
-- Performing Test has_std_0x_flag
-- Performing Test has_std_0x_flag - Success
-- Required features: cxx_variadic_templates
-- CPM: raft: rmm: adding package [email protected] (v1.11.0)
-- Build spdlog: 1.11.0
-- Build type: Release
-- Generating install
-- CPM: raft: adding package [email protected] (v2.10.0)
-- CMake Version: 3.26.5
-- CUDART: /usr/local/cuda-11.8/lib64/libcudart.so
-- CUDA Driver: /usr/local/cuda-11.8/lib64/stubs/libcuda.so
-- NVRTC: /usr/local/cuda-11.8/lib64/libnvrtc.so
-- Default Install Location: install
-- CUDA Compilation Architectures: 53;60;61;70;72;75;80;86
-- Enable caching of reference results in conv unit tests
-- Enable rigorous conv problem sizes in conv unit tests
-- Using NVCC flags: -DCUTLASS_TEST_LEVEL=0;-DCUTLASS_TEST_ENABLE_CACHED_RESULTS=1;-DCUTLASS_CONV_UNIT_TEST_RIGOROUS_SIZE_ENABLED=1;-DCUTLASS_DEBUG_TRACE_LEVEL=0;$<$<BOOL:1>:-Xcompiler=-Wconversion>;$<$<BOOL:1>:-Xcompiler=-fno-strict-aliasing>
-- CUTLASS Revision: 0c2a10353
-- Configuring cublas ...
-- cuBLAS Disabled.
-- Configuring cuBLAS ... done.
-- Configuring cuDNN ...
-- cuDNN not found.
-- Configuring cuDNN ... done.
-- CPM: raft: adding package [email protected] (546ca606a17f480fa9d58d1752cce2aad6575bc4)
-- CPM: raft: cuco: adding package [email protected] (branch/1.9.1)
-- Found libcudacxx: /deps/cuml/cpp/build/_deps/libcudacxx-src/lib/cmake/libcudacxx/libcudacxx-config.cmake (found version "1.9.1.0") 
-- rapids-cmake [libcudacxx]: applied diff install_rules.diff to fix issue: 'libcudacxx 1.X installs incorrect files [https://github.com/NVIDIA/libcudacxx/pull/428]'
-- CPM: adding package [email protected] (ee697b3b58d5f51623dd3b308f290581b58dbe5d)
-- The C compiler identification is GNU 11.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- No Conda environment detected
-- Found fmtlib at $<BUILD_INTERFACE:deps/cuml/cpp/build/_deps/fmt-src/include>;$<INSTALL_INTERFACE:include>
-- RapidJSON found. Headers: /workspace/deps_build/include
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Treelite VERSION: 3.2.0
-- CPM: adding package [email protected] (78e3548cda5e7c263d125e8c10e733ebf2c4ebbd)
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) (Required is at least version "1.8.12")
add_doxygen_target: doxygen exe not found
-- CPM: adding package [email protected] (branch-23.06)
[ 11%] Creating directories for 'cumlprims_mg-populate'
[ 22%] Performing download step (git clone) for 'cumlprims_mg-populate'
Cloning into 'cumlprims_mg-src'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Cloning into 'cumlprims_mg-src'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Cloning into 'cumlprims_mg-src'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
-- Had to git clone more than once: 3 times.
CMake Error at cumlprims_mg-subbuild/cumlprims_mg-populate-prefix/tmp/cumlprims_mg-populate-gitclone.cmake:39 (message):
  Failed to clone repository: '[email protected]:rapidsai/cumlprims_mg.git'


gmake[2]: *** [CMakeFiles/cumlprims_mg-populate.dir/build.make:102: cumlprims_mg-populate-prefix/src/cumlprims_mg-populate-stamp/cumlprims_mg-populate-download] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/cumlprims_mg-populate.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2

CMake Error at /usr/share/cmake/Modules/FetchContent.cmake:1622 (message):
  Build step for cumlprims_mg failed: 2
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FetchContent.cmake:1762:EVAL:2 (__FetchContent_directPopulate)
  /usr/share/cmake/Modules/FetchContent.cmake:1762 (cmake_language)
  build/cmake/CPM_0.35.6.cmake:963 (FetchContent_Populate)
  build/cmake/CPM_0.35.6.cmake:773 (cpm_fetch_package)
  build/cmake/CPM_0.35.6.cmake:299 (CPMAddPackage)
  build/_deps/rapids-cmake-src/rapids-cmake/cpm/find.cmake:167 (CPMFindPackage)
  cmake/thirdparty/get_cumlprims_mg.cmake:38 (rapids_cpm_find)
  cmake/thirdparty/get_cumlprims_mg.cmake:65 (find_and_configure_cumlprims_mg)
  CMakeLists.txt:239 (include)

[mindok7520]

Currently, the error is avoided using the following options:

 -DENABLE_CUMLPRIMS_MG=OFF \
 -DSINGLEGPU=ON \
 -DBUILD_CUML_STD_COMMS=OFF 

[vyasr]

Hi @mindok7520, sorry for the churn here. There were some upstream issues in other repositories affecting 25.04 branch builds yesterday. If you clean your build directory and pull the latest 25.04 you should be able to build again.

For cumlprims_mg, the problem is that is a private repository and CMake is trying to fetch it for you during the build but you don't have access. How are you setting up your environment for the build? If you are using conda, you can install all dependencies of cuml that way (we provide convenient environment files here: https://github.com/rapidsai/cuml/tree/branch-25.04/conda/environments). If you were already doing that, what you probably ran into when trying to build older branches of cuml was that you need an older version of cumlprims_mg. You could create a suitable environment by using the environment yaml files from that branch (which would install the appropriate dependencies).

[mindok7520]

Hello @vyasr , I checked and now I can build using conda.
But what I want is to build directly from source without using conda.
Will that ever change?

[vyasr]

You certainly do not have to use conda to set up your build environment. However, if you do not use conda then you will have to install the dependencies yourself and ensure that you have suitable compilers. We do not maintain installation instructions for e.g. specific Linux distros, so if you want to install from source without conda you will have to do some work to find the appropriate system packages to install etc.