Skip to content

Use natively hip_bfloat16 as a member of BFloat16 class to take advan… #3114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
anatoliylitv wants to merge 1 commit into
base: release/2.9
Choose a base branch
from
+127 −96

Use natively hip_bfloat16 as a member of BFloat16 class to take advan…

70a53bc
Select commit
Loading
Failed to load commit list.
Draft

Use natively hip_bfloat16 as a member of BFloat16 class to take advan… #3114

Use natively hip_bfloat16 as a member of BFloat16 class to take advan…
70a53bc
Select commit
Loading
Failed to load commit list.
ROCm Repo Management API / Jenkins failed Mar 31, 2026 in 39m 32s

Initialize/Build PyTorch: error in 'error' step

Initialize / Build PyTorch / Shell Script

Error in sh step, with arguments #!/usr/bin/bash set -o pipefail ./build_pytorch.sh 2>&1 | tee build_pytorch.log .

script returned exit code 1
Build log 
Build log truncated.

[2026-03-31T14:30:38.998Z]    79 |   init();
[2026-03-31T14:30:38.998Z]       |   ^
[2026-03-31T14:30:38.998Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:38.998Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:38.998Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:86:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:38.998Z]    86 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:38.998Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:38.998Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeReduce<gloo::float16, gloo::HipDevicePointer<gloo::float16>>::HipLocalNativeReduce' requested here
[2026-03-31T14:30:38.998Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:38.998Z]       |                                  ^
[2026-03-31T14:30:38.998Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:48:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeReduce<gloo::float16, gloo::HipDevicePointer<gloo::float16>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<gloo::float16>> &, gloo::HipDevicePointer<gloo::float16> &, const gloo::HipReductionFunction<gloo::float16> *&, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:38.998Z]    48 |   return make_unique<HipLocalNativeReduce<T, Dst>>(
[2026-03-31T14:30:38.998Z]       |          ^
[2026-03-31T14:30:38.998Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:405:22: note: in instantiation of function template specialization 'gloo::cudaDeviceReduce<gloo::float16, gloo::HipDevicePointer<gloo::float16>>' requested here
[2026-03-31T14:30:38.998Z]   405 |     localReduceOp_ = cudaDeviceReduce(
[2026-03-31T14:30:38.998Z]       |                      ^
[2026-03-31T14:30:38.998Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<gloo::float16, gloo::HipDeviceWorkspace<gloo::float16>>::init<gloo::HipDeviceWorkspace<gloo::float16>>' requested here
[2026-03-31T14:30:38.998Z]    79 |   init();
[2026-03-31T14:30:38.998Z]       |   ^
[2026-03-31T14:30:38.998Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:38.998Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:38.998Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:199:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:38.998Z]   199 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:38.998Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:38.998Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeBroadcast<gloo::float16, gloo::HipDevicePointer<gloo::float16>>::HipLocalNativeBroadcast' requested here
[2026-03-31T14:30:38.998Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:38.998Z]       |                                  ^
[2026-03-31T14:30:38.998Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:72:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeBroadcast<gloo::float16, gloo::HipDevicePointer<gloo::float16>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<gloo::float16>> &, gloo::HipDevicePointer<gloo::float16> &, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:38.998Z]    72 |   return make_unique<HipLocalNativeBroadcast<T, Src>>(
[2026-03-31T14:30:38.998Z]       |          ^
[2026-03-31T14:30:38.998Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:408:9: note: in instantiation of function template specialization 'gloo::cudaDeviceBroadcast<gloo::float16, gloo::HipDevicePointer<gloo::float16>>' requested here
[2026-03-31T14:30:38.999Z]   408 |         cudaDeviceBroadcast(streams_, devicePtrs_, scratch_, 0, totalNumElems_);
[2026-03-31T14:30:38.999Z]       |         ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<gloo::float16, gloo::HipDeviceWorkspace<gloo::float16>>::init<gloo::HipDeviceWorkspace<gloo::float16>>' requested here
[2026-03-31T14:30:38.999Z]    79 |   init();
[2026-03-31T14:30:38.999Z]       |   ^
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:86:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:38.999Z]    86 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:38.999Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:38.999Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeReduce<c10::BFloat16, gloo::HipHostPointer<c10::BFloat16>>::HipLocalNativeReduce' requested here
[2026-03-31T14:30:38.999Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:38.999Z]       |                                  ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:48:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeReduce<c10::BFloat16, gloo::HipHostPointer<c10::BFloat16>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<c10::BFloat16>> &, gloo::HipHostPointer<c10::BFloat16> &, const gloo::HipReductionFunction<c10::BFloat16> *&, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:38.999Z]    48 |   return make_unique<HipLocalNativeReduce<T, Dst>>(
[2026-03-31T14:30:38.999Z]       |          ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:380:22: note: in instantiation of function template specialization 'gloo::cudaDeviceReduce<c10::BFloat16, gloo::HipHostPointer<c10::BFloat16>>' requested here
[2026-03-31T14:30:38.999Z]   380 |     localReduceOp_ = cudaDeviceReduce(
[2026-03-31T14:30:38.999Z]       |                      ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<c10::BFloat16>::init<gloo::HipHostWorkspace<c10::BFloat16>>' requested here
[2026-03-31T14:30:38.999Z]    79 |   init();
[2026-03-31T14:30:38.999Z]       |   ^
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:199:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:38.999Z]   199 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:38.999Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:38.999Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeBroadcast<c10::BFloat16, gloo::HipHostPointer<c10::BFloat16>>::HipLocalNativeBroadcast' requested here
[2026-03-31T14:30:38.999Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:38.999Z]       |                                  ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:72:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeBroadcast<c10::BFloat16, gloo::HipHostPointer<c10::BFloat16>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<c10::BFloat16>> &, gloo::HipHostPointer<c10::BFloat16> &, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:38.999Z]    72 |   return make_unique<HipLocalNativeBroadcast<T, Src>>(
[2026-03-31T14:30:38.999Z]       |          ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:383:9: note: in instantiation of function template specialization 'gloo::cudaDeviceBroadcast<c10::BFloat16, gloo::HipHostPointer<c10::BFloat16>>' requested here
[2026-03-31T14:30:38.999Z]   383 |         cudaDeviceBroadcast(streams_, devicePtrs_, scratch_, 0, totalNumElems_);
[2026-03-31T14:30:38.999Z]       |         ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<c10::BFloat16>::init<gloo::HipHostWorkspace<c10::BFloat16>>' requested here
[2026-03-31T14:30:38.999Z]    79 |   init();
[2026-03-31T14:30:38.999Z]       |   ^
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:86:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:38.999Z]    86 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:38.999Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:38.999Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeReduce<c10::BFloat16, gloo::HipDevicePointer<c10::BFloat16>>::HipLocalNativeReduce' requested here
[2026-03-31T14:30:38.999Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:38.999Z]       |                                  ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:48:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeReduce<c10::BFloat16, gloo::HipDevicePointer<c10::BFloat16>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<c10::BFloat16>> &, gloo::HipDevicePointer<c10::BFloat16> &, const gloo::HipReductionFunction<c10::BFloat16> *&, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:38.999Z]    48 |   return make_unique<HipLocalNativeReduce<T, Dst>>(
[2026-03-31T14:30:38.999Z]       |          ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:405:22: note: in instantiation of function template specialization 'gloo::cudaDeviceReduce<c10::BFloat16, gloo::HipDevicePointer<c10::BFloat16>>' requested here
[2026-03-31T14:30:38.999Z]   405 |     localReduceOp_ = cudaDeviceReduce(
[2026-03-31T14:30:38.999Z]       |                      ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<c10::BFloat16, gloo::HipDeviceWorkspace<c10::BFloat16>>::init<gloo::HipDeviceWorkspace<c10::BFloat16>>' requested here
[2026-03-31T14:30:38.999Z]    79 |   init();
[2026-03-31T14:30:38.999Z]       |   ^
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:199:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:38.999Z]   199 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:38.999Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:38.999Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeBroadcast<c10::BFloat16, gloo::HipDevicePointer<c10::BFloat16>>::HipLocalNativeBroadcast' requested here
[2026-03-31T14:30:38.999Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:38.999Z]       |                                  ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:72:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeBroadcast<c10::BFloat16, gloo::HipDevicePointer<c10::BFloat16>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<c10::BFloat16>> &, gloo::HipDevicePointer<c10::BFloat16> &, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:38.999Z]    72 |   return make_unique<HipLocalNativeBroadcast<T, Src>>(
[2026-03-31T14:30:38.999Z]       |          ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:408:9: note: in instantiation of function template specialization 'gloo::cudaDeviceBroadcast<c10::BFloat16, gloo::HipDevicePointer<c10::BFloat16>>' requested here
[2026-03-31T14:30:38.999Z]   408 |         cudaDeviceBroadcast(streams_, devicePtrs_, scratch_, 0, totalNumElems_);
[2026-03-31T14:30:38.999Z]       |         ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<c10::BFloat16, gloo::HipDeviceWorkspace<c10::BFloat16>>::init<gloo::HipDeviceWorkspace<c10::BFloat16>>' requested here
[2026-03-31T14:30:38.999Z]    79 |   init();
[2026-03-31T14:30:38.999Z]       |   ^
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:38.999Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:86:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:38.999Z]    86 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:38.999Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:38.999Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeReduce<c10::Half, gloo::HipHostPointer<c10::Half>>::HipLocalNativeReduce' requested here
[2026-03-31T14:30:38.999Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:38.999Z]       |                                  ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:48:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeReduce<c10::Half, gloo::HipHostPointer<c10::Half>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<c10::Half>> &, gloo::HipHostPointer<c10::Half> &, const gloo::HipReductionFunction<c10::Half> *&, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:38.999Z]    48 |   return make_unique<HipLocalNativeReduce<T, Dst>>(
[2026-03-31T14:30:38.999Z]       |          ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:380:22: note: in instantiation of function template specialization 'gloo::cudaDeviceReduce<c10::Half, gloo::HipHostPointer<c10::Half>>' requested here
[2026-03-31T14:30:38.999Z]   380 |     localReduceOp_ = cudaDeviceReduce(
[2026-03-31T14:30:38.999Z]       |                      ^
[2026-03-31T14:30:38.999Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<c10::Half>::init<gloo::HipHostWorkspace<c10::Half>>' requested here
[2026-03-31T14:30:39.000Z]    79 |   init();
[2026-03-31T14:30:39.000Z]       |   ^
[2026-03-31T14:30:39.000Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:39.000Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:199:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:39.000Z]   199 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:39.000Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:39.000Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeBroadcast<c10::Half, gloo::HipHostPointer<c10::Half>>::HipLocalNativeBroadcast' requested here
[2026-03-31T14:30:39.000Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:39.000Z]       |                                  ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:72:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeBroadcast<c10::Half, gloo::HipHostPointer<c10::Half>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<c10::Half>> &, gloo::HipHostPointer<c10::Half> &, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:39.000Z]    72 |   return make_unique<HipLocalNativeBroadcast<T, Src>>(
[2026-03-31T14:30:39.000Z]       |          ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:383:9: note: in instantiation of function template specialization 'gloo::cudaDeviceBroadcast<c10::Half, gloo::HipHostPointer<c10::Half>>' requested here
[2026-03-31T14:30:39.000Z]   383 |         cudaDeviceBroadcast(streams_, devicePtrs_, scratch_, 0, totalNumElems_);
[2026-03-31T14:30:39.000Z]       |         ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<c10::Half>::init<gloo::HipHostWorkspace<c10::Half>>' requested here
[2026-03-31T14:30:39.000Z]    79 |   init();
[2026-03-31T14:30:39.000Z]       |   ^
[2026-03-31T14:30:39.000Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:39.000Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:86:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:39.000Z]    86 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:39.000Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:39.000Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeReduce<c10::Half, gloo::HipDevicePointer<c10::Half>>::HipLocalNativeReduce' requested here
[2026-03-31T14:30:39.000Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:39.000Z]       |                                  ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:48:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeReduce<c10::Half, gloo::HipDevicePointer<c10::Half>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<c10::Half>> &, gloo::HipDevicePointer<c10::Half> &, const gloo::HipReductionFunction<c10::Half> *&, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:39.000Z]    48 |   return make_unique<HipLocalNativeReduce<T, Dst>>(
[2026-03-31T14:30:39.000Z]       |          ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:405:22: note: in instantiation of function template specialization 'gloo::cudaDeviceReduce<c10::Half, gloo::HipDevicePointer<c10::Half>>' requested here
[2026-03-31T14:30:39.000Z]   405 |     localReduceOp_ = cudaDeviceReduce(
[2026-03-31T14:30:39.000Z]       |                      ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<c10::Half, gloo::HipDeviceWorkspace<c10::Half>>::init<gloo::HipDeviceWorkspace<c10::Half>>' requested here
[2026-03-31T14:30:39.000Z]    79 |   init();
[2026-03-31T14:30:39.000Z]       |   ^
[2026-03-31T14:30:39.000Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:11:
[2026-03-31T14:30:39.000Z] In file included from /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:23:
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_native.h:199:9: warning: ignoring return value of type 'hipError_t' declared with 'nodiscard' attribute [-Wunused-value]
[2026-03-31T14:30:39.000Z]   199 |         hipDeviceEnablePeerAccess(devB, 0);
[2026-03-31T14:30:39.000Z]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-03-31T14:30:39.000Z] /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/unique_ptr.h:1070:34: note: in instantiation of member function 'gloo::HipLocalNativeBroadcast<c10::Half, gloo::HipDevicePointer<c10::Half>>::HipLocalNativeBroadcast' requested here
[2026-03-31T14:30:39.000Z]  1070 |     { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
[2026-03-31T14:30:39.000Z]       |                                  ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_collectives_device.h:72:10: note: in instantiation of function template specialization 'std::make_unique<gloo::HipLocalNativeBroadcast<c10::Half, gloo::HipDevicePointer<c10::Half>>, std::vector<gloo::HipStream> &, std::vector<gloo::HipDevicePointer<c10::Half>> &, gloo::HipDevicePointer<c10::Half> &, unsigned long &, unsigned long &>' requested here
[2026-03-31T14:30:39.000Z]    72 |   return make_unique<HipLocalNativeBroadcast<T, Src>>(
[2026-03-31T14:30:39.000Z]       |          ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:408:9: note: in instantiation of function template specialization 'gloo::cudaDeviceBroadcast<c10::Half, gloo::HipDevicePointer<c10::Half>>' requested here
[2026-03-31T14:30:39.000Z]   408 |         cudaDeviceBroadcast(streams_, devicePtrs_, scratch_, 0, totalNumElems_);
[2026-03-31T14:30:39.000Z]       |         ^
[2026-03-31T14:30:39.000Z] /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc:79:3: note: in instantiation of function template specialization 'gloo::HipAllreduceBcube<c10::Half, gloo::HipDeviceWorkspace<c10::Half>>::init<gloo::HipDeviceWorkspace<c10::Half>>' requested here
[2026-03-31T14:30:39.000Z]    79 |   init();
[2026-03-31T14:30:39.000Z]       |   ^
[2026-03-31T14:30:39.000Z] 40 warnings and 5 errors generated when compiling for gfx908.
[2026-03-31T14:30:39.000Z] failed to execute:/opt/rocm/llvm/bin/clang++  --offload-arch=gfx90a --offload-arch=gfx908 --offload-arch=gfx942 -O3  -c -x hip /var/lib/jenkins/pytorch/build/third_party/gloo/hip/gloo/hip_allreduce_bcube.cc -o "/var/lib/jenkins/pytorch/build/third_party/gloo/gloo/CMakeFiles/gloo_hip.dir/__/__/__/build/third_party/gloo/hip/gloo/./gloo_hip_generated_hip_allreduce_bcube.cc.o" --offload-compress -fclang-abi-compat=17 -DROCM_VERSION=70201 -DTORCH_HIP_VERSION=702 -DPYTORCH_LAYERNORM_FAST_RECIPROCAL -DNDEBUG -fPIC -D__HIP_PLATFORM_AMD__=1 -DCUDA_HAS_FP16=1 -DUSE_ROCM -D__HIP_NO_HALF_OPERATORS__=1 -D__HIP_NO_HALF_CONVERSIONS__=1 -DTORCH_HIP_VERSION=702 -Wno-shift-count-negative -Wno-shift-count-overflow -Wno-duplicate-decl-specifier -DCAFFE2_USE_MIOPEN -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_HIP -std=c++17 -DHIPBLAS_V2 -DHIP_ENABLE_WARP_SYNC_BUILTINS -DHIPBLASLT_OUTER_VEC -DUSE_ROCM_CK_GEMM -fPIC -D__HIP_PLATFORM_AMD__=1 -DCUDA_HAS_FP16=1 -D__HIP_NO_HALF_OPERATORS__=1 -D__HIP_NO_HALF_CONVERSIONS__=1 -DHIP_VERSION=7 -Wno-shift-count-negative -Wno-shift-count-overflow -Wno-duplicate-decl-specifier -DUSE_MIOPEN -fno-gpu-rdc -Wno-defaulted-function-deleted -I/opt/rocm/include -I/var/lib/jenkins/pytorch/build/third_party/gloo/hip -I/var/lib/jenkins/pytorch/cmake/../third_party/tensorpipe/third_party/libuv/include -I/var/lib/jenkins/pytorch/cmake/../third_party/googletest/googlemock/include -I/var/lib/jenkins/pytorch/cmake/../third_party/googletest/googletest/include -I/var/lib/jenkins/pytorch/third_party/protobuf/src -I/opt/conda/envs/py_3.12/include -I/var/lib/jenkins/pytorch/third_party/XNNPACK/include -I/var/lib/jenkins/pytorch/third_party/ittapi/include -I/var/lib/jenkins/pytorch/cmake/../third_party/eigen -I/opt/rocm/include -I/opt/rocm-7.2.1/include -I/var/lib/jenkins/pytorch/third_party/gloo/PREPEND -I/var/lib/jenkins/pytorch/build/third_party/gloo/hip -I/var/lib/jenkins/pytorch/third_party/gloo -I/var/lib/jenkins/pytorch/build/third_party/gloo -I/opt/rocm/include -I/var/lib/jenkins/pytorch/build/third_party/gloo/hip -I/var/lib/jenkins/pytorch -I/var/lib/jenkins/pytorch/build -I/var/lib/jenkins/pytorch/cmake/../third_party/tensorpipe/third_party/libuv/include -I/var/lib/jenkins/pytorch/cmake/../third_party/googletest/googlemock/include -I/var/lib/jenkins/pytorch/cmake/../third_party/googletest/googletest/include -I/var/lib/jenkins/pytorch/third_party/protobuf/src -I/opt/conda/envs/py_3.12/include -I/var/lib/jenkins/pytorch/third_party/XNNPACK/include -I/var/lib/jenkins/pytorch/third_party/ittapi/include -I/var/lib/jenkins/pytorch/cmake/../third_party/eigen -I/opt/rocm/include -I/opt/rocm-7.2.1/include -I/var/lib/jenkins/pytorch/third_party/gloo/PREPEND -I/var/lib/jenkins/pytorch/build/third_party/gloo/hip -I/var/lib/jenkins/pytorch/third_party/gloo -I/var/lib/jenkins/pytorch/build/third_party/gloo
[2026-03-31T14:30:39.000Z] CMake Error at gloo_hip_generated_hip_allreduce_bcube.cc.o.cmake:200 (message):
[2026-03-31T14:30:39.000Z]   Error generating file
[2026-03-31T14:30:39.000Z]   /var/lib/jenkins/pytorch/build/third_party/gloo/gloo/CMakeFiles/gloo_hip.dir/__/__/__/build/third_party/gloo/hip/gloo/./gloo_hip_generated_hip_allreduce_bcube.cc.o
[2026-03-31T14:30:39.000Z] 
[2026-03-31T14:30:39.000Z] 
[2026-03-31T14:30:39.437Z] [5043/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/primitive_attr.cpp.o
[2026-03-31T14:30:39.437Z] [5044/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/primitive_exec_types.cpp.o
[2026-03-31T14:30:39.437Z] [5045/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm/f32/gemm_utils_f32.cpp.o
[2026-03-31T14:30:39.437Z] [5046/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/query.cpp.o
[2026-03-31T14:30:39.437Z] [5047/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/lrn.cpp.o
[2026-03-31T14:30:39.437Z] [5048/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/reduction/utils.cc.o
[2026-03-31T14:30:39.437Z] [5049/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/reduction/old.cc.o
[2026-03-31T14:30:39.437Z] [5050/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/math/defs.cc.o
[2026-03-31T14:30:39.437Z] [5051/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm/bf16/ref_gemm_bf16.cpp.o
[2026-03-31T14:30:39.437Z] [5052/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/resampling.cpp.o
[2026-03-31T14:30:39.888Z] [5053/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/primitive_desc_iface.cpp.o
[2026-03-31T14:30:39.888Z] [5054/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/scratchpad_debug.cpp.o
[2026-03-31T14:30:39.888Z] [5055/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/memory_desc_wrapper.cpp.o
[2026-03-31T14:30:39.888Z] [5056/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/gemm.cpp.o
[2026-03-31T14:30:39.888Z] [5057/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/prelu.cpp.o
[2026-03-31T14:30:40.319Z] [5058/8156] Building CXX object third_party/onnx/CMakeFiles/onnx_proto.dir/onnx/onnx_onnx_torch-ml.pb.cc.o
[2026-03-31T14:30:40.319Z] [5059/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_engine.cpp.o
[2026-03-31T14:30:40.728Z] [5060/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/text/defs.cc.o
[2026-03-31T14:30:40.728Z] [5061/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/primitive_iface.cpp.o
[2026-03-31T14:30:40.728Z] [5062/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/reduction.cpp.o
[2026-03-31T14:30:40.728Z] [5063/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/sum.cpp.o
[2026-03-31T14:30:41.146Z] [5064/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_batch_normalization_utils.cpp.o
[2026-03-31T14:30:41.146Z] [5065/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/matmul.cpp.o
[2026-03-31T14:30:41.146Z] [5066/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/version_converter/helper.cc.o
[2026-03-31T14:30:41.543Z] [5067/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/reduction/defs.cc.o
[2026-03-31T14:30:41.543Z] [5068/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/reorder.cpp.o
[2026-03-31T14:30:41.543Z] [5069/8156] Building CXX object third_party/fbgemm/CMakeFiles/fbgemm_generic.dir/src/ExecuteKernelU8S8.cc.o
[2026-03-31T14:30:41.543Z] [5070/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/engine.cpp.o
[2026-03-31T14:30:41.948Z] [5071/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/math/old.cc.o
[2026-03-31T14:30:41.948Z] [5072/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/softmax.cpp.o
[2026-03-31T14:30:41.948Z] [5073/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/primitive_attr_postops.cpp.o
[2026-03-31T14:30:41.948Z] [5074/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/scale_utils.cpp.o
[2026-03-31T14:30:42.333Z] [5075/8156] Building CXX object third_party/fbgemm/CMakeFiles/fbgemm_generic.dir/src/EmbeddingSpMDMNBit.cc.o
[2026-03-31T14:30:42.713Z] [5076/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/traditionalml/defs.cc.o
[2026-03-31T14:30:42.713Z] [5077/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/zero_point_utils.cpp.o
[2026-03-31T14:30:42.713Z] [5078/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/primitive_hashing.cpp.o
[2026-03-31T14:30:43.091Z] [5079/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/schema.cc.o
[2026-03-31T14:30:43.091Z] [5080/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/sequence/defs.cc.o
[2026-03-31T14:30:43.475Z] [5081/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/platform.cpp.o
[2026-03-31T14:30:44.399Z] [5082/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/nn/defs.cc.o
[2026-03-31T14:30:44.399Z] [5083/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ncsp_group_normalization.cpp.o
[2026-03-31T14:30:44.399Z] [5084/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_layer_normalization.cpp.o
[2026-03-31T14:30:44.399Z] [5085/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/jit_utils/linux_perf/linux_perf.cpp.o
[2026-03-31T14:30:44.399Z] [5086/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_shuffle.cpp.o
[2026-03-31T14:30:44.776Z] [5087/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_binary.cpp.o
[2026-03-31T14:30:44.776Z] [5088/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm/s8x8s32/ref_gemm_s8x8s32.cpp.o
[2026-03-31T14:30:44.776Z] [5089/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/jit_utils/jit_utils.cpp.o
[2026-03-31T14:30:44.776Z] [5090/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/generator/defs.cc.o
[2026-03-31T14:30:44.776Z] [5091/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_inner_product_int8.cpp.o
[2026-03-31T14:30:45.161Z] [5092/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/nn/old.cc.o
[2026-03-31T14:30:45.161Z] [5093/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/float16.cpp.o
[2026-03-31T14:30:45.548Z] [5094/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/nspc_batch_normalization.cpp.o
[2026-03-31T14:30:45.548Z] [5095/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_prelu.cpp.o
[2026-03-31T14:30:45.548Z] [5096/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_resampling.cpp.o
[2026-03-31T14:30:45.925Z] [5097/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm_x8s8s32x_conv_zp_src_pad_comp.cpp.o
[2026-03-31T14:30:45.925Z] [5098/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_prelu_list.cpp.o
[2026-03-31T14:30:45.925Z] [5099/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/inliner/inliner.cc.o
[2026-03-31T14:30:46.321Z] [5100/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_inner_product.cpp.o
[2026-03-31T14:30:46.321Z] [5101/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_resampling_list.cpp.o
[2026-03-31T14:30:46.321Z] [5102/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_concat.cpp.o
[2026-03-31T14:30:46.321Z] [5103/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/simple_layer_normalization.cpp.o
[2026-03-31T14:30:46.740Z] [5104/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/shape_inference/implementation.cc.o
[2026-03-31T14:30:46.740Z] [5105/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/bfloat16.cpp.o
[2026-03-31T14:30:46.740Z] [5106/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_softmax.cpp.o
[2026-03-31T14:30:47.148Z] [5107/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/gemm_driver.cpp.o
[2026-03-31T14:30:47.148Z] [5108/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/matmul/ref_matmul.cpp.o
[2026-03-31T14:30:47.148Z] [5109/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_layer_normalization_list.cpp.o
[2026-03-31T14:30:47.148Z] [5110/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm/gemm_pack.cpp.o
[2026-03-31T14:30:47.148Z] [5111/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm/gemm.cpp.o
[2026-03-31T14:30:47.148Z] [5112/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/simple_sum.cpp.o
[2026-03-31T14:30:47.148Z] [5113/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_sse41_kernel_b0_sgemm_kern_autogen.cpp.o
[2026-03-31T14:30:47.148Z] [5114/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_reduction.cpp.o
[2026-03-31T14:30:47.576Z] [5115/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/matmul/ref_sparse_matmul.cpp.o
[2026-03-31T14:30:47.997Z] [5116/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm_convolution.cpp.o
[2026-03-31T14:30:47.997Z] [5117/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm/s8x8s32/simple_gemm_s8s8s32.cpp.o
[2026-03-31T14:30:47.997Z] [5118/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ncsp_batch_normalization.cpp.o
[2026-03-31T14:30:47.997Z] [5119/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_deconvolution.cpp.o
[2026-03-31T14:30:47.997Z] [5120/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/cpu_isa_traits.cpp.o
[2026-03-31T14:30:48.393Z] [5121/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/matmul/ref_matmul_int8.cpp.o
[2026-03-31T14:30:48.776Z] [5122/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm_inner_product.cpp.o
[2026-03-31T14:30:48.776Z] [5123/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/tensor/old.cc.o
[2026-03-31T14:30:48.776Z] [5124/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/nhwc_pooling.cpp.o
[2026-03-31T14:30:48.776Z] [5125/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_shuffle_list.cpp.o
[2026-03-31T14:30:49.163Z] [5126/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm_x8s8s32x_convolution_utils.cpp.o
[2026-03-31T14:30:49.163Z] [5127/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/brgemm/brgemm_utils.cpp.o
[2026-03-31T14:30:49.163Z] [5128/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/defs/tensor/defs.cc.o
[2026-03-31T14:30:49.163Z] [5129/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_convolution_int8.cpp.o
[2026-03-31T14:30:49.163Z] [5130/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_group_normalization_list.cpp.o
[2026-03-31T14:30:49.545Z] [5131/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm_convolution_utils.cpp.o
[2026-03-31T14:30:49.545Z] [5132/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_convolution.cpp.o
[2026-03-31T14:30:49.545Z] [5133/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm_inner_product_utils.cpp.o
[2026-03-31T14:30:49.926Z] [5134/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_reduction_list.cpp.o
[2026-03-31T14:30:49.926Z] [5135/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/nchw_pooling.cpp.o
[2026-03-31T14:30:49.926Z] [5136/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/verbose.cpp.o
[2026-03-31T14:30:50.316Z] [5137/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_group_normalization.cpp.o
[2026-03-31T14:30:50.316Z] [5138/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm_x8s8s32x_inner_product.cpp.o
[2026-03-31T14:30:50.316Z] [5139/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/matmul/gemm_x8s8s32x_matmul.cpp.o
[2026-03-31T14:30:51.146Z] [5140/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_binary_list.cpp.o
[2026-03-31T14:30:51.146Z] [5141/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/brgemm/brgemm.cpp.o
[2026-03-31T14:30:51.539Z] [5142/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/amx/jit_avx512_core_amx_gemm_kern.cpp.o
[2026-03-31T14:30:51.953Z] [5143/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_lstm_projection.cpp.o
[2026-03-31T14:30:51.953Z] [5144/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_s16_48x8_copy_bn_kern_autogen.cpp.o
[2026-03-31T14:30:51.953Z] [5145/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_s16_24x8_copy_an_kern_autogen.cpp.o
[2026-03-31T14:30:51.953Z] [5146/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_f32_bf16.cpp.o
[2026-03-31T14:30:51.953Z] [5147/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/matmul/gemm_f32_matmul.cpp.o
[2026-03-31T14:30:51.953Z] [5148/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_lrn_list.cpp.o
[2026-03-31T14:30:51.953Z] [5149/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/amx_tile_configure.cpp.o
[2026-03-31T14:30:51.953Z] [5150/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_softmax_list.cpp.o
[2026-03-31T14:30:51.953Z] [5151/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_lrn.cpp.o
[2026-03-31T14:30:51.953Z] [5152/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/brgemm/brgemm_containers.cpp.o
[2026-03-31T14:30:52.822Z] [5153/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_s16_48x8_copy_an_kern_autogen.cpp.o
[2026-03-31T14:30:52.822Z] [5154/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/brgemm/capi/brgemm_api.cpp.o
[2026-03-31T14:30:52.822Z] [5155/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_s16_24x8_copy_bn_kern_autogen.cpp.o
[2026-03-31T14:30:52.822Z] [5156/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/cell_gru_lbr.cpp.o
[2026-03-31T14:30:52.822Z] [5157/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/matmul/gemm_bf16_matmul.cpp.o
[2026-03-31T14:30:52.822Z] [5158/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_rnn_list.cpp.o
[2026-03-31T14:30:52.822Z] [5159/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/cpu_reducer.cpp.o
[2026-03-31T14:30:53.205Z] [5160/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder.cpp.o
[2026-03-31T14:30:53.205Z] [5161/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/rnn_utils.cpp.o
[2026-03-31T14:30:53.205Z] [5162/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/gemm_x8s8s32x_convolution.cpp.o
[2026-03-31T14:30:53.205Z] [5163/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_batch_normalization.cpp.o
[2026-03-31T14:30:53.205Z] [5164/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_pooling.cpp.o
[2026-03-31T14:30:53.205Z] [5165/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_s16_24x8_copy_at_kern_autogen.cpp.o
[2026-03-31T14:30:53.592Z] [5166/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/cpu_barrier.cpp.o
[2026-03-31T14:30:53.592Z] [5167/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_s16_48x8_copy_bt_kern_autogen.cpp.o
[2026-03-31T14:30:53.592Z] [5168/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/ref_eltwise.cpp.o
[2026-03-31T14:30:53.985Z] [5169/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx2_f32_copy_bn_kern_autogen.cpp.o
[2026-03-31T14:30:53.985Z] [5170/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_s16_24x8_copy_bt_kern_autogen.cpp.o
[2026-03-31T14:30:53.985Z] [5171/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_lstm.cpp.o
[2026-03-31T14:30:53.985Z] [5172/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_eltwise_list.cpp.o
[2026-03-31T14:30:54.374Z] [5173/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_f32_fp8.cpp.o
[2026-03-31T14:30:54.374Z] [5174/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_gemm_bf16bf16f32_kern.cpp.o
[2026-03-31T14:30:54.751Z] [5175/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_gru_lbr.cpp.o
[2026-03-31T14:30:55.140Z] [5176/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_rnn.cpp.o
[2026-03-31T14:30:55.140Z] [5177/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx2_f32_copy_bt_kern_autogen.cpp.o
[2026-03-31T14:30:55.140Z] [5178/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_postgemm_gru.cpp.o
[2026-03-31T14:30:55.140Z] [5179/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_f32_f16.cpp.o
[2026-03-31T14:30:55.140Z] [5180/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_f32_s32.cpp.o
[2026-03-31T14:30:55.528Z] [5181/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_s16_48x8_copy_at_kern_autogen.cpp.o
[2026-03-31T14:30:55.528Z] [5182/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/bf16/jit_avx512_core_gemv_bf16bf16f32_kern.cpp.o
[2026-03-31T14:30:55.909Z] [5183/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_f32_copy_bn_kern_autogen.cpp.o
[2026-03-31T14:30:55.909Z] [5184/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/cell_common.cpp.o
[2026-03-31T14:30:56.289Z] [5185/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/matmul/cpu_matmul_list.cpp.o
[2026-03-31T14:30:56.289Z] [5186/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/gemm_pack.cpp.o
[2026-03-31T14:30:56.289Z] [5187/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_f32_copy_bt_kern_autogen.cpp.o
[2026-03-31T14:30:56.289Z] [5188/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_f32_u8.cpp.o
[2026-03-31T14:30:56.289Z] [5189/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_sum.cpp.o
[2026-03-31T14:30:56.289Z] [5190/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_deconvolution_list.cpp.o
[2026-03-31T14:30:56.289Z] [5191/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/gemv_driver.cpp.o
[2026-03-31T14:30:56.673Z] [5192/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/amx/jit_avx512_core_amx_copy_kern.cpp.o
[2026-03-31T14:30:56.673Z] [5193/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/cell_gru.cpp.o
[2026-03-31T14:30:57.073Z] [5194/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_bf16.cpp.o
[2026-03-31T14:30:57.467Z] [5195/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx2_f32_copy_an_kern_autogen.cpp.o
[2026-03-31T14:30:57.467Z] [5196/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_batch_normalization_list.cpp.o
[2026-03-31T14:30:57.467Z] [5197/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_u8.cpp.o
[2026-03-31T14:30:57.467Z] [5198/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_sse41_f32_copy_bn_kern_autogen.cpp.o
[2026-03-31T14:30:57.467Z] [5199/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_fp4.cpp.o
[2026-03-31T14:30:57.467Z] [5200/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_sse41_f32_copy_an_kern_autogen.cpp.o
[2026-03-31T14:30:57.467Z] [5201/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_s32.cpp.o
[2026-03-31T14:30:57.467Z] [5202/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_sse41_gemv_n_f32_kern.cpp.o
[2026-03-31T14:30:57.467Z] [5203/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_gemv_t_f32_kern.cpp.o
[2026-03-31T14:30:57.865Z] [5204/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_sse41_f32_copy_at_kern_autogen.cpp.o
[2026-03-31T14:30:57.866Z] [5205/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_u4.cpp.o
[2026-03-31T14:30:57.866Z] [5206/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_sse41_f32_copy_bt_kern_autogen.cpp.o
[2026-03-31T14:30:58.268Z] [5207/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx2_f32_copy_at_kern_autogen.cpp.o
[2026-03-31T14:30:58.268Z] [5208/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_f32_f32.cpp.o
[2026-03-31T14:30:58.268Z] [5209/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx512_core_gemm_smalln_tn_f32_kern.cpp.o
[2026-03-31T14:30:58.268Z] [5210/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_s4.cpp.o
[2026-03-31T14:30:58.268Z] [5211/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/brgemm/jit_brgemm_amx_uker.cpp.o
[2026-03-31T14:30:58.268Z] [5212/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx512_core_f32_copy_bn_kern_autogen.cpp.o
[2026-03-31T14:30:58.268Z] [5213/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_f16.cpp.o
[2026-03-31T14:30:58.671Z] [5214/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_sse41_gemv_t_f32_kern.cpp.o
[2026-03-31T14:30:59.088Z] [5215/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx512_core_f32_copy_bt_kern_autogen.cpp.o
[2026-03-31T14:30:59.088Z] [5216/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/gemm_info.cpp.o
[2026-03-31T14:30:59.088Z] [5217/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_f32_s8.cpp.o
[2026-03-31T14:30:59.088Z] [5218/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/brgemm/jit_brdgmm_kernel.cpp.o
[2026-03-31T14:30:59.508Z] [5219/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_f32_copy_at_kern_autogen.cpp.o
[2026-03-31T14:30:59.953Z] [5220/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_s8.cpp.o
[2026-03-31T14:31:00.393Z] [5221/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_comp_s8_s8.cpp.o
[2026-03-31T14:31:00.393Z] [5222/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_pooling_list.cpp.o
[2026-03-31T14:31:00.822Z] [5223/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_regular_fp8.cpp.o
[2026-03-31T14:31:00.822Z] [5224/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_f32_copy_an_kern_autogen.cpp.o
[2026-03-31T14:31:00.822Z] [5225/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_sse41_kernel_sgemm_kern_autogen.cpp.o
[2026-03-31T14:31:01.218Z] [5226/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx512_common_gemm_f32.cpp.o
[2026-03-31T14:31:02.576Z] [5227/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_kernel_sgemm_kern_part1_autogen.cpp.o
[2026-03-31T14:31:02.576Z] [5228/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_gemm_f32.cpp.o
[2026-03-31T14:31:02.576Z] [5229/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx512_core_f32_copy_an_kern_autogen.cpp.o
[2026-03-31T14:31:03.401Z] [5230/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_kernel_b0_sgemm_kern_part1_autogen.cpp.o
[2026-03-31T14:31:03.401Z] [5231/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx2_kernel_sgemm_kern.cpp.o
[2026-03-31T14:31:03.401Z] [5232/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_kernel_b0_sgemm_kern_part2_autogen.cpp.o
[2026-03-31T14:31:03.792Z] [5233/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_inner_product_list.cpp.o
[2026-03-31T14:31:04.180Z] [5234/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_comp_f32_s8.cpp.o
[2026-03-31T14:31:05.029Z] [5235/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/reorder/cpu_reorder_comp_bf16_s8.cpp.o
[2026-03-31T14:31:05.893Z] [5236/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx512_core_f32_copy_at_kern_part1_autogen.cpp.o
[2026-03-31T14:31:05.893Z] [5237/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx512_core_f32_copy_at_kern_part2_autogen.cpp.o
[2026-03-31T14:31:07.265Z] [5238/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/brgemm/jit_brgemm_kernel.cpp.o
[2026-03-31T14:31:07.688Z] [5239/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/x64/CMakeFiles/dnnl_cpu_x64.dir/gemm/f32/jit_avx_kernel_sgemm_kern_part2_autogen.cpp.o
[2026-03-31T14:31:09.073Z] [5240/8156] Building CXX object third_party/onnx/CMakeFiles/onnx.dir/onnx/version_converter/convert.cc.o
[2026-03-31T14:31:09.974Z] [5241/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/simple_resampling.cpp.o
[2026-03-31T14:31:11.900Z] [5242/8156] Building CXX object third_party/ideep/mkl-dnn/src/common/CMakeFiles/dnnl_common.dir/memory_zero_pad.cpp.o
[2026-03-31T14:31:22.442Z] [5243/8156] Building CXX object third_party/fbgemm/CMakeFiles/fbgemm_generic.dir/src/EmbeddingSpMDM.cc.o
[2026-03-31T14:31:39.387Z] [5244/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/cpu_convolution_list.cpp.o
[2026-03-31T14:32:13.861Z] [5245/8156] Building CXX object third_party/ideep/mkl-dnn/src/cpu/CMakeFiles/dnnl_cpu.dir/rnn/ref_rnn.cpp.o
[2026-03-31T14:32:21.910Z] [5246/8156] Building CXX object third_party/fbgemm/CMakeFiles/fbgemm_avx2.dir/src/FbgemmI8Depthwise3DAvx2.cc.o
[2026-03-31T14:33:12.077Z] [5247/8156] Building CXX object third_party/fbgemm/CMakeFiles/fbgemm_avx2.dir/src/FbgemmI8DepthwiseAvx2.cc.o
[2026-03-31T14:33:12.077Z] ninja: build stopped: subcommand failed.
[2026-03-31T14:33:12.469Z] -- Checkout nccl release tag: v2.27.5-1
[2026-03-31T14:33:12.469Z] + sccache_epilogue
[2026-03-31T14:33:12.469Z] + echo '::group::Sccache Compilation Log'
[2026-03-31T14:33:12.469Z] + echo '=================== sccache compilation log ==================='
[2026-03-31T14:33:12.469Z] + python /var/lib/jenkins/pytorch/.ci/pytorch/print_sccache_log.py /root/sccache_error.log
[2026-03-31T14:33:12.469Z] ::group::Sccache Compilation Log
[2026-03-31T14:33:12.469Z] =================== sccache compilation log ===================
[2026-03-31T14:33:12.469Z] + echo '=========== If your build fails, please take a look at the log above for possible reasons ==========='
[2026-03-31T14:33:12.469Z] + sccache --show-stats
[2026-03-31T14:33:12.469Z] =========== If your build fails, please take a look at the log above for possible reasons ===========
[2026-03-31T14:33:12.469Z] Compile requests                   5505
[2026-03-31T14:33:12.469Z] Compile requests executed          5318
[2026-03-31T14:33:12.469Z] Cache hits                            4
[2026-03-31T14:33:12.469Z] Cache hits (C/C++)                    4
[2026-03-31T14:33:12.469Z] Cache misses                       5271
[2026-03-31T14:33:12.469Z] Cache misses (C/C++)               5271
[2026-03-31T14:33:12.469Z] Cache hits rate                    0.08 %
[2026-03-31T14:33:12.469Z] Cache hits rate (C/C++)            0.08 %
[2026-03-31T14:33:12.470Z] Cache timeouts                        0
[2026-03-31T14:33:12.470Z] Cache read errors                     0
[2026-03-31T14:33:12.470Z] Forced recaches                       0
[2026-03-31T14:33:12.470Z] Cache write errors                    0
[2026-03-31T14:33:12.470Z] Cache errors                         27
[2026-03-31T14:33:12.470Z] Cache errors (C/C++)                 27
[2026-03-31T14:33:12.470Z] Compilations                       5271
[2026-03-31T14:33:12.470Z] Compilation failures                 16
[2026-03-31T14:33:12.470Z] Non-cacheable compilations            0
[2026-03-31T14:33:12.470Z] Non-cacheable calls                  23
[2026-03-31T14:33:12.470Z] Non-compilation calls               164
[2026-03-31T14:33:12.470Z] Unsupported compiler calls            0
[2026-03-31T14:33:12.470Z] Average cache write               0.055 s
[2026-03-31T14:33:12.470Z] Average compiler                  6.663 s
[2026-03-31T14:33:12.470Z] Average cache read hit            0.048 s
[2026-03-31T14:33:12.470Z] Failed distributed compilations       0
[2026-03-31T14:33:12.470Z] 
[2026-03-31T14:33:12.470Z] Non-cacheable reasons:
[2026-03-31T14:33:12.470Z] multiple input files                 13
[2026-03-31T14:33:12.470Z] -M                                    8
[2026-03-31T14:33:12.470Z] -E                                    1
[2026-03-31T14:33:12.470Z] unknown source language               1
[2026-03-31T14:33:12.470Z] 
[2026-03-31T14:33:12.470Z] Cache location                  Local disk: "/root/.cache/sccache"
[2026-03-31T14:33:12.470Z] Use direct/preprocessor mode?   yes
[2026-03-31T14:33:12.470Z] Version (client)                0.10.0
[2026-03-31T14:33:12.470Z] Cache size                           29 MiB
[2026-03-31T14:33:12.470Z] Max cache size                       10 GiB
[2026-03-31T14:33:12.470Z] + sccache --stop-server
[2026-03-31T14:33:12.470Z] Stopping sccache server...
[2026-03-31T14:33:12.470Z] Compile requests                   5505
[2026-03-31T14:33:12.470Z] Compile requests executed          5318
[2026-03-31T14:33:12.470Z] Cache hits                            4
[2026-03-31T14:33:12.470Z] Cache hits (C/C++)                    4
[2026-03-31T14:33:12.470Z] Cache misses                       5271
[2026-03-31T14:33:12.470Z] Cache misses (C/C++)               5271
[2026-03-31T14:33:12.470Z] Cache hits rate                    0.08 %
[2026-03-31T14:33:12.470Z] Cache hits rate (C/C++)            0.08 %
[2026-03-31T14:33:12.470Z] Cache timeouts                        0
[2026-03-31T14:33:12.470Z] Cache read errors                     0
[2026-03-31T14:33:12.470Z] Forced recaches                       0
[2026-03-31T14:33:12.470Z] Cache write errors                    0
[2026-03-31T14:33:12.470Z] Cache errors                         27
[2026-03-31T14:33:12.470Z] Cache errors (C/C++)                 27
[2026-03-31T14:33:12.470Z] Compilations                       5271
[2026-03-31T14:33:12.470Z] Compilation failures                 16
[2026-03-31T14:33:12.470Z] Non-cacheable compilations            0
[2026-03-31T14:33:12.470Z] Non-cacheable calls                  23
[2026-03-31T14:33:12.470Z] Non-compilation calls               164
[2026-03-31T14:33:12.470Z] Unsupported compiler calls            0
[2026-03-31T14:33:12.470Z] Average cache write               0.055 s
[2026-03-31T14:33:12.470Z] Average compiler                  6.663 s
[2026-03-31T14:33:12.470Z] Average cache read hit            0.048 s
[2026-03-31T14:33:12.470Z] Failed distributed compilations       0
[2026-03-31T14:33:12.470Z] 
[2026-03-31T14:33:12.470Z] Non-cacheable reasons:
[2026-03-31T14:33:12.470Z] multiple input files                 13
[2026-03-31T14:33:12.470Z] -M                                    8
[2026-03-31T14:33:12.470Z] -E                                    1
[2026-03-31T14:33:12.470Z] unknown source language               1
[2026-03-31T14:33:12.470Z] 
[2026-03-31T14:33:12.470Z] Cache location                  Local disk: "/root/.cache/sccache"
[2026-03-31T14:33:12.470Z] Use direct/preprocessor mode?   yes
[2026-03-31T14:33:12.470Z] Version (client)                0.10.0
[2026-03-31T14:33:12.470Z] Cache size                           29 MiB
[2026-03-31T14:33:12.470Z] Max cache size                       10 GiB
[2026-03-31T14:33:12.470Z] ::endgroup::
[2026-03-31T14:33:12.470Z] + echo ::endgroup::

Initialize / Build PyTorch / Error signal

Error in error step, with arguments `Found build error
Regex: .FAILED:.
Output truncated.

Details

  • Kill older PR Builds (1.7 sec)
  • Initialize (38 min)
    • Download CI scripts (53 sec)
    • Checkout Pytorch (1 min 14 sec)
    • Check base Docker image existence (21 sec)
    • Pull Docker Image (11 min)
    • Build PyTorch (21 min)
      Error: script returned exit code 1 - logs
      Error: *Found build error
      Regex: .FAILED:.
      Error: FAILED: third_party/gloo/gloo/CMakeFiles/gloo_hip.dir////build/third_party/gloo/hip/gloo/gloo_hip_generated_hip_private.hip.o /var/lib/jenkins/pytorch/build/third_party/gloo/gloo/CMakeFiles/gloo_hip.dir////build/third_party/gloo/hip/gloo/gloo_hip_generated_hip_private.hip.o * - logs
      Error: PyTorch build failed hudson.AbortException: script returned exit code 1 - logs
  • Tests (38 sec)
    • Test PyTorch (11 ms)
      • Test PyTorch (15 sec)
    • Test Distributed (9 ms)
      • Test Distributed (15 sec)
    • Test Inductor (7 ms)
      • Test Inductor (16 sec)
    • Test PyTorch Slow (7 ms)
      • Test PyTorch Slow (16 sec)
    • Microbenchmark (34 sec)
      • Microbenchmark (16 sec)
  • Post Build (4.8 sec)
  • Declarative: Post Actions (4 sec)
View more details on ROCm Repo Management API