Add Randomized SVDs by Intron7 · Pull Request #2999 · rapidsai/raft

Intron7 · 2026-04-09T11:19:57Z

This PR adds randomized SVDs to raft based on (Halko et al. 2009) and (Tomás et al. 2024). I also added the possibility for a very limited linear operator. This one is C++ and might be useful for sparse PCA in cuml. It's tested in the C++ layer but not exposed in Python. This PR mimics the implementation I did for rapids-singlecell

sparse svd benchmarks (k=50, n_oversamples=10)
GPU: RTX PRO 6000 Blackwell. raft = best of 3; sklearn/scipy = single run.

rows       nnz      raft GPU   sklearn CPU   scipy ARPACK   sk/raft   scipy/raft
50k        101M     0.27s      7.38s         18.69s         27.3×     69.2×
200k       400M     1.06s      25.61s        62.49s         24.1×     58.8×
500k       1.02B    2.75s      64.97s        153.40s        23.6×     55.8×
982k       2.01B    5.48s      126.16s       307.04s        23.0×     56.1×

~23-27x over sklearn, ~56-69x over scipy ARPACK. speedup is remarkably stable across scales — raft stays sub-linear while CPU methods scale linearly with nnz. at ~1M cells raft finishes in 5.5s vs 2 min (sklearn) / 5 min (scipy).

copy-pr-bot · 2026-04-09T11:20:01Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

aamijar · 2026-04-10T00:16:33Z

/ok to test 307d18e

copy-pr-bot · 2026-04-10T00:16:36Z

/ok to test 307d18e

@aamijar, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

aamijar · 2026-04-10T00:17:36Z

/ok to test e52ad48

viclafargue

Thanks! Here are some comments.

viclafargue · 2026-04-10T09:50:50Z

+template <typename ValueTypeT>
+struct sparse_svd_config {
+  /** @brief Number of singular values/vectors to compute */
+  int n_components;


We should maybe set a default for n_components. A C++ user that forgets to specify a value would face an undefined behavior. Setting it to 0 would allow it to be catched immediately by parameters validation.

viclafargue · 2026-04-10T09:53:12Z

+cdef sparse_svd_config[float] config_float
+cdef sparse_svd_config[double] config_double


Using module-level global config is not thread-safe. Please instantiate them in the function where they are needed.

viclafargue · 2026-04-10T10:02:01Z

+void sparse_randomized_svd(
+  raft::resources const& handle,
+  sparse_svd_config<ValueTypeT> const& config,
+  raft::device_csr_matrix_view<ValueTypeT, int, int, NNZTypeT> A,


Suggested change

raft::device_csr_matrix_view<ValueTypeT, int, int, NNZTypeT> A,

raft::device_csr_matrix_view<const ValueTypeT, int, int, NNZTypeT> A,

The input matrix should be const.

Also for the doc, we sometimes use the paradigm @param [in], @param [inout], @param [out]. @param [in] is supposed to be usable with const data.

viclafargue · 2026-04-10T10:08:44Z

+               Omega.data_handle(), n, block_size),
+             Y.view());
+  }  // Omega freed here
+  cholesky_qr2(handle, Y.view());


We should maybe check the return value of cholesky_qr2 calls and emit a warning when there's a fallback to standard QR.

viclafargue · 2026-04-10T10:51:12Z

+  int rows() const { return m_; }
+  int cols() const { return n_; }


Suggested change

int rows() const { return m_; }

int cols() const { return n_; }

int rows() const { return A_.structure_view().get_n_rows(); }

int cols() const { return A_.structure_view().get_n_cols(); }

viclafargue · 2026-04-10T10:55:21Z

+  raft::device_csr_matrix_view<const ValueTypeT, int, int, NNZTypeT> A_;
+  int m_;
+  int n_;


A_ already stores the dimensions of the matrix, storing m_ and n_ here is redundant, adds complexity and introduces the possibility of bugs. The number of rows and cols can be derived directly from A_. Also A_ should probably be a private member.

viclafargue · 2026-04-10T10:58:18Z

+void sparse_randomized_svd(
+  raft::resources const& handle,
+  sparse_svd_config<ValueTypeT> const& config,
+  OperatorT const& op,


Even though the csr_linear_operator struct is certainly useful, the user should not ever need to construct this exotic type. The public function should expose an interface with a raft::device_csr_matrix_view and the csr_linear_operator utility should be built inside of the function.

viclafargue · 2026-04-10T11:18:07Z

+    rmm::device_uvector<ValueTypeT> Q_copy(m * k, stream);
+    raft::copy(Q_copy.data(), Q.data_handle(), m * k, stream);
+    raft::linalg::qrGetQ(handle, Q_copy.data(), Q.data_handle(), m, k, stream);


Suggested change

rmm::device_uvector<ValueTypeT> Q_copy(m * k, stream);

raft::copy(Q_copy.data(), Q.data_handle(), m * k, stream);

raft::linalg::qrGetQ(handle, Q_copy.data(), Q.data_handle(), m, k, stream);

raft::linalg::qrGetQ(handle, Q.data_handle(), Q.data_handle(), m, k, stream);

qrGetQ already does a copy internally. Using Q.data_handle() twice would allow the operation to work inplace (even the copy could be avoided as src==dst). To double check though.

Additionally, m * k has an integer overflow risk.

viclafargue · 2026-04-10T11:18:18Z

+    rmm::device_uvector<ValueTypeT> Q_copy(m * k, stream);
+    raft::copy(Q_copy.data(), Q.data_handle(), m * k, stream);
+    raft::linalg::qrGetQ(handle, Q_copy.data(), Q.data_handle(), m, k, stream);


Similar comment here.

achirkin

Thanks for the PR! Couple notes about usage of raft primitives below

aamijar · 2026-04-14T22:56:01Z

/ok to test a803923

aamijar · 2026-04-14T22:56:33Z

/ok to test a803923

aamijar · 2026-04-20T17:47:52Z

/ok to test 5dc8079

aamijar

Thanks @Intron7 for the great work! It would be great to see your benchmarks results in the PR description!

I've left my review below:

aamijar · 2026-04-21T01:19:08Z

+#include <raft/sparse/solver/randomized_svds.cuh>
+
+#define FUNC_DEF(ValueType, Suffix)                                                  \
+  void sparse_randomized_svd_##Suffix(                                               \


No need to have a separate function with a suffix here. You can just have the same function name with float and double overloads.

aamijar · 2026-04-21T01:22:36Z

+ * @{
+ */
+
+void sparse_randomized_svd_float(const raft::resources& handle,


You can remove the suffix here.
Also you can use macros with FUNC_DECL for more concise declarations. See https://github.com/rapidsai/raft/blob/main/cpp/include/raft_runtime/solver/lanczos.hpp

aamijar · 2026-04-21T01:23:54Z

+ */
+/** @} */
+
+}  // namespace raft::sparse::solver


Is there a reason to add this empty types file?

aamijar · 2026-04-21T01:48:18Z

+struct csr_linear_operator {
+  /**
+   * @brief Construct from a const CSR matrix view
+   */


Great to see a LinearOperator implementation! It could be useful in the lanczos solver as well (#2705). Is this one similar to https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.LinearOperator.html

I see this is in a detail file, but it might be nice to have a public interface.

This is not a full Linear Operator. Its just a very limited version for raft to do PCA with the SVDs

aamijar · 2026-04-21T02:44:38Z

+/**
+ * @brief Compute truncated SVD of a sparse CSR matrix using randomized algorithm.
+ *
+ * Overload accepting a mutable CSR matrix view (implicitly converted to const).


Do we need this overload at all? If sparse_randomized_svd doesn't modify A then we should just have a const API.

Intron7 · 2026-04-27T13:45:30Z

sparse svd benchmarks (k=50, n_oversamples=10)
GPU: RTX PRO 6000 Blackwell. raft = best of 3; sklearn/scipy = single run.

rows       nnz      raft GPU   sklearn CPU   scipy ARPACK   sk/raft   scipy/raft
50k        101M     0.27s      7.38s         18.69s         27.3×     69.2×
200k       400M     1.06s      25.61s        62.49s         24.1×     58.8×
500k       1.02B    2.75s      64.97s        153.40s        23.6×     55.8×
982k       2.01B    5.48s      126.16s       307.04s        23.0×     56.1×

~23-27x over sklearn, ~56-69x over scipy ARPACK. speedup is remarkably stable across scales — raft stays sub-linear while CPU methods scale linearly with nnz. at ~1M cells raft finishes in 5.5s vs 2 min (sklearn) / 5 min (scipy).

aamijar · 2026-04-28T01:13:08Z

/ok to test b601ea2

aamijar

Thanks @Intron7!

achirkin · 2026-05-06T16:55:21Z

+  }
+
+  // Step 9: Truncate S, optionally compute Vt
+  raft::copy(singular_values.data_handle(), S_full.data_handle(), k, stream);


A nitpick

Suggested change

raft::copy(singular_values.data_handle(), S_full.data_handle(), k, stream);

raft::copy(handle, singular_values, raft::make_const_mdspan(S_full));

Intron7 added 2 commits April 9, 2026 12:33

add randomized svds

d20ea68

add more tests

307d18e

Intron7 requested review from a team as code owners April 9, 2026 11:19

github-project-automation Bot added this to Unstructured Data Processing Apr 9, 2026

Merge branch 'main' into randomized_svds

e52ad48

aamijar assigned Intron7 Apr 10, 2026

aamijar added non-breaking Non-breaking change feature request New feature or request labels Apr 10, 2026

aamijar changed the title ~~add randomized svds~~ Add randomized svds Apr 10, 2026

aamijar changed the title ~~Add randomized svds~~ Add Randomized SVDs Apr 10, 2026

viclafargue reviewed Apr 10, 2026

View reviewed changes

adress issues

8a08ffc

achirkin requested changes Apr 14, 2026

View reviewed changes

Comment thread cpp/include/raft/sparse/solver/detail/cholesky_qr.cuh Outdated

Comment thread cpp/include/raft/sparse/solver/detail/cholesky_qr.cuh

Intron7 and others added 2 commits April 14, 2026 17:46

use raft GEMM

9dcc910

Merge branch 'main' into randomized_svds

a803923

Intron7 requested a review from achirkin April 14, 2026 15:51

fix pre-commit

5dc8079

Intron7 requested a review from viclafargue April 15, 2026 11:44

aamijar requested changes Apr 21, 2026

View reviewed changes

aamijar moved this to In Progress in Unstructured Data Processing Apr 21, 2026

aamijar reviewed Apr 21, 2026

View reviewed changes

Intron7 and others added 2 commits April 21, 2026 19:30

adress templating macros

9b44f7a

Merge branch 'main' into randomized_svds

c5fd1a4

adress adress view and UV only

b601ea2

Intron7 requested a review from aamijar April 27, 2026 14:22

aamijar approved these changes Apr 28, 2026

View reviewed changes

viclafargue mentioned this pull request Apr 28, 2026

[BUG]RuntimeError: cuBLAS error encountered at: rapidsai/cuml#7875

Closed

Merge branch 'main' into randomized_svds

15b6285

achirkin approved these changes May 6, 2026

View reviewed changes

		cdef sparse_svd_config[float] config_float
		cdef sparse_svd_config[double] config_double

	raft::device_csr_matrix_view<ValueTypeT, int, int, NNZTypeT> A,
	raft::device_csr_matrix_view<const ValueTypeT, int, int, NNZTypeT> A,

		int rows() const { return m_; }
		int cols() const { return n_; }

	raft::copy(singular_values.data_handle(), S_full.data_handle(), k, stream);
	raft::copy(handle, singular_values, raft::make_const_mdspan(S_full));

Conversation

Intron7 commented Apr 9, 2026 • edited by aamijar Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-bot Bot commented Apr 9, 2026

Uh oh!

aamijar commented Apr 10, 2026

Uh oh!

copy-pr-bot Bot commented Apr 10, 2026

Uh oh!

aamijar commented Apr 10, 2026

Uh oh!

viclafargue left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

achirkin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aamijar commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aamijar commented Apr 14, 2026

Uh oh!

aamijar commented Apr 20, 2026

Uh oh!

aamijar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Intron7 commented Apr 27, 2026

Uh oh!

aamijar commented Apr 28, 2026

Uh oh!

aamijar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Intron7 commented Apr 9, 2026 •

edited by aamijar

Loading

aamijar commented Apr 14, 2026 •

edited

Loading