Skip to content

Partitioned ingest API #4317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tokoko wants to merge 11 commits into
base: main
Choose a base branch
from
Draft

Partitioned ingest API #4317

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
51aab39
add partitioned ingest, postgres impl
tokoko Apr 18, 2026
311edc3
wire driver manager, soften schema validation claim
tokoko Apr 18, 2026
b29e713
fix(postgresql): harden partitioned ingest commit and abort test
tokoko Apr 18, 2026
82d38cf
fix(driver_manager): harden 1.2.0 ingest wrappers and cover version b…
tokoko Apr 18, 2026
54ca334
fix(postgresql): isolate ingest-commit savepoint from caller state
tokoko Apr 18, 2026
9ccfb23
fix(driver_manager): null-guard ingest wrappers and expand wrapper tests
tokoko Apr 18, 2026
4197392
fix(postgresql): guard ingest-commit savepoint name length and cover …
tokoko Apr 18, 2026
98d3699
test(driver_manager): pin ingest-wrapper NULL-arg guards with valid c…
tokoko Apr 18, 2026
088e114
refactor(postgresql): share ingest-commit savepoint name with tests
tokoko Apr 18, 2026
97ba184
refactor(postgresql): collapse ingest hex encoders to internal::HexId16
tokoko Apr 18, 2026
95d027c
partitioned ingest spec
tokoko May 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions c/driver/postgresql/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ add_arrow_lib(adbc_driver_postgresql
connection.cc
error.cc
database.cc
ingest_partition.cc
postgresql.cc
result_helper.cc
result_reader.cc
Expand Down Expand Up @@ -81,6 +82,7 @@ if(ADBC_BUILD_TESTS)
EXTRA_LABELS
driver-postgresql
SOURCES
partitioned_ingest_test.cc
postgres_type_test.cc
postgresql_test.cc
EXTRA_LINK_LIBS
Expand Down
19 changes: 19 additions & 0 deletions c/driver/postgresql/connection.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,25 @@ class PostgresConnection {
AdbcStatusCode Init(struct AdbcDatabase* database, struct AdbcError* error);
AdbcStatusCode Release(struct AdbcError* error);
AdbcStatusCode Rollback(struct AdbcError* error);
AdbcStatusCode BeginIngestPartitions(const char* target_catalog,
const char* target_db_schema,
const char* target_table, const char* mode,
struct ArrowSchema* schema,
struct AdbcIngestHandle* out_handle,
struct AdbcError* error);
AdbcStatusCode WriteIngestPartition(const uint8_t* handle, size_t handle_len,
struct ArrowArrayStream* data,
struct AdbcIngestReceipt* out_receipt,
struct AdbcError* error);
AdbcStatusCode CommitIngestPartitions(const uint8_t* handle, size_t handle_len,
size_t num_receipts, const uint8_t** receipts,
const size_t* receipt_lens,
int64_t* rows_affected, struct AdbcError* error);
AdbcStatusCode AbortIngestPartitions(const uint8_t* handle, size_t handle_len,
size_t num_receipts, const uint8_t** receipts,
const size_t* receipt_lens,
struct AdbcError* error);

AdbcStatusCode SetOption(const char* key, const char* value, struct AdbcError* error);
AdbcStatusCode SetOptionBytes(const char* key, const uint8_t* value, size_t length,
struct AdbcError* error);
Expand Down
Loading
Loading