Repository navigation
Feat/sample overwrite ipv4 - #34
Merged
Merged
Conversation
Three of the P2 rows closed last time as named refusals. Two of those refusals were mine to make rather than the architecture's, and one of them rested on a claim that was wrong. `SAMPLE` was refused because a sample over roaring containers looked like either a decode pass or a block of write-order ids wearing a sample's name. That is true of sampling *blocks*. It is not true of sampling the id's **low bits**: a container holds 65,536 consecutive ids, so "every tenth id" is a pattern inside each container, built once per phase and intersected. One AND per container, no record decoded, and spread evenly through every window of time rather than picking some of them whole. The stride need not divide the container either - a container beginning part-way through the pattern begins at a phase, and a phase is arithmetic. So `SELECT ... FROM t SAMPLE 10` is answered, and `SAMPLE 1/10` and `SAMPLE 0.1` with it. The fraction is read through the decimal it already parsed as - units and a scale, never a float - so `0.1` is exactly one in ten. What stays refused is a fraction with no whole stride: `0.3` would be one in 3.33, and answering it as one in three is a tenth more data under the name that was asked for. So is `SAMPLE BY`, which names the column a sample is drawn on, where the unit here is the record id. `Rows::Sample` is a row-set variant, not a `Plan` variant: it sits inside the row set a plan already reads, so `big-cluster/src/merge.rs` is untouched and there is no new merge arm. It costs a PQL call (`Sample(<bitmap>, stride=n)`) and one wire tag. `INSERT OVERWRITE TABLE t (a) SELECT ...` empties and then writes. It is still two operations with nothing wrapping them, and that is said rather than hidden - a crash between them leaves the table empty, and the atomic replacement is `EXCHANGE TABLES`. What it buys over writing the two statements is the order: the source query is read whole and merged before anything is cleared, so a `SELECT` that turns out to be unreadable leaves the table alone. It demands `DELETE` as well as `INSERT`. `PARTITION (...)` keeps the refusal, narrowed to it: a shard is computed from the record id and there is nothing to aim at. `IPv4` is read as the `UINT(32)` an address is. The reason it was refused - that a type name has to be a real `ColumnKind` to survive the round-trip gate - was simply wrong: the gate compares `Column` structs, not spellings, and this dialect already reads `TIMESTAMP` as `DATETIME`, `BIGINT` as `UINT(64)` and `TEXT` as `SET`. `UUID` and `IPv6` stay refused, and the difference is what would be lost: both are 128 bits where a bit-sliced value stops at 64, so the only column they could land in is keyed, which answers `=` but not `<`. `Refused::Sample` and `Refused::Overwrite` narrow rather than disappear; `Refused::ALL` is unchanged at 97.
The last of the P2 rows that was refused on my judgement rather than the
architecture's. The refusal said `MUTEX` is the storage an enum wants and that
the missing half - a declared list of members - had nowhere to live. The first
half was right and is what makes this cheap; the second was a gap in the catalog
rather than a fact about it.
A member list does not fit in a fixed-width catalog record, so it follows the
shape `SAVED_QUERY_TEXT` already established: a new record kind carrying NUL-
joined chunks under the field's table and field id, with the total length in the
`FIELD` header's free word so a torn set is detectable. `FieldDef` grows a
`members` list; nothing below the catalog changes at all. No new `FieldKind`, no
`place()` behaviour, no storage format for a value - the field is a mutex and
stores an interned row id exactly as it did.
What the list buys is the two things a mutex cannot say:
- A value outside it is refused on the way in, at `fact::from_literal` and
`fact::from_text` - both write funnels, so the declaration holds for a SQL
insert and for the import route alike. The message names the members, which
is the only question that error raises; the code stays the generic one for a
value a field cannot take, because what to do about it is the same.
- `SHOW CREATE TABLE` answers `ENUM('new', 'paid')` rather than `MUTEX`. A
mutex and an enum are one `FieldKind`, so what tells them apart there is
whether a member list was declared.
A field that declares none is a plain mutex and takes anything. That is what
says the check belongs to the declaration rather than to the kind, and it is
also what every field written before this reads back as: an older `FIELD` record
carries zero in the word that now holds the blob length and is followed by no
member records, which is exactly the shape a field with no members has today.
`a_field_written_without_members_reads_back_as_a_plain_mutex` pins that.
`Enum8` and `Enum16` are the same declaration - the width names how many members
fit and the dictionary has no such ceiling. `Refused::EnumType` narrows to the
two shapes that stay refused: `'a' = 1`, whose number promises which integer the
value is stored as when the dictionary assigns that, and a repeated member,
which is unreachable and would make `SHOW CREATE TABLE` disagree with what was
typed.
`Declared` replaces five positional arguments to `Db::declare`, which is where
clippy drew the line and where a reader would have too.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.