Skip to content

BMP: Faster bitfield reading #2900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RunDevelopment wants to merge 3 commits into
base: main
Choose a base branch
from
+51 −43
Open

BMP: Faster bitfield reading #2900

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1c23e96
BMP: Faster bitfield reading
RunDevelopment Apr 4, 2026
5a86c01
Add more comments
RunDevelopment Apr 4, 2026
f0d882e
Simplified impl a little
RunDevelopment Apr 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 51 additions & 43 deletions src/codecs/bmp/decoder.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,38 +34,23 @@ const BI_CMYK: u32 = 11;
const BI_CMYKRLE8: u32 = 12;
const BI_CMYKRLE4: u32 = 13;

static LOOKUP_TABLE_3_BIT_TO_8_BIT: [u8; 8] = [0, 36, 73, 109, 146, 182, 219, 255];
static LOOKUP_TABLE_4_BIT_TO_8_BIT: [u8; 16] = [
0, 17, 34, 51, 68, 85, 102, 119, 136, 153, 170, 187, 204, 221, 238, 255,
];
static LOOKUP_TABLE_5_BIT_TO_8_BIT: [u8; 32] = [
0, 8, 16, 25, 33, 41, 49, 58, 66, 74, 82, 90, 99, 107, 115, 123, 132, 140, 148, 156, 165, 173,
181, 189, 197, 206, 214, 222, 230, 239, 247, 255,
];
static LOOKUP_TABLE_6_BIT_TO_8_BIT: [u8; 64] = [
0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 45, 49, 53, 57, 61, 65, 69, 73, 77, 81, 85, 89, 93,
97, 101, 105, 109, 113, 117, 121, 125, 130, 134, 138, 142, 146, 150, 154, 158, 162, 166, 170,
174, 178, 182, 186, 190, 194, 198, 202, 206, 210, 215, 219, 223, 227, 231, 235, 239, 243, 247,
251, 255,
];

static R5_G5_B5_COLOR_MASK: Bitfields = Bitfields {
r: Bitfield { len: 5, shift: 10 },
g: Bitfield { len: 5, shift: 5 },
b: Bitfield { len: 5, shift: 0 },
a: Bitfield { len: 0, shift: 0 },
r: Bitfield::from_len_shift(5, 10),
g: Bitfield::from_len_shift(5, 5),
b: Bitfield::from_len_shift(5, 0),
a: Bitfield::from_len_shift(0, 0),
};
const R8_G8_B8_COLOR_MASK: Bitfields = Bitfields {
r: Bitfield { len: 8, shift: 24 },
g: Bitfield { len: 8, shift: 16 },
b: Bitfield { len: 8, shift: 8 },
a: Bitfield { len: 0, shift: 0 },
r: Bitfield::from_len_shift(8, 24),
g: Bitfield::from_len_shift(8, 16),
b: Bitfield::from_len_shift(8, 8),
a: Bitfield::from_len_shift(0, 0),
};
const R8_G8_B8_A8_COLOR_MASK: Bitfields = Bitfields {
r: Bitfield { len: 8, shift: 16 },
g: Bitfield { len: 8, shift: 8 },
b: Bitfield { len: 8, shift: 0 },
a: Bitfield { len: 8, shift: 24 },
r: Bitfield::from_len_shift(8, 16),
g: Bitfield::from_len_shift(8, 8),
b: Bitfield::from_len_shift(8, 0),
a: Bitfield::from_len_shift(8, 24),
};

const RLE_ESCAPE: u8 = 0;
Expand Down Expand Up @@ -770,12 +755,39 @@ fn set_1bit_pixel_run<'a, T: Iterator<Item = &'a u8>>(
struct Bitfield {
shift: u32,
len: u32,
factor_addend: (u32, u32),
}

impl Bitfield {
/// Factors and addends such that `((data * factor + addend) >> 8) as u8`
/// maps the `data` value to the nearest value in the full 0-255 range.
///
/// All constants come from the following site and were adjusted to use a
/// shift of 8: https://rundevelopment.github.io/blog/fast-unorm-conversions#constants
const FACTOR_ADDEND: [(u32, u32); 8] = [
(1 << 8, 0), // len=8: round(x * 255 / 255) = (x * 256 + 0) >> 8
(255 << 8, 0), // len=1: round(x * 255 / 1) = (x * 65280 + 0) >> 8
(85 << 8, 0), // len=2: round(x * 255 / 3) = (x * 21760 + 0) >> 8
(9344, 0), // len=3: round(x * 255 / 7) = (x * 9344 + 0) >> 8
(17 << 8, 0), // len=4: round(x * 255 / 15) = (x * 4352 + 0) >> 8
(2108, 92), // len=5: round(x * 255 / 31) = (x * 2108 + 92) >> 8
(1036, 132), // len=6: round(x * 255 / 63) = (x * 1036 + 132) >> 8
(516, 0), // len=7: round(x * 255 / 127) = (x * 516 + 0) >> 8
];
Comment on lines +768 to +776
Copy link
Copy Markdown
Member

@197g 197g Apr 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do believe many of these are much more apparent in hex. Like 85 = 0x55 is intuitively right and not weird. Obviously there are weirder cases but even for 6bit seeing 0x40c + 0x84 is 'simpler' to correlate with the arithmetic than the decimal variant. I think it also gets rid of the need to write only some of these with a bitshift, i.e. 85 << 8 should be written as 0x5500 and 9344 as 0x2480, the 4-bit case as 0x1100 etc.

Copy link
Copy Markdown
Member Author

@RunDevelopment RunDevelopment Apr 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so you were one that made the previous constants hex. I was wondering who did that, because I don't find them to be intuitive at all in hex :)

Hex constants are a bad fit here IMO. The multiply-add method (MAM) is based on integer approximations for linear functions with rational parameters. MAM only uses bitshifts for fast division to approximate rationals. In fact, there's nothing special about division by powers of two. Any integer power will work. So representing MA constants as hex does not make their function more apparent. You can reinterpret their function in a different context, but that has nothing to do with MAM itself.

Take 85, for example. That's just 255 / 3. Very natural in decimal, no? Yes, there is the alternate interpretation that 85 = 0b01010101 duplicates bits, but that has nothing to do with MAM.

Copy link
Copy Markdown
Member

@197g 197g Apr 4, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connection to hex is that 255 = 1<<8 - 1; the 2**n ± 1 connection makes it natural for me to use a notation where bits stand out. It's certainly not completely arbitrary in a mathematical sense; plus the whole conversion sequence ends with a fixed-point number 0p1 in base 2^8. The reason to prefer base 2^4 over base 2^3 or 2^1 is, apart from slightly simpler fixed-point interpretation, convenience—in that there are probably more IT people fluent in that base than another. Octal would honestly be fine with me, too, binary is too verbose (for the same reason some folk used base-12 but not fewer civilizations base smaller than 10).

Copy link
Copy Markdown
Member Author

@RunDevelopment RunDevelopment Apr 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not saying there isn't a connection. I'm saying that connection doesn't matter for the multiply-add method.

The multiply-add method works by using rational linear functions. For any MAM problem (e.g. round(x * 255 / 3) for x in 0..=3) there exists an infinite set of rational linear functions that solve the problem. We just typically pick functions with coefficients of the form f/2^s * x + a/2^s because hardware is good at dividing by powers of two. If hardware was good at dividing by prime numbers, we'd pick differently.

Choosing to interpret these numbers as fixed-point numbers base two has no advantage, but misleadingly suggests a relevant connection that does not exist.

Copy link
Copy Markdown
Member

@197g 197g Apr 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be taking the notation on the 256-base fixed-point argument alone which you seem to think is easier for at least some, having written 1,2,4 as _ << 8. I think that applies to all and hex obviates the need of switching notation.

And while you could use arbitrary functions very clearly this specific one is a linear one and so I am not buying the argument of arbitrariness. The coefficient matching a slope of roughly 0xff.80/(2^n - 1) is required for this form to work; this heritage is definitely not misleading, it's the first simplification of the necessary & sufficient inequality criteria. At least for me that quotient is simple to grok in hex and awful to do in decimal.

Copy link
Copy Markdown
Member Author

@RunDevelopment RunDevelopment Apr 4, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not buying the argument of arbitrariness

Okay, then I'll explain the MAM a bit more. One way to formulate MAM is this:

Given an expression of the form $\lfloor (x\cdot t+r_d)/d\rfloor$ and an input range $u$ where $u\in\N_1, t\in\N, d\in\N_1, r_d\in\N, r_d&lt;d, x\in\N, x\le u$, find a tuple $(f,a,s)\in\N^3$ such that $\lfloor (x\cdot t+r_d)/d\rfloor = \lfloor (x\cdot f+a)/2^s\rfloor$.

This should be familiar. There are a few ways to find these solution tuples. If you go a more traditional number-theory route with fixed-point arithmetic, you'll get to a fairly well-known result: $f=\lceil t/d\cdot 2^s \rceil$, $a=\lceil r_d/d\cdot 2^s \rceil$, and $s=\lceil \log_2 d + \log_2(u+1) \rceil$. This works but it's an incomplete picture of the solution space.

A slightly modified version of MAM makes it easy to see the whole solution space. Let $(m,n)\in\R^2$ be a solution iff $\lfloor (x\cdot t+r_d)/d\rfloor = \lfloor xm+n\rfloor$. The connection to $(f,a,s)$ should be clear: $(f,a,s)$ is a solution if $(m,n)=(f/2^s,a/2^s)$ is a solution.

For example, for the problem round(x • 255 / 31) for x in 0..=31, the set of all solutions $(m,n)$ looks like this (white pixels):

image

(The magenta vertical line marks $m=255/31\approx 8.2258$. The horizontal magenta line marks $n=15/31\approx 0.48387$, which comes from $round(x \cdot 255 / 31) = \lfloor (x \cdot 255 + 15) / 31\rfloor$.)

This (half-open) polygon is the true nature of MAM. Any point $(m,n)$ within it is a solution.

This is why I said that MAM doesn't have much of a connection with fixed-point or anything base-two really. Fundamentally, MAM is about finding a point in a polygon. The points we pick with solutions $(f,a,s)$ only look like fixed-point numbers, because they represent rational points $(m,n)=(f/2^s,a/2^s)$. But the important property isn't that the denominator is a power of two, but that the rational numbers represent a point inside the polygon. We could have picked points of the form $(m,n) = (p/1234,q/1234)$ for $p,q\in\N$ if we wanted to (e.g. ((x as u32 * 10154 + 560) / 1234) as u8 also works as a 5- to 8-bit unorm conversion).

This is why I dislike representing the constants as fixed-point or hex so much. IMO they suggest an important connection to something related to powers of two or base two, but there's nothing there.


const fn from_len_shift(len: u32, shift: u32) -> Self {
debug_assert!(len <= 8);
debug_assert!(shift + len <= 32);
Bitfield {
shift,
len,
factor_addend: Self::FACTOR_ADDEND[(len % 8) as usize],
}
}

fn from_mask(mask: u32, max_len: u32) -> ImageResult<Bitfield> {
if mask == 0 {
return Ok(Bitfield { shift: 0, len: 0 });
return Ok(Bitfield::from_len_shift(0, 0));
}
let mut shift = mask.trailing_zeros();
let mut len = (!(mask >> shift)).trailing_zeros();
Expand All @@ -789,23 +801,19 @@ impl Bitfield {
shift += len - 8;
len = 8;
}
Ok(Bitfield { shift, len })
Ok(Bitfield::from_len_shift(len, shift))
}

#[inline]
fn read(&self, data: u32) -> u8 {
let data = data >> self.shift;
match self.len {
0 => 0,
1 => ((data & 0b1) * 0xff) as u8,
2 => ((data & 0b11) * 0x55) as u8,
3 => LOOKUP_TABLE_3_BIT_TO_8_BIT[(data & 0b00_0111) as usize],
4 => LOOKUP_TABLE_4_BIT_TO_8_BIT[(data & 0b00_1111) as usize],
5 => LOOKUP_TABLE_5_BIT_TO_8_BIT[(data & 0b01_1111) as usize],
6 => LOOKUP_TABLE_6_BIT_TO_8_BIT[(data & 0b11_1111) as usize],
7 => (((data & 0x7f) << 1) | ((data & 0x7f) >> 6)) as u8,
8 => (data & 0xff) as u8,
_ => panic!(),
}
debug_assert!(self.len <= 8);

// This performs branch-less UNORM conversion using the multiply-add
// method. See `FACTOR_ADDEND` above for more information.
let (factor, addend) = self.factor_addend;
let mask = (1 << self.len) - 1;
let data = (data >> self.shift) & mask;
((data * factor + addend) >> 8) as u8
}
}

Expand Down Expand Up @@ -2185,7 +2193,7 @@ mod test {
#[test]
fn test_bitfield_len() {
for len in 1..9 {
let bitfield = Bitfield { shift: 0, len };
let bitfield = Bitfield::from_len_shift(len, 0);
for i in 0..(1 << len) {
let read = bitfield.read(i);
let calc = (f64::from(i) / f64::from((1 << len) - 1) * 255f64).round() as u8;
Expand Down
Loading