Deserialize directly into a ByteBuffer

[cheddar]

It'd be awesome to be able to deserialize directly into a provided ByteBuffer. For this, I think it makes sense to make the assumption that the ByteBuffer is large enough to hold the output and if it's not, the code can do whatever it wants as long as I can figure out that my ByteBuffer wasn't large enough somehow.

[cowtowncoder]

Looking at code, I think the first thing would be to make ChunkDecoder allow uncompressing into ByteBuffer.
Alternative combinations could include input coming from ByteBuffer, or both input and output being ByteBuffer. Amount of code involved is actually not that big, even with 3 new methods (of which 2 are of course optional for now, since request is for uncompressing into buffer).

After this, there would be question of feeding things incrementally -- that is, finding alternatives to InputStream and OutputStream. For this, block-by-block operation might work best. Output side of things (relevant here) should actually be quite simple; using ByteBuffer for input more complex but can be deferred.

[cowtowncoder]

Thinking about this bit more, I think both compression and decompression could perhaps be best handled via callback interfaces, similar to Uncompressor.

[carryxyh]

Is there any progress on this issue? We really need this feature. :)

In addition, we also hope to provide the ability to write compressed data directly into ByteBuffer.

[cowtowncoder]

I don't think anyone has worked on this. Contributions welcome!

[carryxyh]

Hi, sorry for late..

I submitted a pull request to Netty. I copied part of the code and made some modifications so that netty can write data directly to ByteBuf with lzf:

netty/netty#9876

The problem is that I only modified VanillaChunkEncoder. It seems that UnsafeChunkEncoder is more complicated for this situation.

For the scenario of using ByteBuffer under Unsafe, how can we achieve it? I have no plan yet...

[carryxyh]

Since DirectByteBuffer itself uses unsafe for reading and writing, it seems that we do not need to resort to additional Unsafe... Let me try it...