Hi,
I'm running into a weird issue where calling nioBuffer on a ByteBuf received from lmdb-java results in a NIO ByteBuffer with the desired size but a value of all zeros.
I'm on Netty 4.1.90.Final and lmdbjava 0.8.3 and JDK 19 . The repro is as follows:
package foo;
import io.netty.buffer.ByteBufUtil;
import io.netty.buffer.PooledByteBufAllocator;
import io.netty.buffer.Unpooled;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.lmdbjava.ByteBufProxy;
import org.lmdbjava.DbiFlags;
import org.lmdbjava.Env;
import java.io.File;
import static java.nio.charset.StandardCharsets.UTF_8;
import static org.junit.jupiter.api.Assertions.assertNotNull;
public class ByteBufMinimalReproTest {
@TempDir
File tmp;
@Test
void minimalRepro() {
var env = Env.create(ByteBufProxy.PROXY_NETTY)
.setMapSize(10_485_760)
.setMaxDbs(1)
.open(tmp);
var db = env.openDbi("testdb", DbiFlags.MDB_CREATE);
var alloc = PooledByteBufAllocator.DEFAULT;
var key = alloc.directBuffer(env.getMaxKeySize());
var value = alloc.directBuffer(700);
key.writeCharSequence("greeting", UTF_8);
value.writeCharSequence("Hello World", UTF_8);
System.out.println("Sanity -- Key");
System.out.println(ByteBufUtil.prettyHexDump(key));
System.out.println("Sanity -- Key NIO");
System.out.println(ByteBufUtil.prettyHexDump(Unpooled.wrappedBuffer(key.nioBuffer())));
System.out.println("Sanity -- Value");
System.out.println(ByteBufUtil.prettyHexDump(value));
System.out.println("Sanity -- Value NIO");
System.out.println(ByteBufUtil.prettyHexDump(Unpooled.wrappedBuffer(value.nioBuffer())));
db.put(key, value);
// need a transaction to read values.
try (var txn = env.txnRead()) {
var found = db.get(txn, key);
assertNotNull(found);
System.out.println("Found Value");
System.out.println(ByteBufUtil.prettyHexDump(found));
System.out.println("Found Value - NIO (Bad: all zeros)");
System.out.println(ByteBufUtil.prettyHexDump(Unpooled.wrappedBuffer(
found.nioBuffer())));
var fetchedVal = txn.val();
System.out.println("val via txn");
System.out.println(ByteBufUtil.prettyHexDump(fetchedVal));
System.out.println("val via txn - NIO (Bad: all zeros)");
System.out.println(ByteBufUtil.prettyHexDump(Unpooled.wrappedBuffer(
txn.val().nioBuffer())));
}
env.close();
}
}
Which prints:
Sanity -- Key
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 67 72 65 65 74 69 6e 67 |greeting |
+--------+-------------------------------------------------+----------------+
Sanity -- Key NIO
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 67 72 65 65 74 69 6e 67 |greeting |
+--------+-------------------------------------------------+----------------+
Sanity -- Value
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 65 6c 6c 6f 20 57 6f 72 6c 64 |Hello World |
+--------+-------------------------------------------------+----------------+
Sanity -- Value NIO
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 65 6c 6c 6f 20 57 6f 72 6c 64 |Hello World |
+--------+-------------------------------------------------+----------------+
Found Value
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 65 6c 6c 6f 20 57 6f 72 6c 64 |Hello World |
+--------+-------------------------------------------------+----------------+
Found Value - NIO (Bad: all zeros)
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 00 00 00 00 00 00 |........... |
+--------+-------------------------------------------------+----------------+
val via txn
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 65 6c 6c 6f 20 57 6f 72 6c 64 |Hello World |
+--------+-------------------------------------------------+----------------+
val via txn - NIO (Bad: all zeros)
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 00 00 00 00 00 00 |........... |
+--------+-------------------------------------------------+----------------+
My repro is using the prettyHexDump from ByteBuf, which is why i need to re-wrap nioBuffer. I'm pretty sure this isn't causing any problems because when I write the bytebuf to file, I see all zeros.
The reason I'm doing nioBuffer on this netty buffer is because I need to pass it to Zstd java for decompression.
I'm new to both lmdb java and netty, and it's possible i'm doing something wrong, but I'd appreciate your help with this.
Hi,
I'm running into a weird issue where calling
nioBufferon aByteBufreceived fromlmdb-javaresults in aNIO ByteBufferwith the desired size but a value of all zeros.I'm on
Netty 4.1.90.Finalandlmdbjava 0.8.3and JDK 19 . The repro is as follows:Which prints:
My repro is using the prettyHexDump from ByteBuf, which is why i need to re-wrap nioBuffer. I'm pretty sure this isn't causing any problems because when I write the bytebuf to file, I see all zeros.
The reason I'm doing nioBuffer on this netty buffer is because I need to pass it to Zstd java for decompression.
I'm new to both lmdb java and netty, and it's possible i'm doing something wrong, but I'd appreciate your help with this.