kvdb-overlay

kvdb-overlay

This Rust library serves as a minimal overlay mechanism for various key-value embedded databases.

This mechanism enables us to simulate changes in a database/tree so that keys and values can be dynamically mutated, while avoiding having to change the underlying database. With this, we can perform changes to the trees and access the latest changes in-memory, and then only when we're satisfied with the results, we can actually atomically write it into the actual database.

This functionality can also serve as a rollback-like mechanism for the database.

Currently supported databases:

• sled
• fjall

Usage examples are offered in the repository as test units, and docs can be found on docs.rs/kvdb-overlay.

Reading Without an Extra Copy

Tree::get, TreeOverlay::get, and DatabaseOverlay::get return a Value handle without copying the bytes into a Vec<u8>:

if let Some(value) = overlay.get("my_tree", b"key")? {
    let bytes: &[u8] = value.as_ref();
    // Pass bytes to a slice deserializer, or use Cursor::new(value)
    // with a deserializer that accepts std::io::Read.
}

Cached values borrow the overlay; backend values retain an owned backend handle. Keep the value alive while borrowing its bytes. An overlay cannot be mutated while its returned value is still in use. Tree::get returns an owned handle that can outlive the tree.

To obtain an independent buffer, consume the handle with into_vec():

let owned: Option<Vec<u8>> = overlay.get("my_tree", b"key")?.map(|value| value.into_vec());

This explicitly copies the bytes, allowing the vector to outlive overlay mutation or destruction. clone() and to_owned() clone the handle, not the bytes; they do not detach cached values from the overlay. This is a breaking change from the previous vector-returning get; the separate get_value method has been removed. Other APIs such as insert, remove, and iterators still return owned vectors.

Reading through the handle avoids an extra value-buffer allocation and copy. It does not stream from disk or avoid loading the entire value into memory; backend reads and deserialization may still allocate. Wrapping the value in a synchronous or async cursor does not change that.

License

GNU AGPLv3.