# MinIO MemKV RELEASE.2026-09-17T15-27-52Z Released: 2026-09-17 The monitoring console UI now ships in the binary, a chunked overwrite of an already-persisted key no longer serves a torn value, a valid ENTERPRISE license with a zero account ID is accepted, the LMCache plugin imports again on LMCache 0.5.4 and newer, and RC data-path failures name their cause. --- ## Downloads ### Server Binary | Platform | Architecture | Download | | -------- | ------------ | -------- | | Linux | amd64 | [memkv](https://dl.min.io/aistor/memkv/release/linux-amd64/memkv) | | Linux | arm64 | [memkv](https://dl.min.io/aistor/memkv/release/linux-arm64/memkv) | ### NIXL Plugin (for Dynamo / KVBM integrations) | Platform | Architecture | Download | | -------- | ------------ | -------- | | Linux | amd64 | [libplugin_MEMKV.so](https://dl.min.io/aistor/memkv/release/linux-amd64/libplugin_MEMKV.so) | | Linux | arm64 | [libplugin_MEMKV.so](https://dl.min.io/aistor/memkv/release/linux-arm64/libplugin_MEMKV.so) | ### LD_PRELOAD Shim (for MLPerf-Storage kvcache workloads) | Platform | Architecture | Download | | -------- | ------------ | -------- | | Linux | amd64 | [libmemkv_preload.so](https://dl.min.io/aistor/memkv/release/linux-amd64/libmemkv_preload.so) | | Linux | arm64 | [libmemkv_preload.so](https://dl.min.io/aistor/memkv/release/linux-arm64/libmemkv_preload.so) | ### Packages `.deb`, `.rpm`, and `.apk` packages bundle the server + both `.so` sidecars + the LMCache, sglang, and vLLM Python wheels into a single per-arch install. | Format | Architecture | Download | | ------ | ------------ | -------- | | DEB | amd64 | [memkv\_20260917152752.0.0_amd64.deb](https://dl.min.io/aistor/memkv/release/linux-amd64/memkv_20260917152752.0.0_amd64.deb) | | DEB | arm64 | [memkv\_20260917152752.0.0_arm64.deb](https://dl.min.io/aistor/memkv/release/linux-arm64/memkv_20260917152752.0.0_arm64.deb) | | RPM | amd64 | [memkv-20260917152752.0.0-1.x86_64.rpm](https://dl.min.io/aistor/memkv/release/linux-amd64/memkv-20260917152752.0.0-1.x86_64.rpm) | | RPM | arm64 | [memkv-20260917152752.0.0-1.aarch64.rpm](https://dl.min.io/aistor/memkv/release/linux-arm64/memkv-20260917152752.0.0-1.aarch64.rpm) | | APK | amd64 | [memkv\_20260917152752.0.0_x86_64.apk](https://dl.min.io/aistor/memkv/release/linux-amd64/memkv_20260917152752.0.0_x86_64.apk) | | APK | arm64 | [memkv\_20260917152752.0.0_aarch64.apk](https://dl.min.io/aistor/memkv/release/linux-arm64/memkv_20260917152752.0.0_aarch64.apk) | The NIXL plugin is auto-symlinked to `/opt/nvidia/nvda_nixl/lib/plugins/` when that directory exists (postinstall hook). ### Python Plugins Install from PyPI, for amd64 and arm64: ```bash pip install -U memkv-lmcache memkv-sglang memkv-vllm ``` PyPI serves **1.0.8** for all three, for amd64 and arm64. `memkv-lmcache` 1.0.8 is the one that matters this release — it carries the LMCache compatibility fix below, requires Python 3.10–3.13, and pulls `lmcache>=0.5.4,<0.6`. `memkv-sglang` (Python 3.8+) and `memkv-vllm` (Python 3.10+, vLLM 0.25.0+) are unchanged in content. The wheels `dlopen` the RDMA libraries at runtime; the transport needs host `rdma-core`, otherwise it falls back to TCP. Without network access, the deb/rpm installs the same code at `/usr/share/memkv/wheels/`, stamped **1.0.7** — the version the tagged packages were built with. Those wheels carry the fix, but their version string matches the older, broken 1.0.7 that was on PyPI, so pip would treat them as already installed and do nothing: ```bash cd /usr/share/memkv/wheels pip install --force-reinstall memkv_lmcache-*.whl memkv_sglang-*.whl memkv_vllm-*.whl ``` Where the host has network access, prefer 1.0.8 from PyPI. ### Container Image ```bash docker pull quay.io/minio/memkv:RELEASE.2026-09-17T15-27-52Z docker pull quay.io/minio/memkv:latest ``` The container ships the server and the NIXL plugin (under `/usr/local/lib/plugins/`). For the LD_PRELOAD shim and the Python wheels, use the deb/rpm. ### Verification Each binary is signed with both minisign (preferred) and GPG; sha256sums are published alongside. ```bash # minisign minisign -Vm memkv -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav # sha256 sha256sum -c memkv.sha256sum ``` --- ## Changes since RELEASE.2026-08-14T18-23-15Z ### New Features - **The monitoring console UI ships in the binary.** The release build produced the docs bundle but never the console bundle, so the previous release embedded an empty blob and the admin server's `/` route served a "built without the monitoring console" placeholder — the 1m/1h/24h ring buffers and `GET /v1/metrics/history` were live, only the UI was missing. The admin server now serves the real console: throughput, latency, memory, storage, RDMA, eviction, and cache hit rate, charted over all three windows. Nothing to configure; browse the admin port. This resolves the known limitation recorded against the previous release. A bundle the release build was asked to produce and that comes up missing or empty is now a hard build failure naming the bundle, rather than a cargo warning in a cross-compile log. ### Bug Fixes - **A chunked overwrite of a persisted key could serve a torn value as a successful read.** On FileStore, a chunked or streamed overwrite patched the live chain in place, chunk by chunk, and published the new larger length before any chunk had landed. A chunk that failed partway — an RDMA read error, a dropped stream — left part new bytes and part zero-filled tail, returned as a successful `GET`; a concurrent reader could observe the same mix even when the write went on to succeed. The replacement is now built in a separate chain that is not reachable until every chunk has committed, then swapped in atomically; a failed or abandoned overwrite tears down only that chain and never touches the value it would have replaced. JBOF already made in-place overwrite safe through per-extent rollback and journaling, and takes the unchanged path. One behavior change comes with it: because an overwrite now allocates a genuine fresh chain, the storage saturation guard covers it. A growing overwrite near capacity is rejected with `NoSpace` where it previously bypassed the guard entirely — which is the typed signal the client's TCP-streaming fallback keys off. - **LMCache 0.5.4 and newer broke the LMCache plugin at import.** LMCache moved `Bitmap` out of `lmcache.native_storage_ops` and into `lmcache.lmcache_native`, which the adapter imported unconditionally, so on a current LMCache the adapter raises `ImportError` and the plugin does not load. The import now tries the new location and falls back to the old one. The plugin also now declares what it supports: `lmcache>=0.5.4,<0.6` and Python 3.10–3.13. That narrows the range as well as fixing it — pip will refuse the wheel on LMCache 0.5.3 or older, which the fallback branch would otherwise have served. Published as **1.0.8** on PyPI; the tagged deb/rpm carries the same fix in wheels stamped 1.0.7 — see Python Plugins above. - **A valid ENTERPRISE license with a zero account ID was rejected.** SUBNET issues licenses with `aid: 0`, and the license library rejected any account ID that was not strictly positive, so a license AIStor accepted made MemKV fail at startup with `license contains no or invalid account ID`. Only negative account IDs are refused now. ### Improvements - **RC data-path failures name their cause.** On the RC fallback path — any non-mlx5 pairing, where DC is unavailable — a server-side RDMA read failure surfaced to the operator as `Worker error: RDMA send failed: status=5` after a 30-second stall, with a server log line reporting a timeout that had not happened. Three things hid the real fault: connection teardown was logged as a timeout regardless of what the completion actually said, the client's cached connection lookup skipped the health check its other path performs and kept handing out a connection it knew was dead, and `memkv bench` discarded warmup errors — where the first, informative failure lands. The originating completion status is now what gets logged, a failed connection is not reused, and warmup failures are reported. Diagnosis only; see Known Limitations for what this does not fix. ### Known Limitations - **A server-side data-path failure still costs the client a 30-second timeout.** Per the IB spec, a send-queue error completion puts an RC QP into ERROR, so the server has no QP left on which to return the error response it builds — the client waits out the full RDMA timeout before giving up. An out-of-band failure signal is the fix and has not been built. Relatedly, message size is validated nowhere in the client or server, which is the leading suspect for the `LOC_QP_OP_ERR` that prompted the logging work above; that root cause is still open. - **A chunked overwrite can still lose earlier chunks under capacity pressure.** Whole-value writes hold a per-entry gate that reclaim respects, but a chunked overwrite spans several calls, each gated individually. A gap of more than ~500 ms between chunks while the tier is above the high watermark can let reclaim free the chain in between. Closing it needs a per-key write lease across calls. Distinct from the torn-read fix above: that closes the writer-versus-reader race on the live chain, while eviction or delete racing a reader that has already resolved key to entry is unchanged and needs a separate reclamation project. - **InfiniBand addressing is same-subnet only.** Traffic crossing subnets needs a global route to the local router, which is refused rather than mis-addressed. When the subnet manager reassigns a LID, established sessions fail and reconnect; recovery is the reconnection, not transparent. Both the data path and the LID-reassignment path have been validated on InfiniBand hardware on Nebius; CI itself remains SoftRoCE. - **`MemKVOffloadingSpec` scope.** Single KV cache group; an offloaded block is one GPU block; keys are rank-qualified, so a restore requires the same TP topology that stored them; RDMA only, with no TCP fallback on this path. - **The two vLLM integrations are not interchangeable.** `TieringOffloadingSpec` keys on vLLM `FileMapper` path names and `MemKVOffloadingSpec` on a binary prefix with rank-qualified keys, so a cluster written by one is not read by the other. Pick one per deployment. --- ## Documentation - Hosted docs: - Embedded docs (in the binary): `memkv doc` serves the same site locally. The plugin pages now document the license requirement and the lookup order the client actually follows, the vLLM page covers both offload modes rather than only the tier, and inference performance has its own page under Internals. ## Support - Security disclosures: security@min.io - Licensed customers and contributors: contact MinIO support.