Repository navigation
Conversation
sahlberg
force-pushed
the
dm-era
branch
3 times, most recently
from
November 7, 2026 22:19
55652d4 to
6554a63
Compare
For keeping a copy that nobody else changes up to date (e.g. backups of
VM images) when the source filesystem sits on a Linux dm-era device.
Instead of reading both files and computing checksums, the generator
sends an "era request" -- a sum head with no strong sums, followed by
the destination file's user.rsync.era xattr ("<era> <time>
<era-device-id>") -- and the sender looks up which blocks of the source were written in
that era or later (era.c, using era_invalidate on a metadata snapshot).
It answers with block references for unchanged blocks and literal data
for the rest, so only changed source data is read and the receiver
neither reads nor rewrites unchanged blocks. The whole-file checksum
covers only the literal data, on both sides.
With every file in --era mode the sender also sends the era the
destination will be current with: the era started by the snapshot used
for it, or for a normal transfer the newest logged era. The receiver
removes the xattr before changing a file and sets it again once the
checksum matches, so an interrupted file has no era and gets a normal
update next time. Each file is therefore only re-sent the changes made
since it was itself last updated. Each era device has a log of the
eras our snapshots started and when (/var/lib/rsync/era/<device-id>,
also the lock serializing runs); a file's era is only accepted if the
log has it with the same time, so era numbers that start over after the
era metadata is recreated cannot match.
-X neither copies nor removes the xattr. A run takes at most 8
snapshots per device; after that a file reuses the bitmap of the
newest older era, a superset.
A file without an era, or whose era is from another device or not in
the log, uses normal sums. If the sender cannot handle a requested file
(not on dm-era, ...) it declines with one byte; the receiver leaves the file untouched and asks for a redo, which
uses the normal algorithm and never sends another era request. A
missing era_invalidate on the sender, or a destination without user
xattrs, is a hard error.
--era is only built for standard Linux (not Android or other systems)
and is needed on both sides. It implies --inplace and
--no-whole-file, conflicts with --whole-file, --sparse, --append,
--write-devices and batch mode, and requires protocol 30.
A file written around the time of the snapshot (e.g. a running VM's
disk) is still sent from the era bitmap: writes the snapshot missed are
in its new era or later, which is what the destination records. Its
file-list mtime may include such writes, so the receiver gives the
destination an mtime 1 + --modify-window seconds earlier, and the next
run's quick check sends it again.
Era requests use 16 KiB blocks: block references are free to compute,
and holes and unwritten extents (always sent) are rounded out to whole
blocks. rsync --version lists "dm-era" (or "no dm-era") among its
capabilities.
Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Not sure if this belongs in rsync or it should be a separate tool, but putting it here for discussions.
The default way rsync operates makes it very challenging to handle very large files with small change-sets, for example very large databases or vm images. You read and compute hashes across a huge file only to discover that you only need to transfer a trivial amount of data.
--era adds a different mechanism to discover what data has changed and needs to be transferred. It is linux-only and is based on dm-era. Other platforms, like Windows, have similar features in their block layers.
It requires the source filesysystem to be ontop of a dm-era device and it requires that the destination filesystem is xattr capable. Xattrs are used to store the era of the last sync.
It can not detect if the remote files have been modified by external tools, one later addition to mitigate that risk could be to trying to keep the remote files as immutable except for when an incremental rsync is in process. That is not implemented and it would add a dependency that "remote filesystem must support chattr +i".
It is thus not really a general purpose feature but something that is only useful for "vault" or "backup" style operations where you want to have a (mostly) read-only copy/backup of the files.
I am using it for my very large VM images and it works exceptionally well for my use case.
But again, it is definitely not a general purpose feature so I am not sure how well it would fit in upstream rsync.
Comments?