sed in-place write unsafe (was: Power-fail safe writes)

Denys Vlasenko vda.linux at googlemail.com
Wed Apr 3 13:54:49 UTC 2019


On Wed, Apr 3, 2019 at 12:07 PM Cristian Ionescu-Idbohrn
<cristian.ionescu-idbohrn at axis.com> wrote:
>
> Denys,
>
> On Tue, 2 Apr 2019, Sebastian Brand wrote:
> >
> > I recently had problems when updating a configuration file using sed
> > with in-place edit option (-i), shortly followed by a power fail,
> > which has the end result of an empty configuration file and a system
> > that won't boot.
> > The reason behind this is me relying on sed and sed in-place editing
> > not using a power-fail safe write pattern (copy, edit copy,
> > synchronize copy, rename, synchronize folder).
>
> I'd be interested in your comment about this patch, which fsyncs the
> created temporary file and the directory (after rename) before it's
> done.  Any chance it can end up in upstream busybox in this form or
> another?

Attempts to fix such problems via fsync'ing in generic utilities
usually don't work out long term - this causes performance regressions
when someone wants e.g. to process 60000 files.

Trying to convince thousands of userspace developers
that they are not careful enough with fsync'ing is futile:
a lot of them will still be not careful enough.

ext4 developers were eventually convinced to change fs code
so that the usual "write new file + rename over old one"
is reliable wrt power failure: rename implies fsync,
and rename either succeeds completely, or does not
happen at all.

I take it, your filesystem is not doing that?

I propose, for modification of config files,
to use a wrapper script around sed,
which directs sed result into a new file,
then sync's new file, then renames it over old file,
then sync's the directory.


More information about the busybox mailing list