sed in-place write unsafe (was: Power-fail safe writes)

Cristian Ionescu-Idbohrn cristian.ionescu-idbohrn at axis.com
Wed Apr 3 15:45:27 UTC 2019


On Wed, 3 Apr 2019, Denys Vlasenko wrote:
> On Wed, Apr 3, 2019 at 12:07 PM Cristian Ionescu-Idbohrn <cristian.ionescu-idbohrn at axis.com> wrote:
> >
> > Denys,
> >
> > On Tue, 2 Apr 2019, Sebastian Brand wrote:
> > >
> > > I recently had problems when updating a configuration file using sed
> > > with in-place edit option (-i), shortly followed by a power fail,
> > > which has the end result of an empty configuration file and a system
> > > that won't boot.
> > > The reason behind this is me relying on sed and sed in-place editing
> > > not using a power-fail safe write pattern (copy, edit copy,
> > > synchronize copy, rename, synchronize folder).
> >
> > I'd be interested in your comment about this patch, which fsyncs the
> > created temporary file and the directory (after rename) before it's
> > done.  Any chance it can end up in upstream busybox in this form or
> > another?
> 
> Attempts to fix such problems via fsync'ing in generic utilities
> usually don't work out long term - this causes performance regressions
> when someone wants e.g. to process 60000 files.

True.

> Trying to convince thousands of userspace developers
> that they are not careful enough with fsync'ing is futile:
> a lot of them will still be not careful enough.

Also true.

> ext4 developers were eventually convinced to change fs code
> so that the usual "write new file + rename over old one"
> is reliable wrt power failure: rename implies fsync,
> and rename either succeeds completely, or does not
> happen at all.

Yes.

> I take it, your filesystem is not doing that?

Ubifs.  Appears upstream does not seem interested of fixing it, yet.

> I propose, for modification of config files,
> to use a wrapper script around sed,
> which directs sed result into a new file,
> then sync's new file, then renames it over old file,
> then sync's the directory.

Your comments are appreciated.


Cheers,

-- 
Cristian


More information about the busybox mailing list