[rfc] sed option `-i' (edit in place)

[Johan Adolfsson]

> -----Original Message-----
> From: busybox-bounces at busybox.net 
> [mailto:busybox-bounces at busybox.net] On Behalf Of Cristian 
> Ionescu-Idbohrn
> Sent: den 26 augusti 2007 23:21
> To: busybox at busybox.net
> Subject: Re: [rfc] sed option `-i' (edit in place)
> 
> 
> On Sun, 26 Aug 2007, Denys Vlasenko wrote:
> 
> > On Sunday 26 August 2007 16:00, Cristian Ionescu-Idbohrn wrote:
> > > > > Embedded systems usualy place editable files on flash 
> fs and those
> > > > > fs get used out very much faster compared to real hdfs.
> >
> > I think that sed creates new file, fills it and and then renames
> > new file into old file so fast that first metadata update doesn't
> > even have time to hit the disk storage before it's all done.
> 
> I believe that may be the case with most fs, but not jffs2.  But don't
> take my word for it.
> 
> > What exactly are you trying to optimize out here?
> 
> The shell?
> 
> > Why do you think your solution will be any different from 
> what sed does
> > now wrt storage wear?
> 
> I can't say I _have_ a solution.  I'm just thinking loud ;)
> 
> > > > I think you mean
> > > >
> > > > sed ... <"$file" >/ramfs/tmp-$$ && mv /ramfs/tmp-$$ "$file"
> > >
> > > If one would prefer using mktemp (mkstemp) instead of 
> tmp-$$, it would
> > > get more complicated and inefficient.
> > >
> > > > This is what shells are for - to avoid bloating 
> programs with all
> > > > imaginable small improvements.
> > >
> > > But of course.  That is one way to do it, but is it as 
> efficient as
> > > having sed do it in one go?  What about combining sed and 
> mv code to
> > > achieve the same?
> >
> > Well, what about -R option - do it recursively over a directory?
> > and -find_name '*pattern*' to act only on a subset of files?
> 
> Sorry, I don't follow.  "-R option" to what?
> 
> > I don't like this kind of creeping featuritis.
> 
> I agree.  Still, there are a lot of embeded systems out there using
> busybox and flash storage.
> 
> > Does stock sed do this?
> 
> No.  GNU sed is doing something similar to bb-sed (or the other way
> around):
> 
> ,----
> | open("lines", O_RDONLY|O_LARGEFILE) = 3
> | ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfd35068) = -1 
> ENOTTY (Inappropriate ioctl for device)
> | fstat64(3, {st_mode=S_IFREG|0644, st_size=8, ...}) = 0
> | gettimeofday({1188158729, 155778}, NULL) = 0
> | getpid()                                = 19852
> | open("./sedSTqTlj", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = 10
> | fcntl64(10, F_GETFL)                    = 0x8002 (flags 
> O_RDWR|O_LARGEFILE)
> | fstat64(10, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
> | mmap2(NULL, 131072, PROT_READ|PROT_WRITE, 
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7dfe000
> | _llseek(10, 0, [0], SEEK_CUR)           = 0
> | fchmod(10, 0100644)                     = 0
> | fchown32(10, 1000, 1000)                = 0
> | fstat64(3, {st_mode=S_IFREG|0644, st_size=8, ...}) = 0
> | mmap2(NULL, 131072, PROT_READ|PROT_WRITE, 
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7dde000
> | read(3, "fxx\nbar\n", 131072)           = 8
> | write(10, "fxx\n", 4)                   = 4
> | write(10, "bar\n", 4)                   = 4
> | read(3, "", 131072)                     = 0
> | close(3)                                = 0
> | munmap(0xb7dde000, 131072)              = 0
> | close(10)                               = 0
> | munmap(0xb7dfe000, 131072)              = 0
> | rename("./sedSTqTlj", "lines")          = 0
> | close(1)                                = 0
> | close(2)                                = 0
> `----

Wouldn't we get reduced wearing and possibly a general speed up as well,
if we made sure sed was buffering stuff more and avoiding small writes
to the filesystem?
Then there shouldn't be much difference in flash wearing compared to a copy.

Just a thought.
/Johan