[Bug 585] New: redirect fails to handle certain UTF-8 characters

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=585

            Target: Oleg custom firmware for Asus WL-500 router
             Build: busybox 1.14.3
           Summary: redirect fails to handle certain UTF-8 characters
           Product: Busybox
           Version: 1.14.x
          Platform: Other
               URL: http://code.google.com/p/wl500g/
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Standard Compliance
        AssignedTo: unassigned at busybox.net
        ReportedBy: ecaddict at gmail.com
                CC: busybox-cvs at busybox.net
   Estimated Hours: 0.0


When via SSH the following command is given:
echo "test" > /mnt/testÁ.txt && /opt/bin/find -L /mnt -maxdepth 1 -ls | grep
/test

It produces the following output:
54    4 -rw-r--r--   1 admin    root            5 Aug 28 18:26
/mnt/test\303.txt

That is in the file name after the "test" there is the UTF-8 character with
code 193 (equals 0xC1, that can be encoded in 2 bytes as 0xC3 0x81). However
the created file contains only the first byte of the UTF-8 encoding.
When checked via NFS share, the file name shows also incorrectly. Only some of
the UTF-8 characters fails, most of them works OK (so this could be some small
bug not a full gap of functionality).
As far as I could judge, characters that can be encoded in 3 bytes are impacted
as well (I did not try outside of BMP).

Files containing the same character can be correctly created with touch, so
continuing in the command line:

touch /mnt/test2Á.txt && /opt/bin/find -L /mnt -maxdepth 1 -ls | grep /test

gives:
    54    4 -rw-r--r--   1 admin    root            5 Aug 28 18:26
/mnt/test\303.txt
    55    0 -rw-r--r--   1 admin    root            0 Aug 28 18:29
/mnt/test2\303\201.txt

In this case the test2 file shows the correct octal escape sequence.

What makes this bug particularly nasty is that it creates hard to delete (at
least from command line) files.

I've use the following reference for UTF-8 values:
http://doc.infosnel.nl/extreme_utf-8.html

and this for general reference:
http://www.cl.cam.ac.uk/~mgk25/unicode.html


-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.