Problem with test -r

Vladimír Macek (Scripteo) macek at scripteo.cz
Tue Feb 14 10:19:18 UTC 2023 

[Sorry, I accidentally e-mailed only alice yesterday. Here's the corrected 
and extended version.]

Thank you all for taking care!

Today I found out that the problem lies probably in my server docker 
runtime or underlying system than inside the container:

In attached out.txt:
I tested the command line on alpine containers 3.17 down to 3.12 and you 
can see, the bash worked correctly for me (READABLE-INTERNAL-BASH) only in 
3.13 and 3.12. There was no bash version change between alpine 3.13.12 and 
3.14, when "test -r" went bad for me. So... some rogue patch to bash may 
have caused this?

But there was a busybox version change at that time. There's no other 
change on my side that I'm aware of.

Regular user always executes docker run.

Dockerfile and Makefile are attached.
make image is used to build with varied alpine version in FROM,
make sh to enter the container and run commands shown in out.txt.
The scripts in scripts/ are not involved in the testing, so not attached.

The underlying filesystem (/var/tmp/done/project-job-dir-1095) is ext4 
(Debian 10 running on metal):

/var/tmp/done/project-job-dir-1095# find output/ -ls
  38537360      4 drwxr-x---   5 2001     2000 4096 úno 11 20:27 output/
  38537362      4 drwxrwx---   2 2001     2000 4096 úno 11 20:27 output/files
  38537366     24 -rw-r--r--   1 2002     2000 20512 úno 11 20:27 
output/files/cnb_rate_gbp_czk.csv
  38537365     24 -rw-r--r--   1 2002     2000 20512 úno 11 20:27 
output/files/cnb_rate_eur_czk.csv
  38537364     24 -rw-r--r--   1 2002     2000 20512 úno 11 20:27 
output/files/cnb_rate_usd_czk.csv

lsattr shows "--------------e----" for all these items.

# uname -a
Linux ...  4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64 
GNU/Linux

# dpkg -l docker.io | grep ii
ii  docker.io      18.09.1+dfsg1-7.1+deb10u3 amd64        Linux container 
runtime

BUT, i carefully transferred the job directory to my home Ubuntu system:

# uname -a
Linux Inspiron 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 
x86_64 x86_64 x86_64 GNU/Linux

# dpkg -l docker.io | grep ii
ii  docker.io      20.10.12-0ubuntu4 amd64        Linux container runtime

then the new bash in alpine 3.17 in container works CORRECTLY:

e7a19212db0f:/task/output/files$ echo $BASH_VERSION ; cat 
/etc/alpine-release ; busybox | head -1 ; cd /task/output/files/ ; [ -r 
cnb_rate_eur_czk.csv ] && echo READABLE-INTERNAL-BASH ; /usr/bin/[ -r 
cnb_rate_eur_czk.csv ] && echo READABLE-EXTERNAL ; /bin/sh -c 'echo 
$BB_ASH_VERSION ; [ -r cnb_rate_eur_czk.csv ] && echo READABLE-INTERNAL-SH'
5.2.15(1)-release
3.17.2
BusyBox v1.35.0 (2022-11-19 10:13:10 UTC) multi-call binary.
READABLE-INTERNAL-BASH
READABLE-EXTERNAL
1.35.0
READABLE-INTERNAL-SH

Damn... :-(

On 13. 02. 23 16:34, alice wrote:
> On Mon Feb 13, 2023 at 3:24 PM CET, Vladimír Macek (Scripteo) wrote:
>> Thanks everyone for important pointers!
>>
>> The problem seems to be brought by bash between alpine official container
>> 3.13 and any later versions.
> 3.13 has the same bash version as 3.16- 3.17 has bash 5.2.
>
> so, i assume it's new in 3.17 only.
>
>> The external command /usr/bin/[ and sh/ash from busybox both work
>> correctly, see:
>>
>> 5513a6a8232d:/task/output/files$ cat /etc/alpine-release
>> 3.17.1
>>
>> 5513a6a8232d:/task/output/files$ busybox | head -1
>> BusyBox v1.35.0 (2022-11-19 10:13:10 UTC) multi-call binary.
>>
>> 5513a6a8232d:/task/output/files$ echo $BASH_VERSION
>> 5.2.15(1)-release
>>
>> 5513a6a8232d:/task/output/files$ [ -r cnb_rate_eur_czk.csv ] && echo READABLE
>> 5513a6a8232d:/task/output/files$ [[ -r cnb_rate_eur_czk.csv ]] && echo READABLE
>>
>> 5513a6a8232d:/task/output/files$ /usr/bin/[ -r cnb_rate_eur_czk.csv ] &&
>> echo READABLE
>> READABLE
>> 5513a6a8232d:/task/output/files$ /usr/bin/[[ -r cnb_rate_eur_czk.csv ]] &&
>> echo READABLE
>> READABLE
>>
>> 5513a6a8232d:/task/output/files$ ls -la /bin/*sh
>> lrwxrwxrwx    1 root     root            12 Jan  9 12:46 /bin/ash ->
>> /bin/busybox
>> -rwxr-xr-x    1 root     root        789064 Dec 14 02:37 /bin/bash
>> lrwxrwxrwx    1 root     root            12 Jan  9 12:46 /bin/fdflush ->
>> /bin/busybox
>> lrwxrwxrwx    1 root     root            12 Jan  9 12:46 /bin/sh ->
>> /bin/busybox
>>
>> 5513a6a8232d:/task/output/files$ /bin/sh
>>
>> /task/output/files $ set| head -1
>> BB_ASH_VERSION='1.35.0'
>> /task/output/files $ [ -r cnb_rate_eur_czk.csv ] && echo READABLE
>> READABLE
>>
>> There was a bash-5.1.16-r0 in alpine:3.13, which was the latest version,
>> where test -r worked correctly for me.
>>
>> So sorry for bugging here. I need to find out now what to do next to pursue
>> this.
> you can probably open an issue, after checking if someone else hasn't already:
> https://savannah.gnu.org/support/?group=bash
> (i assume that's the right place.)
>
> if you get it fixed and a patch merged, i'll happily backport it to 3.17.
>
>> Thanks again,
>>
>> Vlada
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/busybox/attachments/20230214/42b6a8b6/attachment-0001.html>
-------------- next part --------------
$ echo $BASH_VERSION ; cat /etc/alpine-release ; busybox | head -1 ; cd /task/output/files/ ; [ -r cnb_rate_eur_czk.csv ] && echo READABLE-INTERNAL-BASH ; /usr/bin/[ -r cnb_rate_eur_czk.csv ] && echo READABLE-EXTERNAL ; /bin/sh -c 'echo $BB_ASH_VERSION ; [ -r cnb_rate_eur_czk.csv ] && echo READABLE-INTERNAL-SH'

5.2.15(1)-release
3.17.1
BusyBox v1.35.0 (2022-11-19 10:13:10 UTC) multi-call binary.
READABLE-EXTERNAL
1.35.0
READABLE-INTERNAL-SH

5.1.16(1)-release
3.16.4
BusyBox v1.35.0 (2022-08-01 15:14:44 UTC) multi-call binary.
READABLE-EXTERNAL
1.35.0
READABLE-INTERNAL-SH

5.1.16(1)-release
3.15.7
BusyBox v1.34.1 (2022-07-19 20:11:24 UTC) multi-call binary.
READABLE-EXTERNAL
1.34.1
READABLE-INTERNAL-SH

5.1.16(1)-release
3.14.9
BusyBox v1.33.1 () multi-call binary.
READABLE-EXTERNAL
1.33.1
READABLE-INTERNAL-SH

5.1.16(1)-release
3.14.0
BusyBox v1.33.1 () multi-call binary.
READABLE-EXTERNAL
1.33.1
READABLE-SH-INTERNAL

5.1.16(1)-release
3.13.12
BusyBox v1.32.1 () multi-call binary.
READABLE-INTERNAL-BASH
READABLE-EXTERNAL

READABLE-INTERNAL-SH

5.0.17(1)-release
3.12.12
BusyBox v1.31.1 () multi-call binary.
READABLE-INTERNAL-BASH
READABLE-EXTERNAL

READABLE-INTERNAL-SH
-------------- next part --------------
FROM alpine:3.14.0

LABEL maintainer="devs at mixulo.com"

RUN apk add --no-cache \
	curl \
	bash \
	tar \
	tzdata \
 && addgroup --gid 2000 mixulo \
 && adduser --uid 2001 --home / --no-create-home --ingroup mixulo --gecos "Project Setup-Teardown" --disabled-password setup \
 && adduser --uid 2002 --home /home/task --ingroup mixulo --gecos "Project Task" --disabled-password task

USER setup

COPY ./scripts/*.sh /scripts/

# To test this container, use `-u task` parameter for `docker run` when using this.
CMD /scripts/dummy.sh
-------------- next part --------------
MODULE := setup-teardown

REGISTRY ?= project

IMAGE_NAME := $(REGISTRY)/$(MODULE)

.PHONY: image push prod push push-prod run-setup run run-teardown sh

BUILD_HOST := $(shell hostname --fqdn)
BUILD_LOG := build-logs/docker-build-$(REGISTRY)-$(MODULE)-$(BUILD_HOST)-$(shell date +%Y%m%d-%H%M%S-%N-%Z).log
BUILD_HEADER = "Image $(IMAGE_NAME) built $(shell date +"%FT%T%z") on $(BUILD_HOST) from $(shell git describe --abbrev=0 --always --dirty)"

image:
	echo "$(BUILD_HEADER)" >>$(BUILD_LOG)
	docker build -t $(IMAGE_NAME) . 2>&1 | tee -a $(BUILD_LOG)

clean-image:
	echo "$(BUILD_HEADER) (clean target)" >>$(BUILD_LOG)
	docker build --pull --no-cache -t $(IMAGE_NAME) . 2>&1 | tee -a $(BUILD_LOG)

push: image
	docker push $(IMAGE_NAME)

prod: image
	{ docker tag $(IMAGE_NAME) $(IMAGE_NAME):prod 2>&1 ; echo Tagged $(IMAGE_NAME):prod ; } | tee -a $(BUILD_LOG)

push-prod: prod
	docker push $(IMAGE_NAME):prod

# Development targets

RUN_FLAGS = \
	--rm \
	-e JOB_SPEC_inputBatchUrl \
	-e JOB_SPEC_outputFilePostUrl \
	-e JOB_SPEC_statusUrl \
	-e TZ \
	--security-opt=no-new-privileges \
	-v /var/tmp/done/project-job-dir-1095:/task

run-setup:
	docker run $(RUN_FLAGS) --network host --user setup $(RUN_ARGS) $(IMAGE_NAME) /scripts/setup.sh

run:
	docker run $(RUN_FLAGS) --network none -e DUMMY_SLEEP=1 $(RUN_ARGS) $(IMAGE_NAME)

run-teardown:
	docker run $(RUN_FLAGS) --network host $(IMAGE_NAME) $(RUN_ARGS) /scripts/teardown.sh

sh:
	docker run -ti $(RUN_FLAGS) $(RUN_ARGS) $(IMAGE_NAME) /bin/bash

More information about the busybox mailing list