The environment tests (Layer 2) in this directory require real cloud storage / HDFS / Broker services to run. They are automatically skipped in default CI builds.

The repository root provides run-fs-env-test.sh, which supports running tests via command-line arguments or pre-set environment variables:

# S3 tests
./run-fs-env-test.sh s3 \
  --s3-endpoint=https://s3.us-east-1.amazonaws.com \
  --s3-region=us-east-1 \
  --s3-bucket=my-test-bucket \
  --s3-ak=AKIAIOSFODNN7EXAMPLE \
  --s3-sk=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# Azure tests
./run-fs-env-test.sh azure \
  --azure-account=myaccount \
  --azure-key=base64key== \
  --azure-container=testcontainer

# HDFS tests (Simple Auth)
./run-fs-env-test.sh hdfs \
  --hdfs-host=namenode.example.com \
  --hdfs-port=8020

# Kerberos tests
./run-fs-env-test.sh kerberos \
  --kdc-principal=hdfs/namenode@REALM \
  --kdc-keytab=/path/to/hdfs.keytab \
  --hdfs-host=namenode.example.com \
  --hdfs-port=8020

# COS (Tencent Cloud) tests
./run-fs-env-test.sh cos \
  --cos-endpoint=https://cos.ap-guangzhou.myqcloud.com \
  --cos-region=ap-guangzhou \
  --cos-bucket=my-cos-bucket \
  --cos-ak=SecretId \
  --cos-sk=SecretKey

# OSS (Alibaba Cloud) tests
./run-fs-env-test.sh oss \
  --oss-endpoint=https://oss-cn-hangzhou.aliyuncs.com \
  --oss-region=cn-hangzhou \
  --oss-bucket=my-oss-bucket \
  --oss-ak=AccessKeyId \
  --oss-sk=AccessKeySecret

# OBS (Huawei Cloud) tests
./run-fs-env-test.sh obs \
  --obs-endpoint=https://obs.cn-north-4.myhuaweicloud.com \
  --obs-region=cn-north-4 \
  --obs-bucket=my-obs-bucket \
  --obs-ak=AK \
  --obs-sk=SK

# Broker tests
./run-fs-env-test.sh broker \
  --broker-host=broker.example.com \
  --broker-port=8060

# Run all environment tests (requires all environment variables to be pre-set)
./run-fs-env-test.sh all

You can also export environment variables first and then run tests directly with Maven:

# Export credentials
export DORIS_FS_TEST_S3_ENDPOINT=https://s3.us-east-1.amazonaws.com
export DORIS_FS_TEST_S3_REGION=us-east-1
export DORIS_FS_TEST_S3_BUCKET=my-test-bucket
export DORIS_FS_TEST_S3_AK=your-access-key
export DORIS_FS_TEST_S3_SK=your-secret-key

# Run S3-related environment tests
cd fe
mvn test -pl fe-filesystem/fe-filesystem-s3 \
  -Dtest.excludedGroups=none \
  -Dgroups=s3 \
  -Dcheckstyle.skip=true \
  -DfailIfNoTests=false \
  -Dmaven.build.cache.enabled=false \
  --also-make

# Run HDFS + Kerberos environment tests
mvn test -pl fe-filesystem/fe-filesystem-hdfs \
  -Dtest.excludedGroups=none \
  -Dgroups="hdfs | kerberos" \
  -Dcheckstyle.skip=true \
  -DfailIfNoTests=false \
  -Dmaven.build.cache.enabled=false \
  --also-make

• putAndHeadObject — Upload a small file → verify size and etag via headObject
• listObjects — Upload multiple files → verify returned count via listObjects
• copyAndDeleteObject — Upload → copy → verify → delete → verify non-existence
• multipartUpload_completeSucceeds — initiate → uploadPart × 2 → complete → verify via headObject
• abortMultipartUpload_leavesNoObject — initiate → abort → object does not exist
• getPresignedUrl_returnsValidUrlAndUploadWorks — Generate presigned URL → PUT upload → verify object exists

• exists — Existing / non-existing objects
• deleteRemovesObject — Upload → delete → exists returns false
• renameMovesObject — Upload → rename → old does not exist / new exists
• listReturnsCorrectEntries — Upload multiple → verify via list
• inputOutputRoundTrip — Write → read → content matches (including UTF-8 + emoji)
• inputFileLength — Upload with known size → verify length()
• getPresignedUrl_returnsValidUrlAndUploadWorks — Generate presigned URL → PUT upload → verify object exists

Same test cases as T-E2, using the wasbs:// scheme.

Same test cases as T-E2, using the Tencent Cloud / Alibaba Cloud / Huawei Cloud SDKs respectively.

• mkdirsAndExists — Create multi-level directories → exists returns true
• deleteRecursive — Create a directory with files → recursive delete
• renameFile / renameDirectory — File / directory rename
• listFiles / listDirectories — List files / subdirectories
• inputOutputRoundTrip — Write → read → content matches
• inputFileLength — Verify file size

• loginSucceeds — Log in with a real principal / keytab
• doAsExecutesAction — Verify return value of proxied execution
• doAsPropagatesIOException — IOException is correctly propagated
• hdfsOperationWithKerberos — HDFS exists works correctly under Kerberos

• existsReturnsFalseForMissing — Non-existing path → false
• writeAndRead — Write via outputFile → read via inputFile → content matches
• deleteRemovesFile — Write → delete → exists returns false
• listReturnsFiles — Write multiple → verify via list

1. Each test creates temporary data with a unique UUID prefix in the target storage. @AfterAll attempts cleanup. If a test is interrupted abnormally, manual cleanup may be required.

2. S3 multipart upload tests upload ~5 MB of data. Ensure the test bucket has sufficient space and permissions.

3. Environment tests are not run by default (the module POM sets <excludedGroups>${test.excludedGroups}</excludedGroups> with a default value of environment). They only run when explicitly enabled with -Dtest.excludedGroups=none -Dgroups=<tag>.

4. Never commit credentials to the code repository. Use environment variables or CI/CD secret management instead.