Skip to content

Forward port Lint Version script fixes #1008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 1 commit into from
Sep 21, 2020
Merged

Forward port Lint Version script fixes #1008

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/master_only.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ jobs:
run: echo ::set-env name=RELEASE_VERSION::${GITHUB_REF#refs/*/}
- name: Push versioned Docker image
run: |
source infra/scripts/setup-common-functions.sh
# Build and push semver tagged commits
# Regular expression should match MAJOR.MINOR.PATCH[-PRERELEASE[.IDENTIFIER]]
# eg. v0.7.1 v0.7.2-alpha v0.7.2-rc.1
Expand All @@ -52,10 +53,10 @@ jobs:
docker push gcr.io/kf-feast/feast-${{ matrix.component }}:${VERSION_WITHOUT_PREFIX}

# Also update "latest" image if tagged commit is pushed to stable branch
HIGHEST_SEMVER_TAG=$(git tag -l --sort -version:refname | head -n 1)
HIGHEST_SEMVER_TAG=$(get_tag_release -m)
echo "Only push to latest tag if tag is the highest semver version $HIGHEST_SEMVER_TAG"

if [ "${VERSION_WITHOUT_PREFIX}" == "${HIGHEST_SEMVER_TAG:1}" ]
if [ "${VERSION_WITHOUT_PREFIX}" = "${HIGHEST_SEMVER_TAG:1}" ]

@pyalex pyalex Sep 16, 2020 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe [[ "${VERSION_WITHOUT_PREFIX}" == "${HIGHEST_SEMVER_TAG:1}" ]] ?

@mrzzy mrzzy Sep 16, 2020 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[] implies the test command. According to its manpage, it uses single = for string comparison.

then
docker tag gcr.io/kf-feast/feast-${{ matrix.component }}:${GITHUB_SHA} gcr.io/kf-feast/feast-${{ matrix.component }}:latest
docker push gcr.io/kf-feast/feast-${{ matrix.component }}:latest
Expand Down
44 changes: 43 additions & 1 deletion infra/scripts/setup-common-functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,4 +173,46 @@ wait_for_docker_image(){
done

set -$oldopt
}
}

# Usage: TAG=$(get_tag_release [-ms])
# Parses the last release from git tags.
# Options:
# -m - Use only tags that are tagged on the current branch
# -s - Use only stable version tags. (ie no prerelease tags).
get_tag_release() {
local GIT_TAG_CMD="git tag -l"
# Match only Semver tags
# Regular expression should match MAJOR.MINOR.PATCH[-PRERELEASE[.IDENTIFIER]]
# eg. v0.7.1 v0.7.2-alpha v0.7.2-rc.1
local TAG_REGEX='^v[0-9]+\.[0-9]+\.[0-9]+(-([0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*))?$'
local OPTIND opt
while getopts "ms" opt; do
case "${opt}" in
m)
GIT_TAG_CMD="$GIT_TAG_CMD --merged"
;;
s)
# Match only stable version tags.
TAG_REGEX="^v[0-9]+\.[0-9]+\.[0-9]+$"
;;
*)
echo "get_tag_release(): Error: Bad arguments: $@"
return 1
;;
esac
done
shift $((OPTIND-1))

# Retrieve tags from git and filter as per regex.
local FILTERED_TAGS=$(bash -c "$GIT_TAG_CMD" | grep -P "$TAG_REGEX")

# Sort version tags in highest semver version first.
# To make sure that prerelease versions (ie versions vMAJOR.MINOR.PATCH-PRERELEASE suffix)
# are sorted after stable versions (ie vMAJOR.MINOR.PATCH), we append '_' after
# eachustable version as '_' is after '-' found in prerelease version
# alphanumerically and remove after sorting.
local SEMVER_SORTED_TAGS=$(echo "$FILTERED_TAGS" | sed -e '/-/!{s/$/_/}' | sort -rV \
| sed -e 's/_$//')
echo $(echo "$SEMVER_SORTED_TAGS" | head -n 1)
}
13 changes: 7 additions & 6 deletions infra/scripts/validate-version-consistency.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
# versions against the given merge branch.
set -e

source infra/scripts/setup-common-functions.sh

# Fetch tags and current branch
git fetch --prune --unshallow --tags || true
BRANCH_NAME=${TARGET_MERGE_BRANCH-$(git rev-parse --abbrev-ref HEAD)}
Expand All @@ -32,8 +34,8 @@ then
elif echo "$BRANCH_NAME" | grep -P $RELEASE_BRANCH_REGEX &>/dev/null
then
# Use last release tag tagged on the release branch
LAST_MERGED_TAG=$(git tag -l --sort -version:refname --merged | head -n 1)
FEAST_MASTER_VERSION=${LAST_MERGED_TAG#"v"}
LAST_MERGED_TAG=$(get_tag_release -m)
FEAST_RELEASE_VERSION=${LAST_MERGED_TAG#"v"}
else
# Do not enforce version linting as we don't know if the target merge branch FEAST_RELEASE_VERSION="_ANY"
FEAST_RELEASE_VERSION="_ANY"
Expand All @@ -52,12 +54,12 @@ STABLE_TAG_REGEX="^v[0-9]+\.[0-9]+\.[0-9]+$"
if [ $BRANCH_NAME = "master" ]
then
# Use last stable tag repo wide
LAST_STABLE_TAG=$(git tag --sort -version:refname | grep -P "$STABLE_TAG_REGEX" | head -n 1)
LAST_STABLE_TAG=$(get_tag_release -s)
FEAST_STABLE_VERSION=${LAST_STABLE_TAG#"v"}
elif echo "$BRANCH_NAME" | grep -P $RELEASE_BRANCH_REGEX &>/dev/null
then
# Use last stable tag tagged on the release branch
LAST_STABLE_MERGE_TAG=$(git tag --sort -version:refname --merged | grep -P "$STABLE_TAG_REGEX" | head -n 1)
LAST_STABLE_MERGE_TAG=$(get_tag_release -sm)
FEAST_STABLE_VERSION=${LAST_STABLE_MERGE_TAG#"v"}
else
# Do not enforce version linting as we don't know if the target merge branch
Expand Down Expand Up @@ -100,7 +102,6 @@ declare -a files_to_validate_version=(
"docs/contributing/development-guide.md,4,${FEAST_MASTER_VERSION}"
"docs/administration/audit-logging.md,1,${FEAST_STABLE_VERSION}"
"docs/getting-started/deploying-feast/docker-compose.md,1,${FEAST_STABLE_VERSION}"
"docs/getting-started/deploying-feast/kubernetes.md,1,${FEAST_STABLE_VERSION}"
"README.md,1,${FEAST_STABLE_VERSION}"
"CHANGELOG.md,2,${FEAST_STABLE_VERSION}"
)
Expand All @@ -123,7 +124,7 @@ for i in "${files_to_validate_version[@]}"; do
echo "Testing whether versions are correctly set within file: $FILE_PATH"
ACTUAL_OCCURRENCES=$(grep -c -P "\bv?$VERSION\b" "$FILE_PATH" || true)

if [ "${ACTUAL_OCCURRENCES}" -eq "${EXPECTED_OCCURRENCES}" ]; then
if [ "${ACTUAL_OCCURRENCES}" -ge "${EXPECTED_OCCURRENCES}" ]; then

@mrzzy mrzzy Sep 16, 2020

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actual occurrences is greater than that expected occurrences when both pom.xml and tag versions the same. This is not a issue when pom.xml version is SNAPSHOT, but when its not in SNAPSHOT form, lint version scripts will flag extra occurrences and fail.

echo "OK: Expecting $EXPECTED_OCCURRENCES occurrences of '$VERSION' in $FILE_PATH, and found $ACTUAL_OCCURRENCES"
else
echo "FAIL: Expecting $EXPECTED_OCCURRENCES occurrences of '$VERSION' in $FILE_PATH, but found $ACTUAL_OCCURRENCES"
Expand Down