Skip to content

tools: automate icu update #47727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nodejs-github-bot merged 9 commits into from
May 9, 2023
Merged

tools: automate icu update #47727

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
3d9f617
tools: automate icu-small update
marco-ippolito Apr 26, 2023
503ffb7
fix: replace - with .
marco-ippolito Apr 26, 2023
0299d51
fix: md5 from source
marco-ippolito Apr 27, 2023
c82d718
fix: comparing checksum
marco-ippolito Apr 27, 2023
84c6077
fix: mention the script in the maintaining guide
marco-ippolito Apr 27, 2023
e794280
fix: lint
marco-ippolito Apr 27, 2023
b915e51
fix: update timezone.txt
marco-ippolito Apr 28, 2023
12a7f50
fix: remove tz update
marco-ippolito May 2, 2023
15ff4c5
fix: use version as whole
marco-ippolito May 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: replace - with .
  • Loading branch information
@marco-ippolito
marco-ippolito committed May 3, 2023
commit 503ffb7d71d8c4ebfbb747a11196419b28fd5432
10 changes: 4 additions & 6 deletions tools/dep_updaters/update-icu.sh
View file
Open in desktop
Copy link
Copy Markdown
Member

@richardlau richardlau Apr 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll likely need to update ./test/fixtures/tz-version.txt to match the timezone version shipped in the new version of ICU.
Refs: #47456 (comment)

Copy link
Copy Markdown
Member

@srl295 srl295 Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes.

Copy link
Copy Markdown
Member

@srl295 srl295 Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and there are potential 'golden' value breakage, but not much to do about that.

Copy link
Copy Markdown
Member Author

@marco-ippolito marco-ippolito Apr 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont we have an action that does this? #47302

Copy link
Copy Markdown
Member

@richardlau richardlau Apr 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marco-ippolito The timezone data gets baked into the ICU data file, which updating ICU will overwrite. If ./test/fixtures/tz-version.txt doesn't match the timezone data version in the ICU data file then the associated test will fail.

Copy link
Copy Markdown
Member Author

@marco-ippolito marco-ippolito Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have managed to get the latest version from https://data.iana.org/time-zones/releases/ but its a bit hacky @srl295 do you know any api we could fetch the latest version?

Copy link
Copy Markdown
Member

@srl295 srl295 Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what you should do is extract the tz version from the ICU you're updating to, and put that in tz-version.txt.

Maybe

$ node -p process.versions.tz | tee test/fixtures/tz-version.txt

And then in reviewing the PR make sure it's not a regression.

Copy link
Copy Markdown
Member

@richardlau richardlau Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that will work because the update script added here doesn't rebuild Node.js, so that will output the tz version of whatever node binary is on the runner instead of the one from the data. I presume the tz version must be in the updated ICU files but maybe it'll be too complicated to extract without building Node.js with the new data?

I suppose we could ignore it and fix up manually if the opened PR breaks because of a different tz version in the new ICU data.

Copy link
Copy Markdown
Member

@srl295 srl295 Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@richardlau it ought to build to make sure node works… the PR will do that, though. the tz version is deep in the gzipped binary .dat file.

Yes, it could be ignored and fixed manually . As i said it's likely there will be some manual tweaks as Node has been made more sensitive to these changes.

In that case, just don't modify tz-version.txt at all in this workflow.

Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,13 @@ NEW_VERSION="$("$NODE" --input-type=module <<'EOF'
const res = await fetch('https://api.github.com/repos/unicode-org/icu/releases/latest');
if (!res.ok) throw new Error(`FetchError: ${res.status} ${res.statusText}`, { cause: res });
const { tag_name } = await res.json();
console.log(tag_name.replace('release-', ''));
console.log(tag_name.replace('release-', '').replace('-','.'));
EOF
)"

ICU_VERSION_H="$DEPS_DIR/icu-small/source/common/unicode/uvernum.h"
CURRENT_MAJOR_VERSION=$(grep "#define U_ICU_VERSION_MAJOR_NUM" "$ICU_VERSION_H" | cut -d ' ' -f3 | tr -d '\r')
CURRENT_MINOR_VERSION=$(grep "#define U_ICU_VERSION_MINOR_NUM" "$ICU_VERSION_H" | cut -d ' ' -f3 | tr -d '\r')

CURRENT_VERSION="$CURRENT_MAJOR_VERSION-$CURRENT_MINOR_VERSION"
CURRENT_VERSION="$(grep "#define U_ICU_VERSION " "$ICU_VERSION_H" | cut -d'"' -f2)"

echo "Comparing $NEW_VERSION with $CURRENT_VERSION"

Expand All @@ -30,8 +28,8 @@ if [ "$NEW_VERSION" = "$CURRENT_VERSION" ]; then
exit 0
fi

NEW_MAJOR_VERSION=$(echo "$NEW_VERSION" | cut -d '-' -f1)
NEW_MINOR_VERSION=$(echo "$NEW_VERSION" | cut -d '-' -f2)
NEW_MAJOR_VERSION=$(echo "$NEW_VERSION" | cut -d '.' -f1)
NEW_MINOR_VERSION=$(echo "$NEW_VERSION" | cut -d '.' -f2)

NEW_VERSION_TGZ="https://github.com/unicode-org/icu/releases/download/release-${NEW_VERSION}/icu4c-${NEW_MAJOR_VERSION}_${NEW_MINOR_VERSION}-src.tgz"

Expand Down