Repository navigation
Alert and uptime tests: measure, close the gaps, trim - #209
Open
matuscongrady wants to merge 1 commit into
Open
matuscongrady wants to merge 1 commit into
matuscongrady wants to merge 1 commit into
Conversation
The alarm notification handler test now checks the body it sends to the Console against the public reportAlarmEventInputSchema and carries email, Teams and console-channel destinations, so a config-to-wire vocabulary slip (email instead of e_mail) fails the test instead of a production alarm. Three uptime-check validation cases were duplicates of others and are folded or removed. The alert PR-tier lane runs the channel-test wording test it already owned, and the signing and deployment-card unit tests deleted in the Console branch leave the registry. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second sweep over S63 (alerts) and S65 (uptime, alarms, synthetics). I wrote 28 faults for the alert routing and Slack app and 22 for uptime ingestion, the prober, the alarm handler and the alarm readers, each a small product edit a customer would notice, then ran the slice lanes against each one. Before this change the tests caught 23 of 27 scored alert faults and 18 of 22 uptime faults. With the two branches they catch all of them. One alert fault (removing the rule deduplication) turned out to have no customer effect because the per-delivery lease already prevents the second send, so it is listed but not scored.
The public side is small. The alarm notification handler test now validates the body it sends to the Console with the public reportAlarmEventInputSchema and carries email, Teams and console-channel destinations; before, sending the email channel in config vocabulary (email instead of e_mail) passed the test and would have made the Console reject every such alarm. The uptime-check validation spec loses three duplicate cases. The registry drops the two Console unit files deleted in the Console branch and makes the alert PR lane run the channel-test wording test it already owned.
The Console branch (stacktape/console-app, same branch name) holds the scenario additions and the deletions: the engine's resolution page broadcast to the channel, the Slack app channel test through its own organization (fix a084c0f), console-channel references resolved inside the organization only with missing names recorded, secret events bypassing project filters, the journey's recovery tick carrying seconds, and the hourly rollup recomputed after a crash between the hour upsert and the minute delete. Two assertions compared Date objects with toMatchObject, which Bun treats as always equal, so the certificate renewal check from fix ae3d5a2 proved nothing; both compare ISO strings now. Deleted there: the signing tests, the deployment confirmation parser test, the report-id, ARN and snooze-subject unit tests, each proven redundant by a database fault run, plus wording and guard tests with no customer-visible fault. Case counts go from 44 to 32 (S63) and 22 to 19 (S65).
No product bug was found. This PR does not move the Console submodule pointer.
Follow-up after Sol's review: the Console branch gained four scenario cases for deleted assertions a customer could notice (alarm names with colons, December snooze rollover, card durations, HTML email details), each with a fault run behind it. Nothing changed on the public side; the alert score is now 31 of 31.