[ci-scan] Skip AsyncProfilerTests on Android and tvOS (refs #127951)#127963
Draft
github-actions[bot] wants to merge 2 commits intomainfrom
Draft
[ci-scan] Skip AsyncProfilerTests on Android and tvOS (refs #127951)#127963github-actions[bot] wants to merge 2 commits intomainfrom
github-actions[bot] wants to merge 2 commits intomainfrom
Conversation
The AsyncProfilerTests.RuntimeAsync_* tests consistently fail on android-arm64 and tvos-arm64 because EventPipe async profiler events are not emitted on mobile platforms, causing assertion and collection-empty failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
Tagging subscribers to this area: @akoeplinger, @matouskozak, @simonrozsival |
kotlarmilos
added a commit
that referenced
this pull request
May 8, 2026
…27961) ## Description Refines the `ci-failure-scan` agentic-workflow prompt to address failure modes seen in past runs: - Generic `ErrorMessage` signatures (bare test names, exception types, or truncated prefixes) that match `[PASS]` / `[SKIP]` lines for the same test and turn the KBE into a false-positive matcher against passing builds. - Malformed JSON fences (4-backtick opens, mismatched fence lengths, multiple skeletons in one body) that cause Build Analysis to silently skip the issue. - Issues filed under the `Known Build Error` label with no JSON block at all, so nothing matches future failures. - Muting PRs that link to a non-existent issue number or duplicate an in-flight fix authored by a maintainer (because the search was scoped too narrowly to `[ci-scan]` PRs). - Muting PRs opened against issues whose area owners had already provided a PR - Wrong-KBE links where the candidate KBE matched only on test name but was filed against a different architecture / failure signature. ## Changes The body of `.github/workflows/ci-failure-scan.md` is reorganized into a clear walk-through: 1. **Two-pass KBE → PR flow** is now a numbered six-step pre-flight walk (existing KBE / area tracker / muting PR / in-flight fix PR / issue resolves / mute is welcome) followed by an explicit action selection (file KBE, open muting PR, optionally open fix PR). 2. **KBE-match verification** — four questions (test, signature, OS, architecture) the agent must answer before linking an existing KBE. Wrong answers mean filing a fresh KBE rather than reusing the wrong one. 3. **Body checks** — eight explicit checks on the issue body covering the JSON fence, exact ```json opening, single-line/no-escapes signature, and a negative-match test against `[PASS]` / `[SKIP]` and build-time output. 4. **Bad → Good** examples for both signature shape (bare test name, truncated prefix, bare exception type) and platform/csproj scope (`linux-arm`-only, single-arch NativeAOT, single stress mode). 5. Coverage-discipline section trimmed to its unique contribution (pipeline ordering, per-pipeline tally, run summary). Redundant `Submit` section removed — its content is now covered by the numbered walk. ## Test run results Workflow run [25570821336](https://github.com/dotnet/runtime/actions/runs/25570821336) was dispatched against this branch (commit `3cd6399dd70`, pre-Copilot-fixup) and completed successfully (~28 min). Outputs: | # | Type | Title | Linked tracker | |---|---|---|---| | [#127963](#127963) | PR (draft) | `[ci-scan] Skip AsyncProfilerTests on Android and tvOS` | #127951 | | [#127964](#127964) | PR (draft) | `[ci-scan] Skip System.Net.Sockets IPv6 tests on Android` | #127565 | | [#127965](#127965) | Issue (KBE) | `[ci-scan] Known Build Error: System.Net.NameResolution DnsGetHostAddresses_LocalhostSubdomainWithTrailingDot fails on Android` | new | | [#127966](#127966) | Issue (regression) | `[ci-scan] Test failure: XslCompiledTransformApiTests (82 tests) on all NativeAOT legs — PlatformNotSupportedException (Reflection.Emit)` | new | | [#127967](#127967) | PR (draft) | `[ci-scan] Exclude Vector3Interop from GC stress` | #127827 | --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reasoning
The
AsyncProfilerTestsclass tests EventPipe-based async profiling (RuntimeAsync events). On Android and tvOS, the EventPipe runtime events are not emitted by the mobile runtime, causing all ~30RuntimeAsync_*tests to fail with either:Assert.NotEmpty() Failure: Collection was empty— no EventPipe events emittedExpected callstack with 4 frames, got 3— incomplete call stacks from partial EventPipe dataThis is a test-environment limitation on mobile platforms, not a product bug. The
[ActiveIssue]annotation at the class level is the right fix because all tests in the class share the same EventPipe dependency and all fail on these platforms.KBE verification (4-question check):
AsyncProfilerTestsRuntimeAsync tests.ErrorMessagematches the same failure signature —Assert.NotEmpty() FailureandExpected callstack with 4 frames.Impact on platforms
runtime-extra-platforms(def 154) ·android-arm64·Ubuntu.2204.Amd64.Android.29.Open· no stress · exit code 1runtime-extra-platforms(def 154) ·tvossimulator-x64·OSX.1015.Amd64.Open· no stress · exit code 1Errors log
First build it occurred
runtime-extra-platforms build 1411505 — finished 2026-05-07. Also present in latest build 1413202. ≥2 occurrences in the scanned window. Note: this is computed within the ~10-build scan window and may not be the true origin.
Linked issue
Linked KBE: #127951
Note
🔒 Integrity filter blocked 2 items
The following items were blocked because they don't meet the GitHub integrity level.
search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".To allow these resources, lower
min-integrityin your GitHub frontmatter: