-
Notifications
You must be signed in to change notification settings - Fork 70
feat(ccwidgets): add e2e tests for real time transcript and real time assist #729
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Kesari3008
wants to merge
12
commits into
next
Choose a base branch
from
RealTime-Assist-E2E
base: next
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
8d70374
feat(ccwidgets): add e2e tests for real time transcript and real time…
Kesari3008 4a01bae
feat(ccwidgets): refined tests for real time trasncripts
Kesari3008 98e7afe
feat(ccwidgets): refined tests for real time assist
Kesari3008 0b146b6
feat(ccwidgets): small changes
Kesari3008 09aa387
feat(ccwidgets): reverting config to strictly use chrome
Kesari3008 6619af0
Merge branch 'next' of https://github.com/webex/widgets into RealTime…
Kesari3008 633ecfd
feat(ccwidgets): temp config to troubleshoot e2e in pipeline
Kesari3008 7760066
feat(ccwidgets): playwright config revert
Kesari3008 80db8ab
Merge branch 'next' of https://github.com/webex/widgets into RealTime…
Kesari3008 1969bba
feat(ccwidgets): github runner resource exhaustion
Kesari3008 7c5c768
feat(ccwidgets): failed e2e test fixes
Kesari3008 da690b2
feat(ccwidgets): config update
Kesari3008 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| import {Page, expect, Locator} from '@playwright/test'; | ||
| import {AWAIT_TIMEOUT, AI_ASSIST_SUGGESTION_TIMEOUT} from '../constants'; | ||
|
|
||
| /** | ||
| * Utility functions for exercising the AI Assistant widget (launcher, landing | ||
| * page, Real-Time Assist chat, and adaptive-card feedback controls) in e2e | ||
| * tests. | ||
| * | ||
| * The suggestion content itself is generated by a live backend AI pipeline | ||
| * during the test call, so these helpers deliberately avoid asserting on | ||
| * exact transcript/suggestion text and instead verify structure and state | ||
| * transitions (panel open, spinner clears, chat entries appear, feedback | ||
| * controls toggle). | ||
| * | ||
| * @packageDocumentation | ||
| */ | ||
|
|
||
| /** | ||
| * Enables the AI Assistant widget via its sample-app checkbox. | ||
| * Unlike most widgets, AI Assistant defaults to unchecked, so this must be | ||
| * called explicitly before the widget will render. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function enableAIAssistantWidget(page: Page): Promise<void> { | ||
| const checkbox = page.getByTestId('samples:widget-aiAssistant'); | ||
| const isChecked = await checkbox.isChecked().catch(() => false); | ||
| if (!isChecked) { | ||
| await checkbox.check({timeout: AWAIT_TIMEOUT}); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Opens the AI Assistant panel by clicking its launcher button. | ||
| * No-ops if the panel is already open (launcher only renders when closed). | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function openAIAssistant(page: Page): Promise<void> { | ||
| const launcher = page.getByTestId('ai-assistant:launcher'); | ||
| if (await launcher.isVisible().catch(() => false)) { | ||
| await launcher.click({timeout: AWAIT_TIMEOUT}); | ||
| } | ||
| await expect(page.getByTestId('ai-assistant:panel')).toBeVisible({timeout: AWAIT_TIMEOUT}); | ||
| } | ||
|
|
||
| /** | ||
| * Closes the AI Assistant panel via its header close button. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function closeAIAssistant(page: Page): Promise<void> { | ||
| const closeButton = page.getByTestId('ai-assistant:header-close'); | ||
| if (await closeButton.isVisible().catch(() => false)) { | ||
| await closeButton.click({timeout: AWAIT_TIMEOUT}); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Returns true while the landing page (feature disabled / no active | ||
| * interaction) is shown instead of the Real-Time Assist chat. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function isShowingLanding(page: Page): Promise<boolean> { | ||
| return page | ||
| .getByTestId('ai-assistant:landing') | ||
| .isVisible() | ||
| .catch(() => false); | ||
| } | ||
|
|
||
| /** | ||
| * Clicks "Get Suggestions" and waits for the request to settle: either the | ||
| * chat/context-form appears (success) or the inline error message appears | ||
| * (failure). Does not throw on failure - callers assert the outcome. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function requestRealTimeAssistSuggestions(page: Page): Promise<void> { | ||
| const getSuggestionsButton = page.getByTestId('ai-assistant:get-suggestions'); | ||
| await expect(getSuggestionsButton).toBeVisible({timeout: AWAIT_TIMEOUT}); | ||
| await getSuggestionsButton.click({timeout: AWAIT_TIMEOUT}); | ||
|
|
||
| // The button is replaced by a spinner while the request is in flight; | ||
| // wait for the request to settle one way or another. | ||
| await Promise.race([ | ||
| page.getByTestId('ai-assistant:context-form').waitFor({state: 'visible', timeout: AI_ASSIST_SUGGESTION_TIMEOUT}), | ||
| page.getByTestId('ai-assistant:error').waitFor({state: 'visible', timeout: AI_ASSIST_SUGGESTION_TIMEOUT}), | ||
| ]).catch(() => {}); | ||
| } | ||
|
|
||
| /** | ||
| * Waits for at least one assistant suggestion (adaptive card or greeting) to | ||
| * appear in the Real-Time Assist chat. | ||
| * @param page - The Playwright page object | ||
| * @returns Locator for the first assistant chat entry | ||
| */ | ||
| export async function waitForFirstSuggestion(page: Page): Promise<Locator> { | ||
| const assistantEntry = page.getByTestId('ai-assistant:chat-assistant').first(); | ||
| await assistantEntry.waitFor({state: 'visible', timeout: AI_ASSIST_SUGGESTION_TIMEOUT}); | ||
| return assistantEntry; | ||
| } | ||
|
|
||
| /** | ||
| * Clicks the like/dislike feedback control (identified by its accessible | ||
| * name) on the first rendered suggestion card and returns its locator so the | ||
| * caller can assert on the resulting `data-active` state. | ||
| * @param page - The Playwright page object | ||
| * @param kind - Which control to click | ||
| */ | ||
| export async function clickSuggestionFeedback(page: Page, kind: 'like' | 'dislike'): Promise<Locator> { | ||
| const label = kind === 'like' ? 'Like suggestion' : 'Dislike suggestion'; | ||
| const control = page.getByLabel(label).first(); | ||
| await expect(control).toBeVisible({timeout: AWAIT_TIMEOUT}); | ||
| await control.click({timeout: AWAIT_TIMEOUT}); | ||
| return control; | ||
| } | ||
|
|
||
| /** | ||
| * Clicks the copy control on the first rendered suggestion card. | ||
| * @param page - The Playwright page object | ||
| * @returns Locator for the copy control | ||
| */ | ||
| export async function clickSuggestionCopy(page: Page): Promise<Locator> { | ||
| const control = page.getByLabel('Copy suggestion').first(); | ||
| await expect(control).toBeVisible({timeout: AWAIT_TIMEOUT}); | ||
| await control.click({timeout: AWAIT_TIMEOUT}); | ||
| return control; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| import {Page, Locator} from '@playwright/test'; | ||
| import {TRANSCRIPT_ENTRY_TIMEOUT} from '../constants'; | ||
|
|
||
| /** | ||
| * Utility functions for exercising the Real-Time Transcript widget in e2e | ||
| * tests. | ||
| * | ||
| * Transcript content is generated by a live speech-to-text pipeline from the | ||
| * dummy audio played on the call, so these helpers verify that entries | ||
| * eventually appear rather than asserting on exact transcribed text. | ||
| * | ||
| * @packageDocumentation | ||
| */ | ||
|
|
||
| /** | ||
| * Waits for the Real-Time Transcript panel to be visible. | ||
| * The panel only renders while `store.currentTask` exists (an active call). | ||
| * @param page - The Playwright page object | ||
| * @param timeout - Optional timeout override in ms | ||
| */ | ||
| export async function waitForRealTimeTranscriptPanel(page: Page, timeout: number = TRANSCRIPT_ENTRY_TIMEOUT) { | ||
| const root = page.getByTestId('real-time-transcript:root'); | ||
| await root.waitFor({state: 'visible', timeout}); | ||
| return root; | ||
| } | ||
|
|
||
| /** | ||
| * Waits for at least one transcript entry to appear in the live transcript | ||
| * feed. | ||
| * @param page - The Playwright page object | ||
| * @returns Locator for the first transcript item | ||
| */ | ||
| export async function waitForFirstTranscriptEntry(page: Page): Promise<Locator> { | ||
| const item = page.getByTestId('real-time-transcript:item').first(); | ||
| await item.waitFor({state: 'visible', timeout: TRANSCRIPT_ENTRY_TIMEOUT}); | ||
| return item; | ||
| } | ||
|
|
||
| /** | ||
| * Returns the count of transcript entries currently rendered. | ||
| * @param page - The Playwright page object | ||
| */ | ||
| export async function getTranscriptEntryCount(page: Page): Promise<number> { | ||
| return page.getByTestId('real-time-transcript:item').count(); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| import {test} from '@playwright/test'; | ||
| import createRealTimeAssistTests from '../tests/real-time-assist-test.spec'; | ||
| import createRealTimeTranscriptTests from '../tests/real-time-transcript-test.spec'; | ||
|
|
||
| test.describe('Real-Time Assist Tests', createRealTimeAssistTests); | ||
| test.describe('Real-Time Transcript Tests', createRealTimeTranscriptTests); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replacing
test:e2ewith only the two suffixed scripts leaves no root script namedtest:e2e, while the modified Playwright README still usesyarn test:e2efor every all-suite, single-suite, project, UI, debug, and headed invocation. Those documented local commands now fail during script resolution before Playwright starts; retain a compatible alias or update the documented callers to usetest:e2e:cc.AGENTS.md reference: AGENTS.md:L78-L78
Useful? React with 👍 / 👎.