Add e2e test infrastructure and implementations for AI Transport guides#3213
Add e2e test infrastructure and implementations for AI Transport guides#3213GregHolmes wants to merge 2 commits intomainfrom
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
GregHolmes
force-pushed
the
AIT-489-Introduce-E2E-testing-for-Guides-code
branch
from
6bb14d3 to
05b88e2
Compare
There was a problem hiding this comment.
Overall looks great, thanks for this! Couple of minor comments
I think there is an outstanding question of how we can get this to work for different languages, in which case I think we might want to consider execing processes for the publisher/subscriber and asserting on the stdout output (or something better?)
Either way, this is a great start for JS, so happy to get this merged, and then iterate from there
| }); | ||
|
|
||
| await publish(pubChannel, 'Reply with exactly: OK'); | ||
| await new Promise((resolve) => setTimeout(resolve, 2000)); |
There was a problem hiding this comment.
Using timeouts like this are prone to flake - instead, we could wrap the channel.subscribe call with a promise that resolves with the first message received. I expect we could benefit from a helper function that can be used more generally in these tests to yield messages from the subscription that match a predicate e.g. something like
async function waitForMessage(
channel: Ably.RealtimeChannel,
predicate: MessagePredicate = () => true,
timeoutMs = 2000
): Promise<Ably.Message>
| let doneTimer: ReturnType<typeof setTimeout> | null = null; | ||
|
|
||
| const resetTimer = () => { | ||
| if (doneTimer) clearTimeout(doneTimer); |
There was a problem hiding this comment.
This is breaking the eslint curly rule: Expected { after 'if' condition.eslintcurly
| switch (message.action) { | ||
| case 'message.create': | ||
| console.log('\n[Response started]', message.serial); | ||
| responses.set(message.serial, message.data as string); |
There was a problem hiding this comment.
This line has a ts error:
Argument of type 'string | undefined' is not assignable to parameter of type 'string'.
Type 'undefined' is not assignable to type 'string'.ts(2345)
and one on the line below
Type 'string | undefined' is not assignable to type 'string | null'.
Type 'undefined' is not assignable to type 'string | null'.ts(2322)
Can we please add a lint and lint:fix script to the top level package json?
Introduce guides/ai-transport/ directory with full runnable code for all 8 AI Transport guides (4 providers x 2 streaming patterns), each with publisher, subscriber, and e2e tests hitting real LLM APIs and Ably. - OpenAI, Anthropic, Vercel AI SDK, and LangGraph providers - Message-per-token and message-per-response streaming patterns - 24 e2e tests total (3 per guide): lifecycle ordering, response reconstruction, and token concatenation verification - Yarn workspaces with shared Vitest config - Exclude guides/ from root Jest config
Move all guide contents into javascript/ subdirectories to prepare for future multi-language support (Java, Python, Swift). Update workspace paths, vitest config, and tsconfig extends accordingly.
GregHolmes
force-pushed
the
AIT-489-Introduce-E2E-testing-for-Guides-code
branch
from
05b88e2 to
0cafe96
Compare
2 participants
Introduce guides/ai-transport/ directory with full runnable code for all 8 AI Transport guides (4 providers x 2 streaming patterns), each with publisher, subscriber, and e2e tests hitting real LLM APIs and Ably.