-
Notifications
You must be signed in to change notification settings - Fork 370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test: [M3-6632] - Add Machine Image region tests #9413
test: [M3-6632] - Add Machine Image region tests #9413
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Appreciated the background context from the volumes test sessions when looking at this. 👍🏼 Ran successfully and LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you manually time these to get a rough estimate of how long these requests will take to complete?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jdamore-linode there is a part of me that REALLY wants to mock this. These are pretty big timeouts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mjac0bs Sort of! I'll usually run the tests a few times (just in the course of normal development) and get a rough gauge for how long these operations normally take, and in this case I set the timeout to be roughly double what I was observing.
@abailly-akamai Yeah, they are big :/ my hands are tied for the region tests, though -- there's not much value in using mocked data from a DC testing perspective, and machine image tests in particular are pretty important for the DC testing process.
I do want the timeouts to be high in this case because I want to be able to distinguish between cases where image uploading/processing is slow, and cases where it's broken completely. There are a couple mitigating factors that may hopefully make this less of a problem, though:
- The timeouts aren't really reflective of the length of time it takes for these operations to occur; from what I've seen, it usually takes 1-2 minutes to upload an Image, and 2-4 minutes to capture an Image from a Linode
- Right now the plan is for region tests to run only once daily or when manually kicked off (not against PRs*), so the run length of these tests shouldn't impact our day-to-day productivity
- As we develop slower end-to-end flows for the region testing pipeline, places where we have overlapping end-to-end test coverage in the core suite can be removed or replaced with mocked flows, hopefully giving us a lot of opportunity to improve the performance of our core suite
* All said and done, if the length of time it takes to run the region tests against a single region is roughly in line with our current test suite, then we could run region tests against PRs side-by-side with our existing test suite without it impacting our productivity, but it's not really on my radar right now
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Approving cause I don't have any alternative to adding timeouts other than mocking the response and I am not sure that's want you'd want to do if you want true e2e 🤷
Would be probably good to make a note of execution expectations it in the code for legacy?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jdamore-linode there is a part of me that REALLY wants to mock this. These are pretty big timeouts
Description 📝
This adds a handful of region tests (end-to-end) for Machine Image operations:
/images/create/upload
/images/create/disk
How to test 🧪
CY_TEST_SUITE=region CY_TEST_REGION=us-east yarn cy:run -s "cypress/e2e/region/images/*.spec.ts"
(It takes roughly 5 minutes to run all of the tests against 1 region on my machine)