Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Improve E2E test firewall creation and teardown #551

Conversation

ykim-akamai
Copy link
Contributor

@ykim-akamai ykim-akamai commented Jul 25, 2024

📝 Description

Here are some improvements around creation and deletion of E2E Test Firewall:

  • Now it will show the entire PLAY RECAP when there is failure in any of the playbook steps
  • Firewall will be teardown regardless of the test result but it will still return the correct exit code at the end

✔️ How to Test

git submodule update --init may be required if submodule directory isn't downloaded locally

Verifying error handling when firewall creation fails:

  1. export wrong LINODE_TOKEN
  2. make create-e2e-firewall

Verifying proper teardown when test execution fails:

  1. Make a test to fail with any reason (e.g. change some assertion arbitrarily)
  2. Run the test e.g. make TEST_ARGS="-v domain_basic" test
  3. Test returns failure and cloud firewall is teared down

📷 Preview

If applicable, include a screenshot or code snippet of this change. Otherwise, please remove this section.

@ykim-akamai ykim-akamai added the testing for updates to the testing suite in the changelog. label Jul 25, 2024
@ykim-akamai ykim-akamai requested a review from a team as a code owner July 25, 2024 19:38
@ykim-akamai ykim-akamai requested review from jriddle-linode and yec-akamai and removed request for a team July 25, 2024 19:38
Copy link
Contributor

@yec-akamai yec-akamai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested locally a failed integration test, but still can find the e2e firewall is not cleaned up. Also I find the error output is not rendered in color as it used to be, which makes it a bit more difficult to find the error message in the output. Could you also look into that?

@ykim-akamai ykim-akamai requested a review from yec-akamai August 1, 2024 17:47
Copy link
Contributor

@yec-akamai yec-akamai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally with a failed case and the firewall was cleaned up successfully! Also output is rendered normally. Nice work 🎉

Copy link
Member

@zliang-akamai zliang-akamai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works well on my end, nice work!

@ykim-akamai ykim-akamai merged commit c860c0a into linode:dev Aug 9, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing for updates to the testing suite in the changelog.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants