Implement terraform based system tests #227

mtojek · 2021-01-19T13:57:21Z

~~I'm opening this as draft to present the progress on the TF-based system tests.~~

~~Please do not consider this as final form, as I will modify it soon.~~

Issue: #89

How to test this:

export AWS_ACCESS_KEY_ID="secret"
export AWS_SECRET_ACCESS_KEY="secret"

cd test/packages/aws
elastic-package test system --data-streams ec2_metrics -v

Tf definitions in: data_stream/ec2_metrics/_dev/deploy/tf/main.tf (or other .tf files)

EDIT:

I switched from SNS to EC2 metrics during tests as metrics tend to appear quicker.

elasticmachine · 2021-01-19T22:31:05Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #227 updated
- Start Time: 2021-01-29T17:05:56.456+0000
Duration: 15 min 39 sec
Commit: e7e8764

Test stats 🧪

Test	Results
Failed	0
Passed	291
Skipped	0
Total	291

mtojek · 2021-01-21T15:17:25Z

Status update:

I managed to write the TF service deployer, but it needs to nice Terraform definition, which will actually perform something useful. A test scenario consist of two parts:

Test configuration (mainly vars): https://github.com/mtojek/elastic-package/blob/89-terraform-definitions/test/packages/aws/data_stream/sns/_dev/test/system/test-default-config.yml
Terraform definition which will bring up required AWS resources: https://github.com/mtojek/elastic-package/blob/89-terraform-definitions/test/packages/aws/data_stream/sns/_dev/deploy/tf/main.tf

@kaiyan-sheng I was wondering if you wouldn't like to play with it (git checkout), share some feedback and write some Terraform files which can usually do sth useful (most wanted). I wrote a sample which creates SNS topic, but due to lack of messages, I don't see any CloudWatch metrics :(

FYI I haven't enabled the AWS account in the CI due to security reasons and cost-savings. Let's first check it on our workstations.

mtojek · 2021-01-28T15:44:16Z

@ycombinator Thanks a lot for the review! I addressed your comments and introduced small changes in README. Please take a look and re-review whenever you've time.

ycombinator · 2021-01-29T04:42:05Z

docs/howto/system_testing.md

+
+`<service deployer>` - a name of the supported service deployer: `docker` or `tf` (terraform).
+
+### Docker Compose service deployer

 The `docker-compose.yml` file defines the integration service(s) for the package. If your package has a logs data stream, the log files from your package's integration service must be written to a volume. For example, the `apache` package has the following definition in it's integration service's `docker-compose.yml` file.


This sentence made sense earlier because the sample file structure above showed a docker-compose.yml file. Now the sample file structure above just says <service deployer files> so this sentence doesn't make as much sense.

I would add another introductory sentence to bridge this gap, something like:

When using the Docker Compose service deployer, the <service deployer files> must include a docker-compose.yml file.

Added here and for the TF service deployer

ycombinator · 2021-01-29T04:42:58Z

docs/howto/system_testing.md

+            <service deployer files>
+```
+
+`<service deployer>` - a name of the supported service deployer: `docker` or `tf` (terraform).


Might be worth clarifying that the docker name is for the Docker Compose service deployer and the tf name is for the Terraform service deployer. It's pretty obvious but it doesn't hurt to be explicit and tie these names to the sections that follow.

Good point, docker and docker compose are different products. Fixed

ycombinator · 2021-01-29T04:43:53Z

docs/howto/system_testing.md

-### Data stream-level configuration
+### Terraform service deployer
+
+The `*.tf` files define the infrastructure using the Terraform syntax. The terraform based service can be handy to boot up


Similar to my comment on the Docker Composer service deployer, I would add an introductory sentence to bridge the gap between what's shown in the sample file structure and this sentence here.

ycombinator · 2021-01-29T04:44:32Z

docs/howto/system_testing.md

+### Terraform service deployer
+
+The `*.tf` files define the infrastructure using the Terraform syntax. The terraform based service can be handy to boot up
+resources on AWS and use them for testing (e.g. spawn EC2 instance and collect its metrics).


Should we specifically mention AWS here? Or can we be more generic and say resources supported by Terraform or something like that?

ycombinator · 2021-01-29T04:50:19Z

internal/testrunner/runners/system/servicedeployer/terraform.go

+
+	// Build Terraform executor container name
+	serviceContainer := fmt.Sprintf("%s_terraform_1", service.project)
+	outCtxt.Hostname = serviceContainer


Do we need to set outCtxt.Hostname here? AFAICT it's not useful because the Agent configuration will never really need to talk to the Terraform executor container. Instead it will need to use the outCtxt.CustomProperties to talk to the service that is deployed by the Terraform executor.

We can try this, I hope everything will still work (it's not expected value somewhere). Fixed.

docs/howto/system_testing.md

Co-authored-by: Shaunak Kashyap <[email protected]>

ycombinator

LGTM. Very nice work on this!

ycombinator

LGTM.

mtojek added 2 commits January 19, 2021 13:33

Terraform executor as Docker image

75c6c41

Implement docker-compose for tf executor

fe54067

mtojek added 4 commits January 20, 2021 12:49

Enable _dev/deploy for data streams

4eb1f8b

Fix

2690b65

Merge branch '89-enable-deploy-everywhere' into 89-terraform-definitions

039e39a

Gohack: github.com/mtojek/package-spec/code/go

b324b83

mtojek mentioned this pull request Jan 20, 2021

[System test runner] Add more service deployers #89

Closed

mtojek added 20 commits January 20, 2021 15:05

Install service deployer as static resource

22c92c9

Cleanup

62a6e85

Define sample test config for AWS SNS

7ddd7fe

WIP

0f25284

Fix

6f7e4af

Build terraform environment

6bacf60

Merge branch 'master' into 89-terraform-definitions

deb2c26

Mount TF_DIR

1916b4f

Set envs in policies

6248a86

Fix: lint

3ce33c8

Fix: build dir (empty value)

3b5db75

Fix: yml file

958c43b

Fix: service name

ebdf53a

Fix: brackets

dcfb15c

Fix

2b8fac7

Fix: reference issue

12efcff

Fix: period

6ae050c

Set time

ba9cd9f

Do not print request body to prevent leaks

b908c9d

Fix: format

c1ded23

mtojek requested a review from ycombinator January 21, 2021 15:17

mtojek mentioned this pull request Jan 28, 2021

Terraform deployer: implement env-file #235

Closed

mtojek added 6 commits January 28, 2021 15:41

Link: env-file

c044a01

Link GH issue

a128259

Fix comment

38dd80e

Next improvements

133bf5d

nit: separator

f14ad49

Docs

8c116fb

mtojek requested a review from ycombinator January 28, 2021 15:44

ycombinator reviewed Jan 29, 2021

View reviewed changes

docs/howto/system_testing.md Outdated Show resolved Hide resolved

ycombinator reviewed Jan 29, 2021

View reviewed changes

docs/howto/system_testing.md Outdated Show resolved Hide resolved

mtojek and others added 6 commits January 29, 2021 11:40

Update docs/howto/system_testing.md

ebf857c

Co-authored-by: Shaunak Kashyap <[email protected]>

Update docs/howto/system_testing.md

8871d00

Co-authored-by: Shaunak Kashyap <[email protected]>

Introductory sentences

b43c3a1

Docker compose, terraform

2ea691f

More generic

5934ad0

Dont set hostname

4d4011f

mtojek requested a review from ycombinator January 29, 2021 11:55

ycombinator approved these changes Jan 29, 2021

View reviewed changes

mtojek added 3 commits January 29, 2021 17:29

Check if there is a single definition of service deployer

a97a874

Fix

34097cd

Another fix

e7e8764

mtojek requested a review from ycombinator January 29, 2021 17:22

ycombinator approved these changes Jan 29, 2021

View reviewed changes

mtojek merged commit dcebdb3 into elastic:master Jan 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement terraform based system tests #227

Implement terraform based system tests #227

mtojek commented Jan 19, 2021 •

edited

Loading

elasticmachine commented Jan 19, 2021 •

edited

Loading

Build stats

Test stats 🧪

mtojek commented Jan 21, 2021 •

edited

Loading

mtojek commented Jan 28, 2021

ycombinator Jan 29, 2021

mtojek Jan 29, 2021

ycombinator Jan 29, 2021

mtojek Jan 29, 2021 •

edited

Loading

ycombinator Jan 29, 2021

mtojek Jan 29, 2021

ycombinator Jan 29, 2021

mtojek Jan 29, 2021

ycombinator Jan 29, 2021

mtojek Jan 29, 2021

ycombinator left a comment •

edited

Loading

ycombinator left a comment

Implement terraform based system tests #227

Implement terraform based system tests #227

Conversation

mtojek commented Jan 19, 2021 • edited Loading

elasticmachine commented Jan 19, 2021 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

mtojek commented Jan 21, 2021 • edited Loading

mtojek commented Jan 28, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtojek Jan 29, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ycombinator left a comment • edited Loading

Choose a reason for hiding this comment

ycombinator left a comment

Choose a reason for hiding this comment

mtojek commented Jan 19, 2021 •

edited

Loading

elasticmachine commented Jan 19, 2021 •

edited

Loading

mtojek commented Jan 21, 2021 •

edited

Loading

mtojek Jan 29, 2021 •

edited

Loading

ycombinator left a comment •

edited

Loading