Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mute list): change allowlist to mute list #3039

Merged
merged 6 commits into from
Dec 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ name: "CodeQL"

on:
push:
branches: [ "master", prowler-2, prowler-3.0-dev ]
branches: [ "master", "prowler-4.0-dev" ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ "master" ]
branches: [ "master", "prowler-4.0-dev" ]
schedule:
- cron: '00 12 * * *'

Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ on:
push:
branches:
- "master"
- "prowler-4.0-dev"
pull_request:
branches:
- "master"
- "prowler-4.0-dev"
jobs:
build:
runs-on: ubuntu-latest
Expand Down
12 changes: 6 additions & 6 deletions docs/tutorials/configuration_file.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ The following list includes all the AWS checks with configurable variables that
| `organizations_delegated_administrators` | `organizations_trusted_delegated_administrators` | List of Strings |
| `ecr_repositories_scan_vulnerabilities_in_latest_image` | `ecr_repository_vulnerability_minimum_severity` | String |
| `trustedadvisor_premium_support_plan_subscribed` | `verify_premium_support_plans` | Boolean |
| `config_recorder_all_regions_enabled` | `allowlist_non_default_regions` | Boolean |
| `drs_job_exist` | `allowlist_non_default_regions` | Boolean |
| `guardduty_is_enabled` | `allowlist_non_default_regions` | Boolean |
| `securityhub_enabled` | `allowlist_non_default_regions` | Boolean |
| `config_recorder_all_regions_enabled` | `mute_non_default_regions` | Boolean |
| `drs_job_exist` | `mute_non_default_regions` | Boolean |
| `guardduty_is_enabled` | `mute_non_default_regions` | Boolean |
| `securityhub_enabled` | `mute_non_default_regions` | Boolean |

## Azure

Expand All @@ -50,8 +50,8 @@ The following list includes all the AWS checks with configurable variables that
aws:

# AWS Global Configuration
# aws.allowlist_non_default_regions --> Allowlist Failed Findings in non-default regions for GuardDuty, SecurityHub, DRS and Config
allowlist_non_default_regions: False
# aws.mute_non_default_regions --> Mute Failed Findings in non-default regions for GuardDuty, SecurityHub, DRS and Config
mute_non_default_regions: False

# AWS IAM Configuration
# aws.iam_user_accesskey_unused --> CIS recommends 45 days
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ There are different log levels depending on the logging information that is desi

- **DEBUG**: It will show low-level logs from Python.
- **INFO**: It will show all the API calls that are being invoked by the provider.
- **WARNING**: It will show all resources that are being **allowlisted**.
- **WARNING**: It will show all resources that are being **muted**.
- **ERROR**: It will show any errors, e.g., not authorized actions.
- **CRITICAL**: The default log level. If a critical log appears, it will **exit** Prowler’s execution.

Expand Down
68 changes: 34 additions & 34 deletions docs/tutorials/allowlist.md → docs/tutorials/mutelist.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# Allowlisting
# Mute Listing
Sometimes you may find resources that are intentionally configured in a certain way that may be a bad practice but it is all right with it, for example an AWS S3 Bucket open to the internet hosting a web site, or an AWS Security Group with an open port needed in your use case.

Allowlist option works along with other options and adds a `WARNING` instead of `INFO`, `PASS` or `FAIL` to any output format.
Mute List option works along with other options and adds a `MUTED` instead of `INFO`, `PASS` or `FAIL` to any output format.

You can use `-w`/`--allowlist-file` with the path of your allowlist yaml file, but first, let's review the syntax.
You can use `-w`/`--mutelist-file` with the path of your mutelist yaml file, but first, let's review the syntax.

## Allowlist Yaml File Syntax
## Mute List Yaml File Syntax

### Account, Check and/or Region can be * to apply for all the cases.
### Resources and tags are lists that can have either Regex or Keywords.
### Tags is an optional list that matches on tuples of 'key=value' and are "ANDed" together.
### Use an alternation Regex to match one of multiple tags with "ORed" logic.
### For each check you can except Accounts, Regions, Resources and/or Tags.
########################### ALLOWLIST EXAMPLE ###########################
Allowlist:
########################### MUTE LIST EXAMPLE ###########################
Mute List:
Accounts:
"123456789012":
Checks:
Expand Down Expand Up @@ -79,10 +79,10 @@ You can use `-w`/`--allowlist-file` with the path of your allowlist yaml file, b
Tags:
- "environment=prod" # Will ignore every resource except in account 123456789012 except the ones containing the string "test" and tag environment=prod

## Allowlist specific regions
If you want to allowlist/mute failed findings only in specific regions, create a file with the following syntax and run it with `prowler aws -w allowlist.yaml`:
## Mute specific regions
If you want to mute failed findings only in specific regions, create a file with the following syntax and run it with `prowler aws -w mutelist.yaml`:

Allowlist:
Mute List:
Accounts:
"*":
Checks:
Expand All @@ -93,50 +93,50 @@ If you want to allowlist/mute failed findings only in specific regions, create a
Resources:
- "*"

## Default AWS Allowlist
Prowler provides you a Default AWS Allowlist with the AWS Resources that should be allowlisted such as all resources created by AWS Control Tower when setting up a landing zone.
You can execute Prowler with this allowlist using the following command:
## Default AWS Mute List
Prowler provides you a Default AWS Mute List with the AWS Resources that should be muted such as all resources created by AWS Control Tower when setting up a landing zone.
You can execute Prowler with this mutelist using the following command:
```sh
prowler aws --allowlist prowler/config/aws_allowlist.yaml
prowler aws --mutelist prowler/config/aws_mutelist.yaml
```
## Supported Allowlist Locations
## Supported Mute List Locations

The allowlisting flag supports the following locations:
The mutelisting flag supports the following locations:

### Local file
You will need to pass the local path where your Allowlist YAML file is located:
You will need to pass the local path where your Mute List YAML file is located:
```
prowler <provider> -w allowlist.yaml
prowler <provider> -w mutelist.yaml
```
### AWS S3 URI
You will need to pass the S3 URI where your Allowlist YAML file was uploaded to your bucket:
You will need to pass the S3 URI where your Mute List YAML file was uploaded to your bucket:
```
prowler aws -w s3://<bucket>/<prefix>/allowlist.yaml
prowler aws -w s3://<bucket>/<prefix>/mutelist.yaml
```
> Make sure that the used AWS credentials have s3:GetObject permissions in the S3 path where the allowlist file is located.
> Make sure that the used AWS credentials have s3:GetObject permissions in the S3 path where the mutelist file is located.

### AWS DynamoDB Table ARN

You will need to pass the DynamoDB Allowlist Table ARN:
You will need to pass the DynamoDB Mute List Table ARN:

```
prowler aws -w arn:aws:dynamodb:<region_name>:<account_id>:table/<table_name>
```

1. The DynamoDB Table must have the following String keys:
<img src="../img/allowlist-keys.png"/>
<img src="../img/mutelist-keys.png"/>

- The Allowlist Table must have the following columns:
- Accounts (String): This field can contain either an Account ID or an `*` (which applies to all the accounts that use this table as an allowlist).
- The Mute List Table must have the following columns:
- Accounts (String): This field can contain either an Account ID or an `*` (which applies to all the accounts that use this table as an mutelist).
- Checks (String): This field can contain either a Prowler Check Name or an `*` (which applies to all the scanned checks).
- Regions (List): This field contains a list of regions where this allowlist rule is applied (it can also contains an `*` to apply all scanned regions).
- Resources (List): This field contains a list of regex expressions that applies to the resources that are wanted to be allowlisted.
- Tags (List): -Optional- This field contains a list of tuples in the form of 'key=value' that applies to the resources tags that are wanted to be allowlisted.
- Exceptions (Map): -Optional- This field contains a map of lists of accounts/regions/resources/tags that are wanted to be excepted in the allowlist.
- Regions (List): This field contains a list of regions where this mutelist rule is applied (it can also contains an `*` to apply all scanned regions).
- Resources (List): This field contains a list of regex expressions that applies to the resources that are wanted to be muted.
- Tags (List): -Optional- This field contains a list of tuples in the form of 'key=value' that applies to the resources tags that are wanted to be muted.
- Exceptions (Map): -Optional- This field contains a map of lists of accounts/regions/resources/tags that are wanted to be excepted in the mutelist.

The following example will allowlist all resources in all accounts for the EC2 checks in the regions `eu-west-1` and `us-east-1` with the tags `environment=dev` and `environment=prod`, except the resources containing the string `test` in the account `012345678912` and region `eu-west-1` with the tag `environment=prod`:
The following example will mute all resources in all accounts for the EC2 checks in the regions `eu-west-1` and `us-east-1` with the tags `environment=dev` and `environment=prod`, except the resources containing the string `test` in the account `012345678912` and region `eu-west-1` with the tag `environment=prod`:

<img src="../img/allowlist-row.png"/>
<img src="../img/mutelist-row.png"/>

> Make sure that the used AWS credentials have `dynamodb:PartiQLSelect` permissions in the table.

Expand All @@ -151,7 +151,7 @@ prowler aws -w arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME
Make sure that the credentials that Prowler uses can invoke the Lambda Function:

```
- PolicyName: GetAllowList
- PolicyName: GetMuteList
PolicyDocument:
Version: '2012-10-17'
Statement:
Expand All @@ -160,14 +160,14 @@ Make sure that the credentials that Prowler uses can invoke the Lambda Function:
Resource: arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME
```

The Lambda Function can then generate an Allowlist dynamically. Here is the code an example Python Lambda Function that
generates an Allowlist:
The Lambda Function can then generate an Mute List dynamically. Here is the code an example Python Lambda Function that
generates an Mute List:

```
def handler(event, context):
checks = {}
checks["vpc_flow_logs_enabled"] = { "Regions": [ "*" ], "Resources": [ "" ], Optional("Tags"): [ "key:value" ] }

al = { "Allowlist": { "Accounts": { "*": { "Checks": checks } } } }
al = { "Mute List": { "Accounts": { "*": { "Checks": checks } } } }
return al
```
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ nav:
- Slack Integration: tutorials/integrations.md
- Configuration File: tutorials/configuration_file.md
- Logging: tutorials/logging.md
- Allowlist: tutorials/allowlist.md
- Mute List: tutorials/mutelist.md
- Check Aliases: tutorials/check-aliases.md
- Custom Metadata: tutorials/custom-checks-metadata.md
- Ignore Unused Services: tutorials/ignore-unused-services.md
Expand Down
8 changes: 4 additions & 4 deletions prowler/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,12 @@
resolve_security_hub_previous_findings,
verify_security_hub_integration_enabled_per_region,
)
from prowler.providers.common.allowlist import set_provider_allowlist
from prowler.providers.common.audit_info import (
set_provider_audit_info,
set_provider_execution_parameters,
)
from prowler.providers.common.clean import clean_provider_local_output_directories
from prowler.providers.common.mutelist import set_provider_mutelist
from prowler.providers.common.outputs import set_provider_output_options
from prowler.providers.common.quick_inventory import run_provider_quick_inventory

Expand Down Expand Up @@ -173,12 +173,12 @@ def prowler():
# Sort final check list
checks_to_execute = sorted(checks_to_execute)

# Parse Allowlist
allowlist_file = set_provider_allowlist(provider, audit_info, args)
# Parse Mute List
mutelist_file = set_provider_mutelist(provider, audit_info, args)

# Set output options based on the selected provider
audit_output_options = set_provider_output_options(
provider, args, audit_info, allowlist_file, bulk_checks_metadata
provider, args, audit_info, mutelist_file, bulk_checks_metadata
)

# Run the quick inventory for the provider if available
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
Allowlist:
Mute List:
Accounts:
"*":
########################### AWS CONTROL TOWER ###########################
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
### Tags is an optional list that matches on tuples of 'key=value' and are "ANDed" together.
### Use an alternation Regex to match one of multiple tags with "ORed" logic.
### For each check you can except Accounts, Regions, Resources and/or Tags.
########################### ALLOWLIST EXAMPLE ###########################
Allowlist:
########################### MUTE LIST EXAMPLE ###########################
Mute List:
Accounts:
"123456789012":
Checks:
Expand Down
8 changes: 4 additions & 4 deletions prowler/config/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
aws:

# AWS Global Configuration
# aws.allowlist_non_default_regions --> Set to True to allowlist failed findings in non-default regions for AccessAnalyzer, GuardDuty, SecurityHub, DRS and Config
allowlist_non_default_regions: False
# If you want to allowlist/mute failed findings only in specific regions, create a file with the following syntax and run it with `prowler aws -w allowlist.yaml`:
# Allowlist:
# aws.mute_non_default_regions --> Set to True to mute failed findings in non-default regions for GuardDuty, SecurityHub, DRS and Config
mute_non_default_regions: False
# If you want to mute failed findings only in specific regions, create a file with the following syntax and run it with `prowler aws -w mutelist.yaml`:
# Mute List:
# Accounts:
# "*":
# Checks:
Expand Down
2 changes: 1 addition & 1 deletion prowler/lib/banner.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ def print_banner(args):
Color code for results:
- {Fore.YELLOW}INFO (Information){Style.RESET_ALL}
- {Fore.GREEN}PASS (Recommended value){Style.RESET_ALL}
- {orange_color}WARNING (Ignored by allowlist){Style.RESET_ALL}
- {orange_color}MUTED (Muted by muted list){Style.RESET_ALL}
- {Fore.RED}FAIL (Fix required){Style.RESET_ALL}
"""
)
10 changes: 5 additions & 5 deletions prowler/lib/check/check.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
from prowler.lib.logger import logger
from prowler.lib.outputs.outputs import report
from prowler.lib.utils.utils import open_file, parse_json_file
from prowler.providers.aws.lib.allowlist.allowlist import allowlist_findings
from prowler.providers.aws.lib.mutelist.mutelist import mutelist_findings
from prowler.providers.common.models import Audit_Metadata
from prowler.providers.common.outputs import Provider_Output_Options

Expand Down Expand Up @@ -560,10 +560,10 @@ def execute(
audit_info.audit_metadata, services_executed, checks_executed
)

# Allowlist findings
if audit_output_options.allowlist_file:
check_findings = allowlist_findings(
audit_output_options.allowlist_file,
# Mute List findings
if audit_output_options.mutelist_file:
check_findings = mutelist_findings(
audit_output_options.mutelist_file,
audit_info.audited_account,
check_findings,
)
Expand Down
2 changes: 1 addition & 1 deletion prowler/lib/outputs/html.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ def fill_html(file_descriptor, finding, output_options):
row_class = "table-info"
elif finding.status == "FAIL":
row_class = "table-danger"
elif finding.status == "WARNING":
elif finding.status == "MUTED":
row_class = "table-warning"
file_descriptor.write(
f"""
Expand Down
8 changes: 4 additions & 4 deletions prowler/lib/outputs/json.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,8 @@ def generate_json_asff_status(status: str) -> str:
json_asff_status = "PASSED"
elif status == "FAIL":
json_asff_status = "FAILED"
elif status == "WARNING":
json_asff_status = "WARNING"
elif status == "MUTED":
json_asff_status = "MUTED"
else:
json_asff_status = "NOT_AVAILABLE"

Expand Down Expand Up @@ -293,7 +293,7 @@ def generate_json_ocsf_status(status: str):
json_ocsf_status = "Success"
elif status == "FAIL":
json_ocsf_status = "Failure"
elif status == "WARNING":
elif status == "MUTED":
json_ocsf_status = "Other"
else:
json_ocsf_status = "Unknown"
Expand All @@ -307,7 +307,7 @@ def generate_json_ocsf_status_id(status: str):
json_ocsf_status_id = 1
elif status == "FAIL":
json_ocsf_status_id = 2
elif status == "WARNING":
elif status == "MUTED":
json_ocsf_status_id = 99
else:
json_ocsf_status_id = 0
Expand Down
4 changes: 2 additions & 2 deletions prowler/lib/outputs/outputs.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,12 +165,12 @@ def set_report_color(status: str) -> str:
color = Fore.RED
elif status == "ERROR":
color = Fore.BLACK
elif status == "WARNING":
elif status == "MUTED":
color = orange_color
elif status == "INFO":
color = Fore.YELLOW
else:
raise Exception("Invalid Report Status. Must be PASS, FAIL, ERROR or WARNING")
raise Exception("Invalid Report Status. Must be PASS, FAIL, ERROR or MUTED")
return color


Expand Down
10 changes: 5 additions & 5 deletions prowler/providers/aws/lib/arguments/arguments.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,14 +117,14 @@ def init_parser(self):
default=None,
help="Shodan API key used by check ec2_elastic_ip_shodan.",
)
# Allowlist
allowlist_subparser = aws_parser.add_argument_group("Allowlist")
allowlist_subparser.add_argument(
# Mute List
mutelist_subparser = aws_parser.add_argument_group("Mute List")
mutelist_subparser.add_argument(
"-w",
"--allowlist-file",
"--mutelist-file",
nargs="?",
default=None,
help="Path for allowlist yaml file. See example prowler/config/aws_allowlist.yaml for reference and format. It also accepts AWS DynamoDB Table or Lambda ARNs or S3 URIs, see more in https://docs.prowler.cloud/en/latest/tutorials/allowlist/",
help="Path for mutelist yaml file. See example prowler/config/aws_mutelist.yaml for reference and format. It also accepts AWS DynamoDB Table or Lambda ARNs or S3 URIs, see more in https://docs.prowler.cloud/en/latest/tutorials/mutelist/",
)

# Based Scans
Expand Down
Loading