$PROFILE_OPT empty when working with assumed roles

[DB-Vincent]

Hi,

I'm trying to get Prowler working with assumed roles when executing the ./prowler -c checkXX -M html -p awsProfileWithSwitchRole -r region -A targetAccountId -R AssumedRole from my local machine. The issue is that I get 'Unable to locate credentials. You can configure credentials by running "aws configure".' and 'parse error: Invalid numeric literal at line 2, column 7' error messages.

From what we've found so far we can say the following things:

• The 'Unable to locate credentials. You can configure credentials by running "aws configure".' errors are not handled in include/assume_role
• The 'assume_role()'-function gets executed multiple times. The first time it gets executed correctly, but the executions after that execute it with the '$PROFILE_OPT' variable empty.
• With the provided hotfix (https://gist.github.com/toniblyx/f93e68c58e766be3be09a861d19ec718), no errors are shown, but the checks are executed on the 'awsProfileWithSwitchRole'-account and not on the 'targetAccountId'-account (as it should).

[toniblyx]

Thanks for reporting this issue and discuss it over Discord as well @DB-Vincent.

[halfluke]

I'm having the same issue and I found out that the latest version that works with assume_role seems to be prowler-2.3.0-18122020.
I did a diff on the two include/assume_role files: they are a bit different but I think something else is involved (credentials_report file for sure) and I was not able to fix the latest version

[toniblyx]

Thanks @halfluke that is a very helpful finding. I'll give it a look and see what is going on but please feel free to add here possible fixes if you find them.

[DB-Vincent]

I can confirm, prowler-2.3.0-18122020 works on my machine.

[toniblyx]

@halfluke as per @DB-Vincent finding, can you test it by making this change in prowler file? line 136, from this:

     p )
        PROFILE=$OPTARG
        ;;

to this:

     p )
        PROFILE=$OPTARG
        AWS_PROFILE=$OPTARG
        ;;

Not sure if this is the best way to solve the issue, however I have to diff 2.3 and latest to see what changes added this bug.

[toniblyx]

my gut feeling says that could be when I added the feature of renewing temporary credentials...

[DB-Vincent]

I feel obligated to mention that this is just a work around and that this probably isn't best practice. But it's a start!

[halfluke]

it doesn't seem to work either... I spent some hours trying to troubleshoot but no luck tbh
with change in line 136 + gist hotfix for assume_role file:
Date: Thu Nov 25 06:59:02 AM EST 2021
Access Denied trying to describe regions! Review permissions as described here: https://github.com/toniblyx/prowler/#requirements-and-installation
with change in line 136 only:
Date: Thu Nov 25 06:53:01 AM EST 2021
parse error: Invalid numeric literal at line 2, column 7
parse error: Invalid numeric literal at line 2, column 7
parse error: Invalid numeric literal at line 2, column 7
parse error: Invalid numeric literal at line 2, column 7
Generating AWS IAM Credential Report... - []
WARNING! Generate credential report unsuccessful

A csv file is generated with the correct account name but it has only one line...

[toniblyx]

Ok, I have tested this diff in my environment and it is working fine for this command ./prowler -g cislevel1 -M html -p 12345678912_AWSAdministratorAccess -r us-east-1 -A 11111111111 -R ProwlerExecRole

diff --git a/include/assume_role b/include/assume_role
index 95bd3e0..7445e51 100644
--- a/include/assume_role
+++ b/include/assume_role
@@ -58,6 +58,10 @@ assume_role(){
       textFail "The requested DurationSeconds exceeds the MaxSessionDuration set for the role ${PROWLER_ROLE}"
       EXITCODE=1
       exit $EXITCODE
+    elif [[ "$(grep ExpiredToken $TEMP_STS_ASSUMED_FILE)" ]]; then
+      textFail "ExpiredToken error with role: ${PROWLER_ROLE} and AWS Profile settings: $PROFILE_OPT"
+      EXITCODE=1
+      exit $EXITCODE
     fi

     # assume role command
@@ -76,8 +80,10 @@ assume_role(){
     fi

     # The profile shouldn't be used for CLI
-    PROFILE=""
-    PROFILE_OPT=""
+    if [[ $PROFILE == 'default' ]]; then
+        PROFILE=""
+        PROFILE_OPT=""
+    fi

Could you guys test it? Don't change anything in prowler file code.

[halfluke]

sorry but... what do I change in assume_role?

[toniblyx]

grab it from here, it is easier :) https://gist.github.com/toniblyx/8d4e5e2420b2185332ab35af067f8470

[halfluke]

Access Denied trying to describe regions! Review permissions as described here: https://github.com/toniblyx/prowler/#requirements-and-installation
and if I comment out get_regions in the main prowler file:
Generating AWS IAM Credential Report... - []
WARNING! Access Denied trying to generate credential report

the regions error is handled differently in Prowler 2.3.0.xxxx: I still get it but the script proceeds

I bet the real problem is the second one: the newest versions try to generate credentials report for the CALLER account, not for the "assumed role" account. I think you don't notice that because you have permissions on the caller account as well... but I don't in my environment

[toniblyx]

are you replacing that file in the latest release from master branch? Can you share the exact command you are running. when running aws sts get-caller-identity that principal can assume the role properly and that assumed role has enough permissions? It sounds more like a permissions issue. Deploy this template in the target account and it will help you to make sure that prowler has the right permissions https://github.com/toniblyx/prowler/blob/master/iam/create_role_to_assume_cfn.yaml

[toniblyx]

can you share what permission do you have in the source account? I'll try to replicate

[halfluke]

unfortunately it's on a client account that I cannot change. The role I assume has SecurityAudit in the target account. The keys I have are for a user that can "switch role" and assume the role I need to audit in the target account.
It is a sort of permission issue... but in prowler 2.3.0 it works :)

With the 2.3.0 version of Prowler I have:
└─# ./prowler -p profile -A targetaccount -R roleinthetargetaccount -g cislevel1 -M csv,mono

An error occurred (UnauthorizedOperation) when calling the DescribeRegions operation: You are not authorized to perform this operation.
_
_ __ _ __ _____ | | ___ _ __
| ' | '/ _ \ \ /\ / / |/ _ \ '|
| |) | | | () \ V V /| | / |
| ./|| ___/ _/_/ ||___||v2.3.0-18122020
|| the handy cloud security tool

Date: Thu Nov 25 11:18:15 AM EST 2021

This report is being generated using credentials below:

AWS-CLI Profile: [Profile] AWS API Region: [us-east-1] AWS Filter Region: [all]
AWS Account: [TARGET_ACCOUNT] UserId: [xxxxxxxxxxxx]
Caller Identity ARN: [arn:aws:iam::CALLER_ACCOUNT:user/USER_IN_THE_CALLER_ACCOUNT]
5.0 CIS Level 1 - CIS only - [cislevel1] ***************************

[toniblyx]

ok, the source account has permissions only to assume a role targed account. I have to test that, also remember that with SecurityAudit role only Prowler may be unable to make all checks.

[halfluke]

oh and yes, I replaced the file of the gist in the latest release of course. I had reached that point with my experiments.
Perhaps also @DB-Vincent has a similar configuration and limitation of permissions but... did you check if with your "fix" prowler actually runs the credential reports (and the CIS benchmark) on the source instead of the target?
I can confirm that the csv generated by 2.3.0.xxx contains all the checks for the correct target account. This is the mysterious part that I was not able to understand how to fix. (Well, if I copy assume_role, credentials_report and prowler from version 2.3.0 over to the latest version, it works :-) But it doesn't count that way
I asked /job-function/ViewOnlyAccess permission on the target as well but I can see a terraform policy attached to my role instead... clients!

[DB-Vincent]

@toniblyx I just tested this in my environment, and I do not get any error messages. However, the checks get executed on the awsProfileWithSwitchRole and not the targetAccount.

[toniblyx]

ok, I could finally replicate that case in the latest version and seen that initial credentials are unset that is why assume role works first then gets unset due to a bug in the reload credentials logic.

[toniblyx]

Just to clarify for other users. This bug, initially only affects to those users running Prowler manually in account A that assumes a role in account B by using a custom profile set up in thir own ~/.aws/credentials. For those using EC2 instance or containers with a role attached it does not affect them.

[halfluke]

great stuff, i was not able to figure out the "unset" part and the fact that I also have limited permissions mixed things up. But you are the author for a reason :)

[jfagoagas]

Hi @DB-Vincent,

Can you execute it again with the following assume_role file?

#!/usr/bin/env bash

# Prowler - the handy cloud security tool (copyright 2019) by Toni de la Fuente
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy
# of the License at http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software distributed
# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
# CONDITIONS OF ANY KIND, either express or implied. See the License for the
# specific language governing permissions and limitations under the License.

# both variables are mandatory to be set together
assume_role(){
    if [[ -z $ROLE_TO_ASSUME ]]; then
        echo "$OPTRED ERROR!$OPTNORMAL - Both Account ID (-A) and IAM Role to assume (-R) must be set"
        exit 1
    fi
    # if not session duration set with -T, then will be 1h.
    # In some cases you will need more than 1h.
    if [[ -z $SESSION_DURATION_TO_ASSUME ]]; then
        SESSION_DURATION_TO_ASSUME="3600"
    elif [[ "${SESSION_DURATION_TO_ASSUME}" -gt "43200" ]] || [[ "${SESSION_DURATION_TO_ASSUME}" -lt "900" ]]; then
        echo "$OPTRED ERROR!$OPTNORMAL - Role session duration must be more than 900 seconds and less than 4300 seconds"
        exit 1
    fi

    # temporary file where to store credentials
    TEMP_STS_ASSUMED_FILE=$(mktemp -t prowler.sts_assumed-XXXXXX)

    # check if role arn or role name
    if [[ $ROLE_TO_ASSUME == arn:* ]]; then
        PROWLER_ROLE=$ROLE_TO_ASSUME
    else
        PROWLER_ROLE=arn:${AWS_PARTITION}:iam::$ACCOUNT_TO_ASSUME:role/$ROLE_TO_ASSUME
    fi

    #Check if external ID has bee provided if so execute with external ID if not ignore
    if [[ -z $ROLE_EXTERNAL_ID ]]; then
        # assume role command
        if ! $AWSCLI $PROFILE_OPT sts assume-role --role-arn $PROWLER_ROLE \
            --role-session-name ProwlerAssessmentSession \
            --region $REGION_FOR_STS \
            --duration-seconds $SESSION_DURATION_TO_ASSUME > $TEMP_STS_ASSUMED_FILE
        then
            EXITCODE=1
            exit $EXITCODE
        fi
    else
        if ! $AWSCLI $PROFILE_OPT sts assume-role --role-arn $PROWLER_ROLE \
            --role-session-name ProwlerAssessmentSession \
            --duration-seconds $SESSION_DURATION_TO_ASSUME \
            --region $REGION_FOR_STS \
            --external-id $ROLE_EXTERNAL_ID > $TEMP_STS_ASSUMED_FILE
        then
            EXITCODE=1
            exit $EXITCODE
        fi
    fi
   
    # The profile shouldn't be used for CLI
    if [[ $PROFILE == 'default' ]]; then
        PROFILE=""
        PROFILE_OPT=""
    fi

    # set env variables with assumed role credentials
    export AWS_ACCESS_KEY_ID=$(cat $TEMP_STS_ASSUMED_FILE | jq -r '.Credentials.AccessKeyId')
    export AWS_SECRET_ACCESS_KEY=$(cat $TEMP_STS_ASSUMED_FILE | jq -r '.Credentials.SecretAccessKey')
    export AWS_SESSION_TOKEN=$(cat $TEMP_STS_ASSUMED_FILE | jq -r '.Credentials.SessionToken')
    export AWS_SESSION_EXPIRATION=$(cat $TEMP_STS_ASSUMED_FILE | jq -r '.Credentials.Expiration | sub("\\+00:00";"Z") | fromdateiso8601')
    cleanSTSAssumeFile
}

cleanSTSAssumeFile() {
    rm -fr "${TEMP_STS_ASSUMED_FILE}"
}

[halfluke]

to resume: in my environment with limited permission for the caller account:
hotfix https://gist.github.com/toniblyx/f93e68c58e766be3be09a861d19ec718
and hotfix https://gist.github.com/toniblyx/8d4e5e2420b2185332ab35af067f8470:
Generating AWS IAM Credential Report... - []
WARNING! Access Denied trying to generate credential report -> it tries to evaluate the CALLER account, not the "assumed" one

Latest assume_role above:
Unable to locate credentials. You can configure credentials by running "aws configure".

Can't the same logic as in Prowler 2.3.0.xxx , re-implemented in the current version?

[halfluke]

My fix (no PR yet but let me know if you wish):

1. The assume_role renewal logic in 'prowler' file:

# Function to execute the check
execute_check() {

  if [[ $ACCOUNT_TO_ASSUME ]]; then
    # Following logic looks for time remaining in the session and review it
    # if it is less than 600 seconds, 10 minutes.
    CURRENT_TIMESTAMP=$(date -u "+%s")
    SESSION_TIME_REMAINING=$(expr $AWS_SESSION_EXPIRATION - $CURRENT_TIMESTAMP)
    echo $SESSION_TIME_REMAINING
    MINIMUM_REMAINING_TIME_ALLOWED=600
    if (( $MINIMUM_REMAINING_TIME_ALLOWED > $SESSION_TIME_REMAINING )); then
      echo SUKA
      unset AWS_ACCESS_KEY_ID
      unset AWS_SECRET_ACCESS_KEY
      unset AWS_SESSION_TOKEN
      assume_role
    fi
  fi

if (( $MINIMUM_REMAINING_TIME_ALLOWED > $SESSION_TIME_REMAINING )); then
instead of
if [[ $MINIMUM_REMAINING_TIME_ALLOWED > $SESSION_TIME_REMAINING ]]; then
fixes the "time remaining" comparison check

2. added extra variable in include/aws_profile_loader:

PROFILE_OPT_BAK=$PROFILE_OPT
# Set default region by aws config, fall back to us-east-1

3. my assume_role file:

!/usr/bin/env bash

# Prowler - the handy cloud security tool (copyright 2019) by Toni de la Fuente
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy
# of the License at http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software distributed
# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
# CONDITIONS OF ANY KIND, either express or implied. See the License for the
# specific language governing permissions and limitations under the License.

# both variables are mandatory to be set together

assume_role(){
    echo ASSUMING

    echo PROF $PROFILE
    echo PROF_OPT_BAK $PROFILE_OPT_BAK
    echo PROF_OPT $PROFILE_OPT

    PROFILE_OPT=$PROFILE_OPT_BAK

    echo PROF_OPT $PROFILE_OPT
    if [[ -z $ROLE_TO_ASSUME ]]; then
        echo "$OPTRED ERROR!$OPTNORMAL - Both Account ID (-A) and IAM Role to assume (-R) must be set"
        exit 1
    fi
    # if not session duration set with -T, then will be 1h.
    # In some cases you will need more than 1h.
    if [[ -z $SESSION_DURATION_TO_ASSUME ]]; then
        SESSION_DURATION_TO_ASSUME="3600"
    elif [[ "${SESSION_DURATION_TO_ASSUME}" -gt "43200" ]] || [[ "${SESSION_DURATION_TO_ASSUME}" -lt "900" ]]; then
        echo "$OPTRED ERROR!$OPTNORMAL - Role session duration must be more than 900 seconds and less than 4300 seconds"
        exit 1
    fi

    # temporary file where to store credentials
    TEMP_STS_ASSUMED_FILE=$(mktemp -t prowler.sts_assumed-XXXXXX)
    # check if role arn or role name
    if [[ $ROLE_TO_ASSUME == arn:* ]]; then
        PROWLER_ROLE=$ROLE_TO_ASSUME
    else
        PROWLER_ROLE=arn:${AWS_PARTITION}:iam::$ACCOUNT_TO_ASSUME:role/$ROLE_TO_ASSUME
    fi

    #Check if external ID has bee provided if so execute with external ID if not ignore
    if [[ -z $ROLE_EXTERNAL_ID ]]; then
        # assume role command
        $AWSCLI $PROFILE_OPT sts assume-role --role-arn $PROWLER_ROLE \
            --role-session-name ProwlerAssessmentSession \
            --region $REGION_FOR_STS \
            --duration-seconds $SESSION_DURATION_TO_ASSUME > $TEMP_STS_ASSUMED_FILE
    else
        $AWSCLI $PROFILE_OPT sts assume-role --role-arn $PROWLER_ROLE \
            --role-session-name ProwlerAssessmentSession \
            --duration-seconds $SESSION_DURATION_TO_ASSUME \
            --region $REGION_FOR_STS \
            --external-id $ROLE_EXTERNAL_ID > $TEMP_STS_ASSUMED_FILE
    fi
    echo $TEMP_STS_ASSUMED_FILE
   
    # The profile shouldn't be used for CLI
    PROFILE=""
    PROFILE_OPT=""

    # set env variables with assumed role credentials
    export AWS_ACCESS_KEY_ID=$(cat $TEMP_STS_ASSUMED_FILE | jq -r '.Credentials.AccessKeyId')
    export AWS_SECRET_ACCESS_KEY=$(cat $TEMP_STS_ASSUMED_FILE | jq -r '.Credentials.SecretAccessKey')
    export AWS_SESSION_TOKEN=$(cat $TEMP_STS_ASSUMED_FILE | jq -r '.Credentials.SessionToken')
    export AWS_SESSION_EXPIRATION=$(cat $TEMP_STS_ASSUMED_FILE | jq -r '.Credentials.Expiration | sub("\\+00:00";"Z") | fromdateiso8601')
    echo $AWS_ACCESS_KEY_ID
    echo $AWS_SESSION_TOKEN
    echo $AWS_SESSION_EXPIRATION
    cleanSTSAssumeFile

    #The profile shouldn't be used for CLI
    #PROFILE=""
    #PROFILE_OPT=""

}

cleanSTSAssumeFile() {
    rm -fr "${TEMP_STS_ASSUMED_FILE}"
}

You can remove the 'echo' that I left for debugging purposes. The wrong comparison with [[ ]] would refresh assume_role for each check, but you should see that the whole process now works in all cases (single and multiple assume_role). Hopefully.

[jfagoagas]

Really good catch @halfluke !!!! 👏👏👏👏👏

Please open a PR with all of theses changes 🙏

[halfluke]

I will do. Just running one more time, a complete run with all the extras.
It looks all fixed to me: when the remaining time for the temporary credentials is less than 600, it calls assume_role again, but first restores the original profile, because if you call again assume_role while impersonating the role, you get a Permission Denied error (the role you assumed cannot assume itself). And of course after each assume_role you have to ensure that the temporary credentials exported in environment variables are used instead of the original profile when running prowler, otherwise you'll run prowler on the caller account and not on the target (and if you do not have permissions on the caller account you get access denied again)

[jfagoagas]

Really really good!!!!

Impressive 👏👏

[DB-Vincent]

I've just tested @halfluke's setup and I can confirm it also works on my machine. Seems like the issues is solved with those fixes.

[toniblyx]

Awesome work @halfluke, thanks a lot for the effort put here to you all! Let me know if you can send the PR, I'm going to test it here as well.

[halfluke]

Hi yes I will submit the PR today or tomorrow... unless you notice something wrong in your test?

[toniblyx]

I can't manage to make it work but re-testing it again, feel free to send the PR and I'll keep my tests, probably I didn't do the copy/paste correctly

[halfluke]

ok I will submit the PR today. Perhaps check if you added
PROFILE_OPT_BAK=$PROFILE_OPT
to aws_profile_loader
and
PROFILE_OPT=$PROFILE_OPT_BAK
at the beginning of assume_role()

[toniblyx]

quick question @halfluke the role or user of the profile you are using, what actions have allowed? I'm testing with a role that has only sts:assumerole and getting Access Denied trying to describe regions do you get the same?

[halfluke]

@toniblyx nope, that was one of the problems but it works now. That's why I put some echo to debug it. Old version of prowler would continue even without getting regions, newer versions stop and exit.
With my echo I get:
REGIONS_init
eu-north-1 ap-south-1 eu-west-3 eu-west-2 eu-west-1 ap-northeast-3 ap-northeast-2 ap-northeast-1 sa-east-1 ca-central-1 ap-southeast-1 ap-southeast-2 eu-central-1 us-east-1 us-east-2 us-west-1 us-west-2
REGIONS_end
It should work when you run it on the target account

Not sure of the permissions of my user in the caller account as I cannot read them ( sts:assumerole for sure)
The role I assume in the target account has the following

[toniblyx]

I have merged @halfluke suggestions into the latest branch in 2.7 to test. Once is all right I´ll remove echoes in code.

[DB-Vincent]

I can confirm that the 2.7 branch works in my environment.

[toniblyx]

awesome, it is working fine for me as well so far. Thanks!

[halfluke]

did you find out what your problem with regions was? In the end after all the time it took me, I've noticed that the changes I made are really minimal: the [[ -> (( comparison and a backup of profile_opt to be restored when we re-assume a role :-)

[toniblyx]

yep, it was a really good catch @halfluke! The region issue is fixed now as far as I can see, still testing with different options.

[toniblyx]

This issue has been fixed. Anything related or additional please open a new issue for better handling.

[halfluke]

Hi, Can you please clarify in which version of prowler it is fixed? Thanks

[toniblyx]

branch 2.7. That will be the next release, it is the devel branch at this point.

git clone --branch 2.7 https://github.com/toniblyx/prowler