Generate QHub Costs via `infracost` #1340

HarshCasper · 2022-06-22T17:34:40Z

Fixes #1315

Changes introduced in this PR:

Adds cost report generation using Infracost via qhub cost-report

Types of changes

What types of changes does your PR introduce?

Put an x in the boxes that apply

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds a feature)
Breaking change (fix or feature that would cause existing features to not work as expected)
Documentation Update
Code style update (formatting, renaming)
Refactoring (no functional changes, no API changes)
Build related changes
Other (please describe):

Testing

Requires testing

Yes
No

In case you checked yes, did you write tests?

Yes
No

Documentation

Does your contribution include breaking changes or deprecations?
If so have you updated the documentation?

Yes, docstrings
Yes, main documentation
Yes, deprecation notices

Further comments (optional)

Demo:

$ qhub cost --path=./stages/
       Cost Breakdown        
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━┓
┃               Name ┃ Cost ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━┩
│ Total Monthly Cost │   73 │
│  Total Hourly Cost │  0.1 │
└────────────────────┴──────┘
           Resource Breakdown            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃                         Name ┃ Number ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         Total Detected Costs │     11 │
│    Total Supported Resources │      3 │
│ Total Un-Supported Resources │      1 │
│   Total Non-Priced Resources │      7 │
│ Total Usage-Priced Resources │      3 │
└──────────────────────────────┴────────┘
Access the dashboard here: https://dashboard.infracost.io/share/6ubpwjvv7y2vjch2stant14ifnzzzv3l

viniciusdc

Looks good already for me, a quick note @HarshCasper do you think you can easily add a pytest section for this? if it's too much work, we can open this in a different issue.

Also, can you make --path=... an optional arg with default to stages?

viniciusdc

LGTM

iameskild

Well done @HarshCasper, this is great!

I left one comment, but otherwise this looks good to me 🎉

iameskild · 2022-06-22T23:47:24Z

qhub/cost.py

+        )
+        return infracost_data
+    except subprocess.CalledProcessError:
+        return None


Maybe a small note here about No data found?

Do you want a comment or an error message?

Could we explain what went wrong so perhaps an error message.

iameskild

Hey @HarshCasper, I decided to try this tool out myself to get a feel for it and I ran into a few issues. This is still on the right track 👍

I also opened an issue regarding how to setup and use this subcommand if you can get to it (or perhaps for the next sprint):
#57

iameskild · 2022-06-23T04:08:12Z

qhub/cli/cost.py

+logger = logging.getLogger(__name__)
+
+
+def create_cost_subcommand(subparser):


Could you add a little more info to the help command regarding what the different Resource Breakdown categories means?

iameskild · 2022-06-23T04:08:39Z

qhub/cost.py

+
+            cost_table = Table(title="Cost Breakdown")
+            cost_table.add_column("Name", justify="right", style="cyan", no_wrap=True)
+            cost_table.add_column("Cost", justify="right", style="cyan", no_wrap=True)


Cost ($) perhaps?

Or can we change currencies?

@iameskild We can change currencies! However, do we want to prioritise the feature right now?

I think just adding (USD) is a good solution

I agree, adding a way to change currencies is not a priority but let's add a label. USD as Kim suggested sounds good to me :)

@HarshCasper, would you mind just creating an issue listing a few of the enhancement requests raised in this PR and any others that you might suggest? Thanks a lot!!

iameskild · 2022-06-23T04:09:26Z

qhub/cost.py

+            console = Console()
+            console.print(cost_table)
+            console.print(resource_table)
+            print(f"Access the dashboard here: {data['shareUrl']}")


@HarshCasper when I trying this out myself, I kept running into this issue:

Traceback (most recent call last): File "/Users/eskild/opt/miniconda3/envs/qhub-dev/bin/qhub", line 33, in <module> sys.exit(load_entry_point('qhub', 'console_scripts', 'qhub')()) File "/Users/eskild/Documents/Quansight/qhub/qhub/__main__.py", line 7, in main cli(sys.argv[1:]) File "/Users/eskild/Documents/Quansight/qhub/qhub/cli/__init__.py", line 55, in cli args.func(args) File "/Users/eskild/Documents/Quansight/qhub/qhub/cli/cost.py", line 15, in handle_cost_report infracost_report(args.path) File "/Users/eskild/Documents/Quansight/qhub/qhub/cost.py", line 110, in infracost_report print(f"Access the dashboard here: {data['shareUrl']}") KeyError: 'shareUrl'

Would you mind having a look?

FWIW, I was trying this locally, trying to get an estimate for ~/Documents/Quansight/qhub-deployment-quansight-beta

Hi @iameskild 👋

I tried running it and check what the error was. Fortunately it worked for me:

Cost Breakdown ┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━┓ ┃ Name ┃ Cost ┃ ┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━┩ │ Total Monthly Cost │ 73 │ │ Total Hourly Cost │ 0.1 │ └────────────────────┴──────┘ Resource Breakdown ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓ ┃ Name ┃ Number ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩ │ Total Detected Costs │ 11 │ │ Total Supported Resources │ 3 │ │ Total Un-Supported Resources │ 1 │ │ Total Non-Priced Resources │ 7 │ │ Total Usage-Priced Resources │ 3 │ └──────────────────────────────┴────────┘ Access the dashboard here: https://dashboard.infracost.io/share/cuy3fqeeb3kqvcufrme7c9hlekcnuxa7

Can you please run these two commands and see what the output is:

infracost --version infracost configure get api_key

Hi @iameskild — I took a look at it and found the root cause. The JSON payload that we received did not have dashboards enabled in it, hence QHub errored out.

To enable this, we have to enable the dashboards via:

infracost configure set enable_dashboard true

Can you run this and try it out once? Since we are giving the users control over their Infracost installation, I suggest we add it out in the docs to let the users know to enable dashboards. Or we can add it as a flag that the user needs to add in case they wish to use it. What do you think? cc: @viniciusdc

Thanks for the update @HarshCasper! I tried it with enable_dashboard true and it worked. The dashboard link is very neat!

I think you bring up a good point about how much of the infracost setup process we should leave to the user. I'm inclined to have this tool handle the setup for the user. Handling the install with a qhub cost-estimate subcommand might be a too much to add right now but adding the flag to enable dashboards sounds like a reasonable addition.

And like you said, either way, we should document this :)

Added it 👍

I agree with @iameskild, we should let the user do the config by themselves which restricts the maintenance of this in the future.

kcpevey · 2022-06-23T14:21:46Z

qhub/cost.py

+
+                console = Console()
+                console.print(cost_table)
+                console.print(resource_table)


I think it would be neat to add the ability to write to csv and/or png, but that's just an idea. I don't want to hold up this PR for that.

Given that we are making an online accessible dashboard available to them, I don't think giving them a CSV feature would be much useful. However making them JSON available to compare with previous cost reports would be something I would work on next 👍

kcpevey · 2022-06-23T14:22:31Z

qhub/cost.py

+                console.print(cost_table)
+                console.print(resource_table)
+                try:
+                    print(f"Access the dashboard here: {data['shareUrl']}")


shouldn't these print statements be logger.info statements?

I guess its not a log statement but rather should be showcased on the CLI interface. Hence it needs to be a print statement in my opinion.

kcpevey

We need a documentation page describing how this works. Can you add that to the new repo?

kcpevey · 2022-06-23T14:26:36Z

I'd also like to see more information on what each of these categories mean as part of the print output. Maybe a paragraph before or after the breakdown (or bullet points, whatever makes sense)

HarshCasper · 2022-06-27T08:38:04Z

We need a documentation page describing how this works. Can you add that to the new repo?

It is being tracked here: #57

HarshCasper · 2022-06-27T08:38:48Z

I'd also like to see more information on what each of these categories mean as part of the print output. Maybe a paragraph before or after the breakdown (or bullet points, whatever makes sense)

Would it make sense to have it on docs rather than making the CLI interface cluttered?

iameskild

Hey @HarshCasper, I think there are a few items that @kcpevey and I raised that might need to be addressed before merging.

One of those items is reformatting or including explanations for the output table:

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃                         Name ┃ Number ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         Total Detected Costs │     11 │
│    Total Supported Resources │      3 │
│ Total Un-Supported Resources │      1 │
│   Total Non-Priced Resources │      7 │
│ Total Usage-Priced Resources │      3 │
└──────────────────────────────┴────────┘

Should these numbers add up? Are some of these related to the others? If so, how?

Also given that we are relying on node-pools (which fall under a Usage Resource and don't explicitly get captured here), could you please make a note that the general node-pool will always have one node running will add quite an additional charge.

As an example, on AWS, using an m5.2xlarge instance type for the general node-pool (currently the default), it will cost $0.384/hour * 24 hours * 30 days = $276.84 additional per month.

We can make further enhancements to this tool later but we should also capture those in an issue.

Thanks for your patience :)

qhub/cost.py

HarshCasper · 2022-06-28T07:33:17Z

Hi @iameskild @kcpevey

This is how the updated cost report looks like:

% qhub cost-estimate --path=/Users/harshcasper/code/quansight/qhub-deployment-quansight
         Cost Breakdown          
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃               Name ┃ Cost ($) ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Monthly Cost │       73 │
│  Total Hourly Cost │      0.1 │
└────────────────────┴──────────┘
           Resource Breakdown            
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃                         Name ┃ Number ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│         Total Detected Costs │     11 │
│    Total Supported Resources │      4 │
│ Total Un-Supported Resources │      0 │
│   Total Non-Priced Resources │      7 │
│ Total Usage-Priced Resources │      4 │
└──────────────────────────────┴────────┘
Access the dashboard here: 
https://dashboard.infracost.io/share/5c9spn9dv4sdqttfjlzvxcts9rlu7ib1

QHub rely upon node-pools which is a usage resource but doesn't get captured in 
the above report. A general node-pool will always have one node running will add
quite an additional charge. Please check in with your cloud provider to see the 
associated costs with node pools.                                               

 • Total Monthly Cost: The total monthly cost of the deployment of supported    
   resources.                                                                   
 • Total Hourly Cost: The total hourly cost of the deployment of supported      
   resources.                                                                   
 • Total Detected Costs: The total number of resources detected by Infracost.   
 • Total Supported Resources: The total number of resources supported by        
   Infracost.                                                                   
 • Total Un-Supported Resources: The total number of resources unsupported by   
   Infracost.                                                                   
 • Total Non-Priced Resources: The total number of resources that are not       
   priced.                                                                      
 • Total Usage-Priced Resources: The total number of resources that are priced  
   based on usage.

iameskild

Thanks a lot for the updates! This looks good to me 👍

HarshCasper · 2022-06-29T05:03:41Z

Thanks, @iameskild @viniciusdc and @kcpevey for all the reviews!

HarshCasper added 2 commits June 22, 2022 21:49

basic setup for infracost

35f64b2

feat: add the cost estimate integration to qhub

01722a5

HarshCasper changed the title ~~Infracost cli~~ Generate QHub Costs via infracost Jun 22, 2022

HarshCasper requested review from costrouc, iameskild and viniciusdc June 22, 2022 17:35

viniciusdc reviewed Jun 22, 2022

View reviewed changes

chore: make path optional

180d7d0

viniciusdc approved these changes Jun 22, 2022

View reviewed changes

iameskild approved these changes Jun 22, 2022

View reviewed changes

This was referenced Jun 22, 2022

[DOC] - Add Infracost tutorial nebari-dev/nebari-docs#57

Closed

OGC Disaster Pilot 2022 Sprint Meta Issue #1318

Closed

iameskild suggested changes Jun 23, 2022

View reviewed changes

iameskild added this to the Release v0.4.3 milestone Jun 23, 2022

chore: fixing minor nits

265be90

trallard added needs: review 👀 This PR is complete and ready for reviewing type: enhancement 💅🏼 New feature or request labels Jun 23, 2022

kcpevey reviewed Jun 23, 2022

View reviewed changes

HarshCasper added 2 commits June 27, 2022 14:00

enable infracost dahsboards by default

b247dec

add error log statements

edf6c9a

iameskild suggested changes Jun 27, 2022

View reviewed changes

iameskild reviewed Jun 27, 2022

View reviewed changes

qhub/cost.py Outdated Show resolved Hide resolved

chore: add more information about node pools

d1c7b8a

add infracost note

4088158

iameskild approved these changes Jun 28, 2022

View reviewed changes

Merge branch 'main' into infracost-cli

1850117

HarshCasper merged commit c43829a into main Jun 29, 2022

HarshCasper deleted the infracost-cli branch June 29, 2022 05:03

aktech mentioned this pull request Jun 30, 2022

Fix missing import: rich : broken qhub init with cloud #1353

Merged

15 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate QHub Costs via `infracost` #1340

Generate QHub Costs via `infracost` #1340

HarshCasper commented Jun 22, 2022

viniciusdc left a comment

viniciusdc left a comment

iameskild left a comment

iameskild Jun 22, 2022

HarshCasper Jun 23, 2022

iameskild Jun 27, 2022

iameskild left a comment

iameskild Jun 23, 2022 •

edited

Loading

iameskild Jun 23, 2022

iameskild Jun 23, 2022 •

edited

Loading

HarshCasper Jun 23, 2022

kcpevey Jun 23, 2022

iameskild Jun 23, 2022

iameskild Jun 23, 2022

iameskild Jun 23, 2022

HarshCasper Jun 23, 2022

HarshCasper Jun 23, 2022

HarshCasper Jun 23, 2022

iameskild Jun 23, 2022

HarshCasper Jun 27, 2022

viniciusdc Jun 28, 2022

kcpevey Jun 23, 2022

HarshCasper Jun 27, 2022

kcpevey Jun 23, 2022

HarshCasper Jun 27, 2022

kcpevey left a comment

kcpevey commented Jun 23, 2022

HarshCasper commented Jun 27, 2022

HarshCasper commented Jun 27, 2022

iameskild left a comment

HarshCasper commented Jun 28, 2022

iameskild left a comment

HarshCasper commented Jun 29, 2022

		logger = logging.getLogger(__name__)


		def create_cost_subcommand(subparser):

Generate QHub Costs via infracost #1340

Generate QHub Costs via infracost #1340

Conversation

HarshCasper commented Jun 22, 2022

Changes introduced in this PR:

Types of changes

Testing

Requires testing

In case you checked yes, did you write tests?

Documentation

Further comments (optional)

viniciusdc left a comment

Choose a reason for hiding this comment

viniciusdc left a comment

Choose a reason for hiding this comment

iameskild left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iameskild left a comment

Choose a reason for hiding this comment

iameskild Jun 23, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iameskild Jun 23, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kcpevey left a comment

Choose a reason for hiding this comment

kcpevey commented Jun 23, 2022

HarshCasper commented Jun 27, 2022

HarshCasper commented Jun 27, 2022

iameskild left a comment

Choose a reason for hiding this comment

HarshCasper commented Jun 28, 2022

iameskild left a comment

Choose a reason for hiding this comment

HarshCasper commented Jun 29, 2022

Generate QHub Costs via `infracost` #1340

Generate QHub Costs via `infracost` #1340

iameskild Jun 23, 2022 •

edited

Loading

iameskild Jun 23, 2022 •

edited

Loading