Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support more run modes #71

Open
yingsu00 opened this issue Aug 31, 2022 · 1 comment
Open

Support more run modes #71

yingsu00 opened this issue Aug 31, 2022 · 1 comment

Comments

@yingsu00
Copy link
Contributor

yingsu00 commented Aug 31, 2022

We shall support at least these run modes:

  1. Power
    Run one query at a time, with the pre-designated query order. For example, TPCH requires the Power test run 1 stream of 22 queries, with the query order 14 2 9 20 6 17 18 8 21 13 3 22 16 4 11 15 1 10 19 5 7 12. In regular tracking, we may want to do cold/warm runs, so each query would be run runs times, with the first execution being cold, the rest being warm. We will take the last warm execution as the reporting query. Before the first execution of each query, we need to run macros to clean the caches. E.g. if runs = 3, then the queries would be run clean caches, 14(cold), 14, 14(warm), clean caches, 2(cold), 2, 2(warm)...

  2. Throughput
    Run s streams concurrently, and each stream needs to follow the specified query orders. e.g. See https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-h_v2.17.1.pdf for the query orders for TPCH tests. In Throughput mode, we usually don't do the cold/warm runs as in Power phase, but rather once per query. There can be a pre-warm run before the real run, usually being the Power run. So the runs parameter can actually be neglected.

  3. Concurrent
    Run many queries using a pre-defined query concurrency(threads), e.g. 1000 queries run on 40 threads. Each thread just pick up a random query from the query set. Each query is executed runs times but the order shall be randomized. Usually the runs = 1. This is to simulate a real workload on customers' clusters. Note that we will need to be able to supply randomly chosen queries from the query history, if we have any.

  4. Concurrent-Repeating
    This is what Adding throughput test to benchto #68 does. With pre-defined concurrency level, each thread executes the same listed queries with runs times. With that PR each query would be executed runs * concurrency times. This mode is usually used if we want to run a single query on all threads for a period of time, possibly to reproduce some bugs.

The current implementation supports 1 and 4 but not 2 and 3. We shall support all 4 modes.

@yingsu00 yingsu00 changed the title Support all more run modes Support more run modes Aug 31, 2022
@yingsu00
Copy link
Contributor Author

cc @upbram123

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant