You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Power
Run one query at a time, with the pre-designated query order. For example, TPCH requires the Power test run 1 stream of 22 queries, with the query order 14 2 9 20 6 17 18 8 21 13 3 22 16 4 11 15 1 10 19 5 7 12. In regular tracking, we may want to do cold/warm runs, so each query would be run runs times, with the first execution being cold, the rest being warm. We will take the last warm execution as the reporting query. Before the first execution of each query, we need to run macros to clean the caches. E.g. if runs = 3, then the queries would be run clean caches, 14(cold), 14, 14(warm), clean caches, 2(cold), 2, 2(warm)...
Throughput
Run s streams concurrently, and each stream needs to follow the specified query orders. e.g. See https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-h_v2.17.1.pdf for the query orders for TPCH tests. In Throughput mode, we usually don't do the cold/warm runs as in Power phase, but rather once per query. There can be a pre-warm run before the real run, usually being the Power run. So the runs parameter can actually be neglected.
Concurrent
Run many queries using a pre-defined query concurrency(threads), e.g. 1000 queries run on 40 threads. Each thread just pick up a random query from the query set. Each query is executed runs times but the order shall be randomized. Usually the runs = 1. This is to simulate a real workload on customers' clusters. Note that we will need to be able to supply randomly chosen queries from the query history, if we have any.
Concurrent-Repeating
This is what Adding throughput test to benchto #68 does. With pre-defined concurrency level, each thread executes the same listed queries with runs times. With that PR each query would be executed runs * concurrency times. This mode is usually used if we want to run a single query on all threads for a period of time, possibly to reproduce some bugs.
The current implementation supports 1 and 4 but not 2 and 3. We shall support all 4 modes.
The text was updated successfully, but these errors were encountered:
yingsu00
changed the title
Support all more run modes
Support more run modes
Aug 31, 2022
We shall support at least these run modes:
Power
Run one query at a time, with the pre-designated query order. For example, TPCH requires the Power test run 1 stream of 22 queries, with the query order
14 2 9 20 6 17 18 8 21 13 3 22 16 4 11 15 1 10 19 5 7 12
. In regular tracking, we may want to do cold/warm runs, so each query would be runruns
times, with the first execution being cold, the rest being warm. We will take the last warm execution as the reporting query. Before the first execution of each query, we need to run macros to clean the caches. E.g. ifruns = 3
, then the queries would be runclean caches, 14(cold), 14, 14(warm), clean caches, 2(cold), 2, 2(warm)...
Throughput
Run
s
streams concurrently, and each stream needs to follow the specified query orders. e.g. See https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-h_v2.17.1.pdf for the query orders for TPCH tests. In Throughput mode, we usually don't do the cold/warm runs as in Power phase, but rather once per query. There can be a pre-warm run before the real run, usually being the Power run. So theruns
parameter can actually be neglected.Concurrent
Run many queries using a pre-defined query concurrency(threads), e.g. 1000 queries run on 40 threads. Each thread just pick up a random query from the query set. Each query is executed
runs
times but the order shall be randomized. Usually theruns
= 1. This is to simulate a real workload on customers' clusters. Note that we will need to be able to supply randomly chosen queries from the query history, if we have any.Concurrent-Repeating
This is what Adding throughput test to benchto #68 does. With pre-defined concurrency level, each thread executes the same listed queries with
runs
times. With that PR each query would be executedruns * concurrency
times. This mode is usually used if we want to run a single query on all threads for a period of time, possibly to reproduce some bugs.The current implementation supports 1 and 4 but not 2 and 3. We shall support all 4 modes.
The text was updated successfully, but these errors were encountered: