Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](routine_load) change librdkafka version from 2.0.2 to 1.9.2 to avoid memory leak #28528

Merged
merged 1 commit into from
Dec 18, 2023
Merged

[fix](routine_load) change librdkafka version from 2.0.2 to 1.9.2 to avoid memory leak #28528

merged 1 commit into from
Dec 18, 2023

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented Dec 17, 2023

Proposed changes

=================================================================
03:11:44   ==4386==ERROR: LeakSanitizer: detected memory leaks
03:11:44   
03:11:44   Indirect leak of 1048 byte(s) in 1 object(s) allocated from:
03:11:44       #0 0x56147d7b57d8 in calloc (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x156297d8) (BuildId: f0d4dff9c34d29dd)
03:11:44       #1 0x5614bc89d4b4 in rd_kafka_toppar_new0 (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x547114b4) (BuildId: f0d4dff9c34d29dd)
03:11:44   
03:11:44   Indirect leak of 1000 byte(s) in 1 object(s) allocated from:
03:11:44       #0 0x56147d7b57d8 in calloc (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x156297d8) (BuildId: f0d4dff9c34d29dd)
03:11:44       #1 0x5614bc85e29e in rd_kafka_topic_new0 (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x546d229e) (BuildId: f0d4dff9c34d29dd)
03:11:44   
03:11:44   Indirect leak of 528 byte(s) in 3 object(s) allocated from:
03:11:44       #0 0x56147d7b55ee in malloc (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x156295ee) (BuildId: f0d4dff9c34d29dd)
03:11:44       #1 0x5614bc86e374 in rd_kafka_q_new0 (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x546e2374) (BuildId: f0d4dff9c34d29dd)
03:11:44   
03:11:44   Indirect leak of 128 byte(s) in 1 object(s) allocated from:
03:11:44       #0 0x56147d7b55ee in malloc (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x156295ee) (BuildId: f0d4dff9c34d29dd)
03:11:44       #1 0x5614bc8b93fd in rd_list_init (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x5472d3fd) (BuildId: f0d4dff9c34d29dd)
03:11:44   
03:11:44   Indirect leak of 112 byte(s) in 1 object(s) allocated from:
03:11:44       #0 0x56147d7b57d8 in calloc (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x156297d8) (BuildId: f0d4dff9c34d29dd)
03:11:44       #1 0x5614bc870abd in rd_kafka_op_new0 (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x546e4abd) (BuildId: f0d4dff9c34d29dd)
03:11:44   
03:11:44   Indirect leak of 23 byte(s) in 1 object(s) allocated from:
03:11:44       #0 0x56147d7b55ee in malloc (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x156295ee) (BuildId: f0d4dff9c34d29dd)
03:11:44       #1 0x5614bc85e2dc in rd_kafka_topic_new0 (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x546d22dc) (BuildId: f0d4dff9c34d29dd)
03:11:44   
03:11:44   Indirect leak of 20 byte(s) in 2 object(s) allocated from:
03:11:44       #0 0x56147d79f6ad in strdup (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x156136ad) (BuildId: f0d4dff9c34d29dd)
03:11:44       #1 0x5614bc867935  (/root/doris/be/ut_build_ASAN/test/doris_be_test+0x546db935) (BuildId: f0d4dff9c34d29dd)
03:11:44   
03:11:44   SUMMARY: AddressSanitizer: 2859 byte(s) leaked in 10 allocation(s).

Memory leak happened when upgrade librdkafka version from 1.8.2 to 2.0.2(#28210), refer to confluentinc/librdkafka#4486, bug still exist in lastest version, so we change version to 1.9.2.

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@sollhui
Copy link
Contributor Author

sollhui commented Dec 17, 2023

run buildall

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit e1b0c38dc6f3b8ad7a6e5e48565f3fd93922c777, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4723	4463	4471	4463
q2	364	152	157	152
q3	1458	1234	1247	1234
q4	1113	878	875	875
q5	3160	3131	3153	3131
q6	246	128	129	128
q7	988	492	487	487
q8	2201	2224	2182	2182
q9	6698	6655	6687	6655
q10	3226	3250	3254	3250
q11	324	203	194	194
q12	346	213	209	209
q13	4553	3819	3802	3802
q14	237	211	215	211
q15	571	529	511	511
q16	437	376	387	376
q17	1015	639	577	577
q18	7208	6892	6986	6892
q19	1544	1447	1322	1322
q20	518	289	290	289
q21	3043	2631	2702	2631
q22	342	277	282	277
Total cold run time: 44315 ms
Total hot run time: 39848 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4429	4388	4415	4388
q2	269	166	169	166
q3	3536	3508	3516	3508
q4	2374	2368	2369	2368
q5	5731	5700	5723	5700
q6	244	120	123	120
q7	2385	1876	1868	1868
q8	3510	3533	3507	3507
q9	9041	8992	9008	8992
q10	3926	3984	4021	3984
q11	492	390	387	387
q12	765	605	586	586
q13	4300	3562	3559	3559
q14	287	258	249	249
q15	572	511	523	511
q16	506	486	491	486
q17	1864	1892	1831	1831
q18	8615	8244	8313	8244
q19	1761	1762	1762	1762
q20	2293	1944	1938	1938
q21	6516	6182	6167	6167
q22	500	418	429	418
Total cold run time: 63916 ms
Total hot run time: 60739 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit e1b0c38dc6f3b8ad7a6e5e48565f3fd93922c777, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4723	4475	4494	4475
q2	368	145	156	145
q3	1445	1242	1225	1225
q4	1102	908	909	908
q5	3136	3152	3154	3152
q6	245	127	123	123
q7	976	494	492	492
q8	2204	2200	2172	2172
q9	6708	6654	6682	6654
q10	3219	3259	3271	3259
q11	328	204	193	193
q12	356	209	210	209
q13	4594	3830	3776	3776
q14	247	216	217	216
q15	581	524	522	522
q16	436	382	387	382
q17	1009	608	515	515
q18	7092	7007	7022	7007
q19	1528	1412	1371	1371
q20	523	328	300	300
q21	3061	2632	2649	2632
q22	348	280	283	280
Total cold run time: 44229 ms
Total hot run time: 40008 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4423	4390	4408	4390
q2	267	165	173	165
q3	3542	3515	3505	3505
q4	2385	2365	2362	2362
q5	5730	5750	5723	5723
q6	239	119	123	119
q7	2371	1894	1858	1858
q8	3513	3514	3523	3514
q9	8982	8929	8950	8929
q10	3912	4005	4027	4005
q11	501	396	376	376
q12	759	599	613	599
q13	4279	3604	3567	3567
q14	287	253	252	252
q15	570	516	522	516
q16	508	460	462	460
q17	1868	1838	1860	1838
q18	8674	8275	8304	8275
q19	1756	1733	1742	1733
q20	2262	1933	1949	1933
q21	6556	6199	6196	6196
q22	498	417	430	417
Total cold run time: 63882 ms
Total hot run time: 60732 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit e1b0c38dc6f3b8ad7a6e5e48565f3fd93922c777, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4768	4481	4489	4481
q2	361	154	157	154
q3	1466	1240	1249	1240
q4	1123	907	912	907
q5	3139	3139	3138	3138
q6	248	127	126	126
q7	986	496	481	481
q8	2189	2223	2190	2190
q9	6674	6685	6675	6675
q10	3233	3288	3261	3261
q11	326	205	202	202
q12	352	205	210	205
q13	4570	3818	3816	3816
q14	239	207	219	207
q15	557	523	527	523
q16	446	387	382	382
q17	1026	638	618	618
q18	7218	7020	6970	6970
q19	1527	1474	1348	1348
q20	549	322	313	313
q21	3071	2609	2711	2609
q22	354	289	281	281
Total cold run time: 44422 ms
Total hot run time: 40127 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4413	4396	4409	4396
q2	269	167	168	167
q3	3537	3521	3516	3516
q4	2381	2369	2372	2369
q5	5736	5716	5721	5716
q6	239	123	123	123
q7	2374	1858	1887	1858
q8	3517	3522	3522	3522
q9	9043	8967	8972	8967
q10	3917	3970	4009	3970
q11	508	383	380	380
q12	752	616	609	609
q13	4280	3567	3541	3541
q14	281	260	259	259
q15	580	523	524	523
q16	530	446	471	446
q17	1883	1850	1868	1850
q18	8645	8215	8244	8215
q19	1738	1760	1746	1746
q20	2249	1955	1954	1954
q21	6527	6167	6137	6137
q22	501	426	417	417
Total cold run time: 63900 ms
Total hot run time: 60681 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit e1b0c38dc6f3b8ad7a6e5e48565f3fd93922c777, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4721	4465	4457	4457
q2	370	145	159	145
q3	1459	1245	1198	1198
q4	1107	938	934	934
q5	3134	3158	3178	3158
q6	244	129	128	128
q7	1012	486	483	483
q8	2194	2209	2174	2174
q9	6725	6695	6662	6662
q10	3190	3254	3274	3254
q11	325	197	200	197
q12	352	212	213	212
q13	4560	3814	3772	3772
q14	236	210	219	210
q15	571	530	524	524
q16	435	397	391	391
q17	1021	669	594	594
q18	7263	7001	7035	7001
q19	1541	1440	1383	1383
q20	534	299	307	299
q21	3068	2630	2668	2630
q22	352	280	281	280
Total cold run time: 44414 ms
Total hot run time: 40086 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4385	4376	4425	4376
q2	274	165	170	165
q3	3542	3536	3524	3524
q4	2388	2380	2366	2366
q5	5731	5757	5720	5720
q6	239	118	124	118
q7	2385	1864	1858	1858
q8	3518	3517	3531	3517
q9	9060	8971	9007	8971
q10	3926	3998	3976	3976
q11	507	388	382	382
q12	771	614	641	614
q13	4300	3572	3572	3572
q14	288	260	250	250
q15	571	525	526	525
q16	512	451	458	451
q17	1890	1863	1845	1845
q18	8570	8369	8297	8297
q19	1741	1765	1744	1744
q20	2271	1930	1940	1930
q21	6520	6184	6170	6170
q22	502	418	414	414
Total cold run time: 63891 ms
Total hot run time: 60785 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit e1b0c38dc6f3b8ad7a6e5e48565f3fd93922c777, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4731	4468	4459	4459
q2	361	153	158	153
q3	1462	1243	1240	1240
q4	1114	869	931	869
q5	3137	3148	3147	3147
q6	241	132	128	128
q7	992	481	497	481
q8	2213	2202	2186	2186
q9	6694	6643	6636	6636
q10	3227	3239	3253	3239
q11	330	201	209	201
q12	356	220	208	208
q13	4535	3823	3795	3795
q14	245	211	213	211
q15	574	530	527	527
q16	448	397	383	383
q17	1002	640	568	568
q18	7177	6870	7038	6870
q19	1515	1439	1370	1370
q20	529	309	319	309
q21	3075	2631	2682	2631
q22	350	285	291	285
Total cold run time: 44308 ms
Total hot run time: 39896 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4450	4438	4468	4438
q2	268	168	175	168
q3	3541	3533	3538	3533
q4	2384	2377	2370	2370
q5	5743	5721	5753	5721
q6	242	120	124	120
q7	2377	1880	1908	1880
q8	3531	3502	3506	3502
q9	9059	8970	8969	8969
q10	3906	3970	3993	3970
q11	492	394	405	394
q12	769	604	597	597
q13	4291	3567	3528	3528
q14	283	260	265	260
q15	574	510	522	510
q16	523	468	488	468
q17	1870	1847	1854	1847
q18	8662	8181	8287	8181
q19	1722	1759	1758	1758
q20	2268	1965	1947	1947
q21	6516	6186	6168	6168
q22	496	427	419	419
Total cold run time: 63967 ms
Total hot run time: 60748 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit e1b0c38dc6f3b8ad7a6e5e48565f3fd93922c777, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4732	4470	4537	4470
q2	362	152	159	152
q3	1463	1263	1238	1238
q4	1105	877	889	877
q5	3162	3132	3140	3132
q6	248	127	129	127
q7	1005	494	487	487
q8	2210	2199	2167	2167
q9	6661	6667	6662	6662
q10	3206	3262	3266	3262
q11	327	196	198	196
q12	351	210	202	202
q13	4566	3816	3809	3809
q14	240	216	214	214
q15	564	532	531	531
q16	441	388	399	388
q17	1016	660	584	584
q18	7184	6900	6938	6900
q19	1540	1447	1333	1333
q20	558	298	295	295
q21	3051	2627	2718	2627
q22	347	283	281	281
Total cold run time: 44339 ms
Total hot run time: 39934 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4430	4412	4441	4412
q2	269	162	172	162
q3	3544	3523	3514	3514
q4	2383	2360	2361	2360
q5	5733	5709	5714	5709
q6	241	121	122	121
q7	2383	1882	1870	1870
q8	3508	3515	3503	3503
q9	8975	8961	8913	8913
q10	3912	4001	3987	3987
q11	505	376	381	376
q12	774	591	624	591
q13	4278	3585	3581	3581
q14	289	261	257	257
q15	571	535	522	522
q16	539	445	445	445
q17	1886	1871	1837	1837
q18	8854	8283	8250	8250
q19	1737	1757	1751	1751
q20	2266	1954	1925	1925
q21	6510	6154	6148	6148
q22	493	420	419	419
Total cold run time: 64080 ms
Total hot run time: 60653 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit e1b0c38dc6f3b8ad7a6e5e48565f3fd93922c777, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4755	4505	4445	4445
q2	365	148	161	148
q3	1461	1251	1209	1209
q4	1115	882	918	882
q5	3131	3155	3153	3153
q6	247	129	128	128
q7	968	479	486	479
q8	2258	2223	2190	2190
q9	6675	6703	6698	6698
q10	3222	3266	3289	3266
q11	334	199	212	199
q12	353	202	206	202
q13	4551	3829	3820	3820
q14	238	210	209	209
q15	563	527	516	516
q16	435	389	389	389
q17	1007	614	523	523
q18	7203	7003	6938	6938
q19	1522	1431	1399	1399
q20	519	282	310	282
q21	3079	2711	2670	2670
q22	352	274	286	274
Total cold run time: 44353 ms
Total hot run time: 40019 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4444	4420	4387	4387
q2	269	161	172	161
q3	3541	3526	3507	3507
q4	2386	2381	2368	2368
q5	5741	5740	5725	5725
q6	240	120	123	120
q7	2377	1861	1865	1861
q8	3554	3549	3525	3525
q9	9034	9033	8982	8982
q10	3911	3992	3980	3980
q11	499	388	386	386
q12	762	598	597	597
q13	4269	3537	3535	3535
q14	283	255	258	255
q15	566	517	523	517
q16	514	451	458	451
q17	1892	1850	1853	1850
q18	8738	8282	8239	8239
q19	1775	1733	1748	1733
q20	2249	1948	1950	1948
q21	6533	6198	6176	6176
q22	509	417	427	417
Total cold run time: 64086 ms
Total hot run time: 60720 ms

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 17, 2023
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

morningman
morningman approved these changes Dec 18, 2023
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit ca6cb04 into apache:master Dec 18, 2023
hello-stephen pushed a commit to hello-stephen/doris that referenced this pull request Dec 28, 2023
@sollhui
[fix](routine_load) change librdkafka version from 2.0.2 to 1.9.2 to …
96a4b13 
…avoid memory leak (apache#28528)
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
@sollhui @HappenLee
[fix](routine_load) change librdkafka version from 2.0.2 to 1.9.2 to …
0f09b01 
…avoid memory leak (apache#28528)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morningman morningman morningman approved these changes

@yiguolei yiguolei yiguolei approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.4 reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sollhui @doris-robot @morningman @yiguolei