Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: add LLM based test generation tool #59626

Merged
merged 1 commit into from
Feb 24, 2025
Merged

tests: add LLM based test generation tool #59626

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 83 additions & 18 deletions DEPS.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -489,39 +489,39 @@ def go_deps():
name = "com_github_azure_azure_sdk_for_go_sdk_azcore",
build_file_proto_mode = "disable_global",
importpath = "github.com/Azure/azure-sdk-for-go/sdk/azcore",
sha256 = "0476f5e090fa602f71a5a3f0c9f4578bbd4a3ee3d5c2c98d07c3110a988cc9c0",
strip_prefix = "github.com/Azure/azure-sdk-for-go/sdk/azcore@v1.12.0",
sha256 = "bf0fca85626fc28c5b3b3c06339a42d2cd78637a1401d3db9a83674ced9aa0cb",
strip_prefix = "github.com/Azure/azure-sdk-for-go/sdk/azcore@v1.14.0",
urls = [
"http://bazel-cache.pingcap.net:8080/gomod/github.com/Azure/azure-sdk-for-go/sdk/azcore/com_github_azure_azure_sdk_for_go_sdk_azcore-v1.12.0.zip",
"http://ats.apps.svc/gomod/github.com/Azure/azure-sdk-for-go/sdk/azcore/com_github_azure_azure_sdk_for_go_sdk_azcore-v1.12.0.zip",
"https://cache.hawkingrei.com/gomod/github.com/Azure/azure-sdk-for-go/sdk/azcore/com_github_azure_azure_sdk_for_go_sdk_azcore-v1.12.0.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/Azure/azure-sdk-for-go/sdk/azcore/com_github_azure_azure_sdk_for_go_sdk_azcore-v1.12.0.zip",
"http://bazel-cache.pingcap.net:8080/gomod/github.com/Azure/azure-sdk-for-go/sdk/azcore/com_github_azure_azure_sdk_for_go_sdk_azcore-v1.14.0.zip",
"http://ats.apps.svc/gomod/github.com/Azure/azure-sdk-for-go/sdk/azcore/com_github_azure_azure_sdk_for_go_sdk_azcore-v1.14.0.zip",
"https://cache.hawkingrei.com/gomod/github.com/Azure/azure-sdk-for-go/sdk/azcore/com_github_azure_azure_sdk_for_go_sdk_azcore-v1.14.0.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/Azure/azure-sdk-for-go/sdk/azcore/com_github_azure_azure_sdk_for_go_sdk_azcore-v1.14.0.zip",
],
)
go_repository(
name = "com_github_azure_azure_sdk_for_go_sdk_azidentity",
build_file_proto_mode = "disable_global",
importpath = "github.com/Azure/azure-sdk-for-go/sdk/azidentity",
sha256 = "c1975389670cde19b40c81be4cc96f36768001eb785d62bce133f69a84e0d3e9",
strip_prefix = "github.com/Azure/azure-sdk-for-go/sdk/azidentity@v1.6.0",
sha256 = "11b5939e70cf765a9753155023dd3e3ea42cc40a133307336abc0f8a4e3af404",
strip_prefix = "github.com/Azure/azure-sdk-for-go/sdk/azidentity@v1.7.0",
urls = [
"http://bazel-cache.pingcap.net:8080/gomod/github.com/Azure/azure-sdk-for-go/sdk/azidentity/com_github_azure_azure_sdk_for_go_sdk_azidentity-v1.6.0.zip",
"http://ats.apps.svc/gomod/github.com/Azure/azure-sdk-for-go/sdk/azidentity/com_github_azure_azure_sdk_for_go_sdk_azidentity-v1.6.0.zip",
"https://cache.hawkingrei.com/gomod/github.com/Azure/azure-sdk-for-go/sdk/azidentity/com_github_azure_azure_sdk_for_go_sdk_azidentity-v1.6.0.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/Azure/azure-sdk-for-go/sdk/azidentity/com_github_azure_azure_sdk_for_go_sdk_azidentity-v1.6.0.zip",
"http://bazel-cache.pingcap.net:8080/gomod/github.com/Azure/azure-sdk-for-go/sdk/azidentity/com_github_azure_azure_sdk_for_go_sdk_azidentity-v1.7.0.zip",
"http://ats.apps.svc/gomod/github.com/Azure/azure-sdk-for-go/sdk/azidentity/com_github_azure_azure_sdk_for_go_sdk_azidentity-v1.7.0.zip",
"https://cache.hawkingrei.com/gomod/github.com/Azure/azure-sdk-for-go/sdk/azidentity/com_github_azure_azure_sdk_for_go_sdk_azidentity-v1.7.0.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/Azure/azure-sdk-for-go/sdk/azidentity/com_github_azure_azure_sdk_for_go_sdk_azidentity-v1.7.0.zip",
],
)
go_repository(
name = "com_github_azure_azure_sdk_for_go_sdk_internal",
build_file_proto_mode = "disable_global",
importpath = "github.com/Azure/azure-sdk-for-go/sdk/internal",
sha256 = "dadfa8174b2e526389f17d57ec03bf5e22a40713723500af17f0ef10649849bb",
strip_prefix = "github.com/Azure/azure-sdk-for-go/sdk/internal@v1.9.0",
sha256 = "0a9b4fe658e348b9a6535b1ae0698fa5cf5ba9ba3710530edbdc763eeffe2d24",
strip_prefix = "github.com/Azure/azure-sdk-for-go/sdk/internal@v1.10.0",
urls = [
"http://bazel-cache.pingcap.net:8080/gomod/github.com/Azure/azure-sdk-for-go/sdk/internal/com_github_azure_azure_sdk_for_go_sdk_internal-v1.9.0.zip",
"http://ats.apps.svc/gomod/github.com/Azure/azure-sdk-for-go/sdk/internal/com_github_azure_azure_sdk_for_go_sdk_internal-v1.9.0.zip",
"https://cache.hawkingrei.com/gomod/github.com/Azure/azure-sdk-for-go/sdk/internal/com_github_azure_azure_sdk_for_go_sdk_internal-v1.9.0.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/Azure/azure-sdk-for-go/sdk/internal/com_github_azure_azure_sdk_for_go_sdk_internal-v1.9.0.zip",
"http://bazel-cache.pingcap.net:8080/gomod/github.com/Azure/azure-sdk-for-go/sdk/internal/com_github_azure_azure_sdk_for_go_sdk_internal-v1.10.0.zip",
"http://ats.apps.svc/gomod/github.com/Azure/azure-sdk-for-go/sdk/internal/com_github_azure_azure_sdk_for_go_sdk_internal-v1.10.0.zip",
"https://cache.hawkingrei.com/gomod/github.com/Azure/azure-sdk-for-go/sdk/internal/com_github_azure_azure_sdk_for_go_sdk_internal-v1.10.0.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/Azure/azure-sdk-for-go/sdk/internal/com_github_azure_azure_sdk_for_go_sdk_internal-v1.10.0.zip",
],
)
go_repository(
Expand Down Expand Up @@ -5564,6 +5564,19 @@ def go_deps():
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/onsi/gomega/com_github_onsi_gomega-v1.29.0.zip",
],
)
go_repository(
name = "com_github_openai_openai_go",
build_file_proto_mode = "disable_global",
importpath = "github.com/openai/openai-go",
sha256 = "a44472c58973413a5b2ca7880d028e19cc98e7a99405693f679f464ac4c570c3",
strip_prefix = "github.com/openai/[email protected]",
urls = [
"http://bazel-cache.pingcap.net:8080/gomod/github.com/openai/openai-go/com_github_openai_openai_go-v0.1.0-alpha.59.zip",
"http://ats.apps.svc/gomod/github.com/openai/openai-go/com_github_openai_openai_go-v0.1.0-alpha.59.zip",
"https://cache.hawkingrei.com/gomod/github.com/openai/openai-go/com_github_openai_openai_go-v0.1.0-alpha.59.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/openai/openai-go/com_github_openai_openai_go-v0.1.0-alpha.59.zip",
],
)
go_repository(
name = "com_github_opencontainers_go_digest",
build_file_proto_mode = "disable_global",
Expand Down Expand Up @@ -7046,6 +7059,58 @@ def go_deps():
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/tiancaiamao/gp/com_github_tiancaiamao_gp-v0.0.0-20221230034425-4025bc8a4d4a.zip",
],
)
go_repository(
name = "com_github_tidwall_gjson",
build_file_proto_mode = "disable_global",
importpath = "github.com/tidwall/gjson",
sha256 = "e6d35da84683bc91382a31214d4c3f9a852eec040ca866a1c620a8c0d99f9676",
strip_prefix = "github.com/tidwall/[email protected]",
urls = [
"http://bazel-cache.pingcap.net:8080/gomod/github.com/tidwall/gjson/com_github_tidwall_gjson-v1.14.4.zip",
"http://ats.apps.svc/gomod/github.com/tidwall/gjson/com_github_tidwall_gjson-v1.14.4.zip",
"https://cache.hawkingrei.com/gomod/github.com/tidwall/gjson/com_github_tidwall_gjson-v1.14.4.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/tidwall/gjson/com_github_tidwall_gjson-v1.14.4.zip",
],
)
go_repository(
name = "com_github_tidwall_match",
build_file_proto_mode = "disable_global",
importpath = "github.com/tidwall/match",
sha256 = "2ba41f7f27330d49e0e432cbf96bf90720a33e4a97be58fe53f63a7e66f04d37",
strip_prefix = "github.com/tidwall/[email protected]",
urls = [
"http://bazel-cache.pingcap.net:8080/gomod/github.com/tidwall/match/com_github_tidwall_match-v1.1.1.zip",
"http://ats.apps.svc/gomod/github.com/tidwall/match/com_github_tidwall_match-v1.1.1.zip",
"https://cache.hawkingrei.com/gomod/github.com/tidwall/match/com_github_tidwall_match-v1.1.1.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/tidwall/match/com_github_tidwall_match-v1.1.1.zip",
],
)
go_repository(
name = "com_github_tidwall_pretty",
build_file_proto_mode = "disable_global",
importpath = "github.com/tidwall/pretty",
sha256 = "9327579bf9e610de753eeb7c35c8cf1e3f56e6e5623cbf65e47bc8076023d1ab",
strip_prefix = "github.com/tidwall/[email protected]",
urls = [
"http://bazel-cache.pingcap.net:8080/gomod/github.com/tidwall/pretty/com_github_tidwall_pretty-v1.2.1.zip",
"http://ats.apps.svc/gomod/github.com/tidwall/pretty/com_github_tidwall_pretty-v1.2.1.zip",
"https://cache.hawkingrei.com/gomod/github.com/tidwall/pretty/com_github_tidwall_pretty-v1.2.1.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/tidwall/pretty/com_github_tidwall_pretty-v1.2.1.zip",
],
)
go_repository(
name = "com_github_tidwall_sjson",
build_file_proto_mode = "disable_global",
importpath = "github.com/tidwall/sjson",
sha256 = "fc40fda87c7de55362f6386e069d674772f25c9d76411ee823acb9ac78b20565",
strip_prefix = "github.com/tidwall/[email protected]",
urls = [
"http://bazel-cache.pingcap.net:8080/gomod/github.com/tidwall/sjson/com_github_tidwall_sjson-v1.2.5.zip",
"http://ats.apps.svc/gomod/github.com/tidwall/sjson/com_github_tidwall_sjson-v1.2.5.zip",
"https://cache.hawkingrei.com/gomod/github.com/tidwall/sjson/com_github_tidwall_sjson-v1.2.5.zip",
"https://storage.googleapis.com/pingcapmirror/gomod/github.com/tidwall/sjson/com_github_tidwall_sjson-v1.2.5.zip",
],
)
go_repository(
name = "com_github_tikv_client_go_v2",
build_file_proto_mode = "disable_global",
Expand Down
11 changes: 8 additions & 3 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ go 1.23.6
require (
cloud.google.com/go/kms v1.15.7
cloud.google.com/go/storage v1.38.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.12.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.0.0
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c
github.com/DATA-DOG/go-sqlmock v1.5.0
Expand Down Expand Up @@ -79,6 +79,7 @@ require (
github.com/ngaut/pools v0.0.0-20180318154953-b7bc8c42aac7
github.com/ngaut/sync2 v0.0.0-20141008032647-7a24ed77b2ef
github.com/nishanths/predeclared v0.2.2
github.com/openai/openai-go v0.1.0-alpha.59
github.com/opentracing/basictracer-go v1.0.0
github.com/opentracing/opentracing-go v1.2.0
github.com/otiai10/copy v1.14.0
Expand Down Expand Up @@ -171,6 +172,10 @@ require (
github.com/pierrec/lz4/v4 v4.1.15 // indirect
github.com/qri-io/jsonpointer v0.1.1 // indirect
github.com/segmentio/fasthash v1.0.3 // indirect
github.com/tidwall/gjson v1.14.4 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/zeebo/xxh3 v1.0.2 // indirect
)

Expand All @@ -179,7 +184,7 @@ require (
cloud.google.com/go/compute/metadata v0.3.0 // indirect
cloud.google.com/go/iam v1.1.6 // indirect
cloud.google.com/go/pubsub v1.36.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.9.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 // indirect
github.com/DataDog/zstd v1.5.5 // indirect
Expand Down
24 changes: 18 additions & 6 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,12 @@ filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
gioui.org v0.0.0-20210308172011-57750fc8a0a6/go.mod h1:RSH6KIUZ0p2xy5zHDxgAM4zumjgTw83q2ge/PI+yyw8=
git.sr.ht/~sbinet/gg v0.3.1/go.mod h1:KGYtlADtqsqANL9ueOFkWymvzUvLMQllU5Ixo+8v3pc=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.12.0 h1:1nGuui+4POelzDwI7RG56yfQJHCnKvwfMoU7VsEp+Zg=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.12.0/go.mod h1:99EvauvlcJ1U06amZiksfYz/3aFGyIhWGHVyiZXtBAI=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0 h1:U2rTu3Ef+7w9FHKIAXM6ZyqF3UOWJZ12zIm8zECAFfg=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0/go.mod h1:9kIvujWAA58nmPmWB1m23fyWic1kYZMxD9CxaWn4Qpg=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.9.0 h1:H+U3Gk9zY56G3u872L82bk4thcsy2Gghb9ExT4Zvm1o=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.9.0/go.mod h1:mgrmMSgaLp9hmax62XQTd0N4aAqSE5E0DulSpVYK7vc=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0 h1:nyQWyZvwGTvunIMxi1Y9uXkcyr+I7TeNrr/foo4Kpk8=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0/go.mod h1:l38EPgmsp71HHLq9j7De57JcKOWPyhrsW1Awm1JS6K0=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0 h1:tfLQ34V6F7tVSwoTf/4lH5sE0o6eCJuNDTmH09nDpbc=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.7.0/go.mod h1:9kIvujWAA58nmPmWB1m23fyWic1kYZMxD9CxaWn4Qpg=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 h1:ywEEhmNahHBihViHepv3xPBn1663uRv2t2q/ESv9seY=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0/go.mod h1:iZDifYGJTIgIIkYRNWPENUnqx6bJ2xnSDFI2tjwZNuY=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.0.0 h1:u/LLAOFgsMv7HmNL4Qufg58y+qElGOt5qv0z1mURkRY=
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.0.0/go.mod h1:2e8rMJtl2+2j+HXbTBwnyGpm5Nou7KhvSfxOq8JpTag=
github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 h1:mFRzDkZVAjdal+s7s0MwaRv9igoPqLRdzOLzw/8Xvq8=
Expand Down Expand Up @@ -639,6 +639,8 @@ github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=
github.com/onsi/gomega v1.29.0 h1:KIA/t2t5UBzoirT4H9tsML45GEbo3ouUnBHsCfD2tVg=
github.com/onsi/gomega v1.29.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ=
github.com/openai/openai-go v0.1.0-alpha.59 h1:T3IYwKSCezfIlL9Oi+CGvU03fq0RoH33775S78Ti48Y=
github.com/openai/openai-go v0.1.0-alpha.59/go.mod h1:3SdE6BffOX9HPEQv8IL/fi3LYZ5TUpRYaqGQZbyk11A=
github.com/opentracing/basictracer-go v1.0.0 h1:YyUAhaEfjoWXclZVJ9sGoNct7j4TVk7lZWlQw5UXuoo=
github.com/opentracing/basictracer-go v1.0.0/go.mod h1:QfBfYuafItcjQuMwinw9GhYKwFXS9KnPs5lxoYwgW74=
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
Expand Down Expand Up @@ -832,6 +834,16 @@ github.com/tiancaiamao/appdash v0.0.0-20181126055449-889f96f722a2 h1:mbAskLJ0oJf
github.com/tiancaiamao/appdash v0.0.0-20181126055449-889f96f722a2/go.mod h1:2PfKggNGDuadAa0LElHrByyrz4JPZ9fFx6Gs7nx7ZZU=
github.com/tiancaiamao/gp v0.0.0-20221230034425-4025bc8a4d4a h1:J/YdBZ46WKpXsxsW93SG+q0F8KI+yFrcIDT4c/RNoc4=
github.com/tiancaiamao/gp v0.0.0-20221230034425-4025bc8a4d4a/go.mod h1:h4xBhSNtOeEosLJ4P7JyKXX7Cabg7AVkWCK5gV2vOrM=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.14.4 h1:uo0p8EbA09J7RQaflQ1aBRffTR7xedD2bcIVSYxLnkM=
github.com/tidwall/gjson v1.14.4/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/tikv/client-go/v2 v2.0.8-0.20250214064015-a805ea84f300 h1:imV8hMAanK2keUM3sTnVgZ3qEyjB1RJ/tU0vrz68vAM=
github.com/tikv/client-go/v2 v2.0.8-0.20250214064015-a805ea84f300/go.mod h1:C6LbDj8IjVyDsZlXAcLA57sdDRIQVrHXkgALHx7hk5g=
github.com/tikv/pd/client v0.0.0-20250213082949-e8930327be42 h1:hvB0tf4u5ZU1vAew3MmbyxdYuogDLjIZYzSFt1ACFqo=
Expand Down
1 change: 1 addition & 0 deletions pkg/server/tests/commontest/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ go_test(
"main_test.go",
"tidb_test.go",
],
data = glob(["testdata/**"]),
flaky = True,
shard_count = 50,
deps = [
Expand Down
35 changes: 35 additions & 0 deletions tests/llmtest/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library", "go_test")

go_library(
name = "llmtest_lib",
srcs = ["main.go"],
importpath = "github.com/pingcap/tidb/tests/llmtest",
visibility = ["//visibility:private"],
deps = [
"//tests/llmtest/generator",
"//tests/llmtest/logger",
"//tests/llmtest/testcase",
"@com_github_go_sql_driver_mysql//:mysql",
"@com_github_spf13_cobra//:cobra",
"@org_uber_go_zap//:zap",
],
)

go_binary(
name = "llmtest",
embed = [":llmtest_lib"],
visibility = ["//visibility:public"],
)

go_test(
name = "llmtest_test",
timeout = "short",
srcs = ["main_test.go"],
data = glob(["testdata/**"]),
flaky = True,
deps = [
"//tests/llmtest/generator",
"//tests/llmtest/testcase",
"@com_github_stretchr_testify//require",
],
)
48 changes: 48 additions & 0 deletions tests/llmtest/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# LLMTest

It's a tool to help TiDB developers to generate test cases through LLM for any features, and run the test cases in TiDB.

## Generate

The `LLMTest` uses different "prompt generator" to generate the tests for different features. For example, the `generator/expression.go` includes the expression prompt generator to generate the test cases for expression module.

To generate test cases with a specific prompt generator, you can use the following command:

```bash
./llmtest generate --openai_base_url https://openai.base.url --openai_model deepseek/deepseek-r1 --openai_token XXXXX --parallel 20 --prompt_generator expression --test_count 10
```

Replace the `prompt_generator` with the specific prompt generator you want to use.

## Add a new prompt generator

To add a new prompt generator, you need to implement the `PromptGenerator` interface in the `generator/prompt.go` file. The `PromptGenerator` interface includes the following methods:

```go
// PromptGenerator is the interface for prompt generator.
type PromptGenerator interface {
Name() string
Groups() []string

GeneratePrompt(group string, count int, existCases []*testcase.Case) []openai.ChatCompletionMessageParamUnion
Unmarshal(response string) []testcase.Case
}
```

The `Name` method returns the name of the prompt generator. The `Groups` method returns the sub-classes of the prompt generator. The `GeneratePrompt` method generates the prompt for the test cases. The `Unmarshal` method unmarshals the response from the OpenAI API to the test cases.

The `Groups()` method is used to classify the test cases. For example, the `expression` prompt generator has the following groups:

```go
[]string{"+", "-" ....}
```

## Verify

To verify the generated test cases, you can use the following command:

```bash
./llmtest verify --mysql_dsn "root:123456@tcp(127.0.0.1:3306)/test" --tidb_dsn "root@tcp(127.0.0.1:4000)/test" --prompt_generator expression
```

It'll execute the generated test cases in the TiDB cluster and MySQL to verify the results.
Loading