Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: cherry pick json path index #40313

Open
wants to merge 5 commits into
base: 2.5
Choose a base branch
from
Open

feat: cherry pick json path index #40313

wants to merge 5 commits into from

Conversation

sunby
Copy link
Contributor

@sunby sunby commented Mar 4, 2025 •
edited
Loading

sunby added 3 commits March 4, 2025 11:19
@sunby
enhance: upgrade rust version to 1.83 (milvus-io#39295)
c9fb9fa 
milvus-io#39254

Signed-off-by: sunby <[email protected]>
@sunby
enhance: update tantivy version (milvus-io#39253)
bbc2939 
milvus-io#39254

---------

Signed-off-by: sunby <[email protected]>
@sunby
feat: support json index (milvus-io#36750)
b24914a 
milvus-io#35528

This PR adds json index support for json and dynamic fields. Now you can
only do unary query like 'a["b"] > 1' using this index. We will support
more filter type later.

basic usage:
```
collection.create_index("json_field", {"index_type": "INVERTED",
    "params": {"json_cast_type": DataType.STRING, "json_path":
'json_field["a"]["b"]'}})
```

There are some limits to use this index:
1. If a record does not have the json path you specify, it will be
ignored and there will not be an error.
2. If a value of the json path fails to be cast to the type you specify,
it will be ignored and there will not be an error.
3. A specific json path can have only one json index.
4. If you try to create more than one json indexes for one json field,
sdk(pymilvus<=2.4.7) may return immediately because of internal
implementation. This will be fixed in a later version.

---------

Signed-off-by: sunby <[email protected]>
@sre-ci-robot sre-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines. label Mar 4, 2025
@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sunby
To complete the pull request process, please assign yanliang567 after the PR has been reviewed.
You can assign the PR to them by writing /assign @yanliang567 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mergify mergify bot added the dco-passed DCO check passed. label Mar 4, 2025
@mergify Mergify
Copy link
Contributor

mergify bot commented Mar 4, 2025

@sunby

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

  1. Title Format: The PR title must begin with one of these prefixes:
  • feat: for introducing a new feature.
  • fix: for bug fixes.
  • enhance: for improvements to existing functionality.
  • test: for add tests to existing functionality.
  • doc: for modifying documentation.
  • auto: for the pull request from bot.
  1. Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, fix, enhance, test or doc.

Example:

enhance: improve search performance significantly

Please review and update your PR to comply with these guidelines.

@dosubot dosubot bot added the area/dependency Pull requests that update a dependency file label Mar 4, 2025
@sunby sunby changed the title cherry pick json path index feat: cherry pick json path index Mar 4, 2025
@mergify mergify bot added kind/feature Issues related to feature request from users and removed do-not-merge/invalid-pr-format labels Mar 4, 2025
@mergify Mergify
Copy link
Contributor

mergify bot commented Mar 4, 2025

@sunby cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Mar 4, 2025

@sunby go-sdk check failed, comment rerun go-sdk can trigger the job again.

sunby added 2 commits March 4, 2025 11:59
@sunby
fix: Fall back to a brute-force search if json index type unmatched (m…
614d014 
…ilvus-io#40076)

issue: milvus-io#35528
If the query data type does not match the index type, fall back to a
brute-force search

---------

Signed-off-by: sunby <[email protected]>
@sunby
enhance: make json path index support term filter (milvus-io#40140)
a210165 
issue: milvus-io#35528

---------

Signed-off-by: sunby <[email protected]>
@mergify Mergify
Copy link
Contributor

mergify bot commented Mar 4, 2025

@sunby cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Mar 4, 2025

@sunby go-sdk check failed, comment rerun go-sdk can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Mar 4, 2025

@sunby E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aoiasd aoiasd Awaiting requested review from aoiasd

@bigsheeper bigsheeper Awaiting requested review from bigsheeper

Assignees
No one assigned
Labels
area/compilation area/dependency Pull requests that update a dependency file area/test dco-passed DCO check passed. kind/feature Issues related to feature request from users sig/testing size/XXL Denotes a PR that changes 1000+ lines.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sunby @sre-ci-robot