Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add includeFields parameter to the method extractFieldNamesTypes #376

Merged
merged 3 commits into from
Aug 2, 2024

Conversation

gaobinlong
Copy link
Collaborator

@gaobinlong gaobinlong commented Jul 30, 2024

Description

This is a follow-up PR of #348, in that PR, we moved and refactored the method extractFieldNamesTypes, but the new method has some impact on the PPLTool, in order to remove the impact, this PR add a new parameter includeFields to that method, when it's set to false, the subfields under a text type field will not be resolved and written to the result.

For example, this is a text type field with subfields:

"c": {
      "type": "text",
      "fields": {
        "d": {
          "type":"keyword"
        }
      }
    }

, if includeFields is true, the flattened mapping result contains:

"c": "text",
"c.d": "keyword"

, but if includeFields is false, the flattened mapping result only contains:

"c": "text",

.

In addition, for object field, we do not extract the top level field name and type into the final result since it's useless for both PPLTool and CreateAnomalyDetectorTool actually.
For example, this is an object field with setting type explicitly:

"a": {
      "type": "object",
      "properties": {
        "b": {
          "type":"keyword"
        }
      }
    }

, the flattened mapping result contains:

"a.b": "keyword"

, but doesn't contain:

"a": "object"

.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@gaobinlong
Copy link
Collaborator Author

@zane-neo @xinyual please help to review this PR, thanks!

@zane-neo zane-neo merged commit 5a9dbcd into opensearch-project:main Aug 2, 2024
7 of 8 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 2, 2024
* Add includeFields parameter to the method extractFieldNamesTypes

Signed-off-by: gaobinlong <[email protected]>

* Remove empty line

Signed-off-by: gaobinlong <[email protected]>

---------

Signed-off-by: gaobinlong <[email protected]>
(cherry picked from commit 5a9dbcd)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
zane-neo pushed a commit that referenced this pull request Aug 2, 2024
… (#378)

* Add includeFields parameter to the method extractFieldNamesTypes



* Remove empty line



---------


(cherry picked from commit 5a9dbcd)

Signed-off-by: gaobinlong <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 2, 2024
* Add includeFields parameter to the method extractFieldNamesTypes

Signed-off-by: gaobinlong <[email protected]>

* Remove empty line

Signed-off-by: gaobinlong <[email protected]>

---------

Signed-off-by: gaobinlong <[email protected]>
(cherry picked from commit 5a9dbcd)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
zane-neo pushed a commit that referenced this pull request Aug 2, 2024
… (#379)

* Add includeFields parameter to the method extractFieldNamesTypes



* Remove empty line



---------


(cherry picked from commit 5a9dbcd)

Signed-off-by: gaobinlong <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gaobinlong added a commit to gaobinlong/skills that referenced this pull request Sep 3, 2024
* Add CreateAnomalyDetectorTool

Signed-off-by: gaobinlong <[email protected]>

* Optimize some code

Signed-off-by: gaobinlong <[email protected]>

* Fix test failure

Signed-off-by: gaobinlong <[email protected]>

* Optimize exception

Signed-off-by: gaobinlong <[email protected]>

---------

Signed-off-by: gaobinlong <[email protected]>

Fix warning and format issue for CreateAnomalyDetectorTool (opensearch-project#358)

Signed-off-by: gaobinlong <[email protected]>

Add includeFields parameter to the method extractFieldNamesTypes (opensearch-project#376)

* Add includeFields parameter to the method extractFieldNamesTypes

Signed-off-by: gaobinlong <[email protected]>

* Remove empty line

Signed-off-by: gaobinlong <[email protected]>

---------

Signed-off-by: gaobinlong <[email protected]>

Optimize the prompt for create anomaly detector tool

Signed-off-by: gaobinlong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants