forked from opensearch-project/skills
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add CreateAnomalyDetectorTool (opensearch-project#348)
* Add CreateAnomalyDetectorTool Signed-off-by: gaobinlong <[email protected]> * Optimize some code Signed-off-by: gaobinlong <[email protected]> * Fix test failure Signed-off-by: gaobinlong <[email protected]> * Optimize exception Signed-off-by: gaobinlong <[email protected]> --------- Signed-off-by: gaobinlong <[email protected]> Fix warning and format issue for CreateAnomalyDetectorTool (opensearch-project#358) Signed-off-by: gaobinlong <[email protected]> Add includeFields parameter to the method extractFieldNamesTypes (opensearch-project#376) * Add includeFields parameter to the method extractFieldNamesTypes Signed-off-by: gaobinlong <[email protected]> * Remove empty line Signed-off-by: gaobinlong <[email protected]> --------- Signed-off-by: gaobinlong <[email protected]> Optimize the prompt for create anomaly detector tool Signed-off-by: gaobinlong <[email protected]>
- Loading branch information
1 parent
9406558
commit 00fd8ce
Showing
9 changed files
with
1,241 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
453 changes: 453 additions & 0 deletions
453
src/main/java/org/opensearch/agent/tools/CreateAnomalyDetectorTool.java
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
51 changes: 51 additions & 0 deletions
51
src/main/java/org/opensearch/agent/tools/utils/ToolHelper.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.agent.tools.utils; | ||
|
|
||
| import java.util.Map; | ||
|
|
||
| public class ToolHelper { | ||
| /** | ||
| * Flatten all the fields in the mappings, insert the field to fieldType mapping to a map | ||
| * @param mappingSource the mappings of an index | ||
| * @param fieldsToType the result containing the field to fieldType mapping | ||
| * @param prefix the parent field path | ||
| * @param includeFields whether include the `fields` in a text type field, for some use case like PPLTool, `fields` in a text type field | ||
| * cannot be included, but for CreateAnomalyDetectorTool, `fields` must be included. | ||
| */ | ||
| public static void extractFieldNamesTypes( | ||
| Map<String, Object> mappingSource, | ||
| Map<String, String> fieldsToType, | ||
| String prefix, | ||
| boolean includeFields | ||
| ) { | ||
| if (prefix.length() > 0) { | ||
| prefix += "."; | ||
| } | ||
|
|
||
| for (Map.Entry<String, Object> entry : mappingSource.entrySet()) { | ||
| String n = entry.getKey(); | ||
| Object v = entry.getValue(); | ||
|
|
||
| if (v instanceof Map) { | ||
| Map<String, Object> vMap = (Map<String, Object>) v; | ||
| if (vMap.containsKey("type")) { | ||
| String fieldType = (String) vMap.getOrDefault("type", ""); | ||
| // no need to extract alias into the result, and for object field, extract the subfields only | ||
| if (!fieldType.equals("alias") && !fieldType.equals("object")) { | ||
| fieldsToType.put(prefix + n, (String) vMap.get("type")); | ||
| } | ||
| } | ||
| if (vMap.containsKey("properties")) { | ||
| extractFieldNamesTypes((Map<String, Object>) vMap.get("properties"), fieldsToType, prefix + n, includeFields); | ||
| } | ||
| if (includeFields && vMap.containsKey("fields")) { | ||
| extractFieldNamesTypes((Map<String, Object>) vMap.get("fields"), fieldsToType, prefix + n, true); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } |
4 changes: 4 additions & 0 deletions
4
src/main/resources/org/opensearch/agent/tools/CreateAnomalyDetectorDefaultPrompt.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "CLAUDE": "Human:\" turn\": Here is an example of the create anomaly detector API: POST _plugins/_anomaly_detection/detectors, {\"time_field\":\"timestamp\",\"indices\":[\"server_log*\"],\"feature_attributes\":[{\"feature_name\":\"test\",\"feature_enabled\":true,\"aggregation_query\":{\"test\":{\"sum\":{\"field\":\"value\"}}}}],\"category_field\":[\"ip\"]}, and here are the mapping info containing all the fields in the index ${indexInfo.indexName}: ${indexInfo.indexMapping}, and the optional aggregation methods are count, avg, min, max and sum. Please give me some suggestion about creating an anomaly detector for the index ${indexInfo.indexName}, you need to give the key information: the top 3 suitable aggregation fields which are numeric types(long, integer, double, float, short etc.) and the suitable aggregation method for each field, you should give at most 3 aggregation fields and corresponding aggregation methods, if there are no numeric type fields, both the aggregation field and method are empty string, and also give at most 1 category field if there exists a keyword type field like ip, address, host, city, country or region, if not exist, the category field is empty. Show me a format of keyed and pipe-delimited list wrapped in a curly bracket just like {category_field=the category field if exists|aggregation_field=comma-delimited list of all the aggregation field names|aggregation_method=comma-delimited list of all the aggregation methods}. \n\nAssistant:\" turn\"", | ||
| "OPENAI": "Here is an example of the create anomaly detector API: POST _plugins/_anomaly_detection/detectors, {\"time_field\":\"timestamp\",\"indices\":[\"server_log*\"],\"feature_attributes\":[{\"feature_name\":\"test\",\"feature_enabled\":true,\"aggregation_query\":{\"test\":{\"sum\":{\"field\":\"value\"}}}}],\"category_field\":[\"ip\"]}, and here are the mapping info containing all the fields in the index ${indexInfo.indexName}: ${indexInfo.indexMapping}, and the optional aggregation methods are count, avg, min, max and sum. Please give me some suggestion about creating an anomaly detector for the index ${indexInfo.indexName}, you need to give the key information: the top 3 suitable aggregation fields which are numeric types(long, integer, double, float, short etc.) and the suitable aggregation method for each field, you should give at most 3 aggregation fields and corresponding aggregation methods, if there are no numeric type fields, both the aggregation field and method are empty string, and also give at most 1 category field if there exists a keyword type field like ip, address, host, city, country or region, if not exist, the category field is empty. Show me a format of keyed and pipe-delimited list wrapped in a curly bracket just like {category_field=the category field if exists|aggregation_field=comma-delimited list of all the aggregation field names|aggregation_method=comma-delimited list of all the aggregation methods}. " | ||
| } |
Oops, something went wrong.