feat: new diff mode (#86)

* add: templates with new improved diff formatting - also remove extra whitespace for token efficiency * add: new parse codegen response logic * Merge branch 'main' into feature/new-diff-mode * main: fix: model check for diff mode * add: new apply-changes logic for new diff format * test: test the new diff mode implementation * fix: task-workflow and its corresponding test * wip: new diff mode * feat: new diff mode * remove extraenous import * improve apply-changes reliability * fix: make prompt provide unique search blocks * prompt engineering time * Merge branch 'main' into feature/new-diff-mode * main: chore(release): 1.13.6 [skip ci] * more prompt improvements * add new gpt-4o model version, adjust temperature * feat: use the diff mode specified in the model config * add diff mode and other new properties to model config * feat: add deepseek api support revamp model config * tests: update tests * docs: update Docs * docs: update Acknowledgments * fix: redo-task should also use the configured diff mode unless overwritten * fix: make flexible whitespace matching more robust * test: update redo-task tests * fix: template improvements * chore: template improvements * allow multiple identical changes per file again * more prompt tweaking * more prompt tweaks * try with system prompt * prompt tweaking * feat: a new approach * fix parser * remove extraneous console.log * prompt improvements * we'll get there * improve prompt templates * remove file.diff * add preprocess function * implement more reliable patching * refactor * test: adapt tests to new file structure and apply changes functionality * test: fix cross platform path resolution * missed one * improve change application * fix code modification application * fix code modification application * improving code application * add: integration for the application of changes * test: more assertions
gmickel · Aug 8, 2024 · b20e55b · b20e55b
1 parent 5c5a1d6
commit b20e55b
Show file tree

Hide file tree

Showing 47 changed files with 2,876 additions and 1,378 deletions.
diff --git a/.env.template b/.env.template
@@ -1,4 +1,5 @@
 ANTHROPIC_API_KEY=
 OPENAI_API_KEY=
 GROQ_API_KEY=
+DEEPSEEK_API_KEY=
 MODEL=claude-3-5-sonnet-20240620
diff --git a/README.md b/README.md
@@ -165,14 +165,21 @@ codewhisper list-models
 # CodeWhisper will prompt you to select a model from the list of available models
 codewhisper task
 
-# To enable the new diff-based code modifications, you can use the following command:
-# This will use diff-based code modifications. Please note that this is still experimental and may not work with all models. It is currently only recommended with Claude 3.5 Sonnet.
+# The mode of operation for generating code modifications is set automatically based on the model.
+# You can override this by using the --diff or --no-diff option.
 codewhisper task --diff
+codewhisper task --no-diff
 
 # You can also specify a model directly
 # Claude-3.5 Sonnet
 codewhisper task -m claude-3-5-sonnet-20240620
 
+# GPT-4o
+codewhisper task -m gpt-4o-2024-08-06
+
+# DeepSeek Coder
+codewhisper task -m deepseek-coder
+
 # Or use a local Ollama model (not recommended as it will be slow and inaccurate for comprehensive feature implementation tasks)
 codewhisper task -m ollama:llama3.1:70b --context-window 131072 --max-tokens 8192
 
@@ -184,21 +191,27 @@ codewhisper task --undo
 codewhisper task --redo
 ```
 
-> Note: If you are using CodeWhisper's LLM integration with `codewhisper task`, you will need to set the respective environment variable for the model you want to use (e.g., `export ANTHROPIC_API_KEY=your_api_key` or `export OPENAI_API_KEY=your_api_key` or `export GROQ_API_KEY=your_api_key`).
+> Note: If you are using CodeWhisper's LLM integration with `codewhisper task`, you will need to set the respective environment variable for the model you want to use (e.g., `export ANTHROPIC_API_KEY=your_api_key` or `export OPENAI_API_KEY=your_api_key` or `export GROQ_API_KEY=your_api_key` or `export DEEPSEEK_API_KEY=your_api_key` ).
 
 For more detailed instructions, see the [Installation](#-installation) and [Usage](#-usage) sections.
 
 ### Supported Models and Current Recommendations
 
 While CodeWhisper supports a variety of providers and models, our current recommendations are based on extensive testing and real-world usage. Here's an overview of the current status:
 
-#### Recommended Models
+#### Model Evaluation
+
+This section is still under development. We are actively testing and evaluating models.
+
+| Model             | Provider  | Recommendation | Editing Mode | Plan Quality | Code Quality | Edit Precision | Notes                                                                               |
+| ----------------- | --------- | -------------- | ------------ | ------------ | ------------ | -------------- | ----------------------------------------------------------------------------------- |
+| Claude-3.5-Sonnet | Anthropic | Highest        | Diff         | Excellent    | Excellent    | High           | Generates exceptional quality plans and results                                     |
+| GPT-4o            | OpenAI    | Excellent      | Diff         | Very Good    | Good         | Medium         | Produces high-quality plans and good results, long max output length (16384 tokens) |
+| GPT-4o-mini       | OpenAI    | Strong         | Diff         | Good         | Good         | Medium         | Good quality plans and results, long max output length (16384 tokens)               |
+| GPT-4o-mini       | OpenAI    | Strong         | Whole\*      | Good         | Very Good    | High           | Improved code quality and precision in whole-file edit mode                         |
+| DeepSeek Coder    | DeepSeek  | Good           | Diff         | Good         | Good         | Medium         | Good quality plans and results, long max output length (16384 tokens)               |
 
-| Model             | Provider  | Recommendation | Notes                                                                 |
-| ----------------- | --------- | -------------- | --------------------------------------------------------------------- |
-| Claude-3.5-Sonnet | Anthropic | Highest        | Generates exceptional quality plans and results                       |
-| GPT-4o            | OpenAI    | Excellent      | Produces high-quality plans and good results                          |
-| GPT-4o-mini       | OpenAI    | Strong         | Good quality plans and results, long max output length (16384 tokens) |
+\* Whole-file edit mode is generally more precise but may lead to issues with maximum output token length, potentially limiting the ability to process larger files or multiple files simultaneously. It can also result in incomplete outputs for very large files, with the model resorting to placeholders like "// other functions here" instead of providing full implementations.
 
 #### Experimental Support
 
@@ -410,6 +423,7 @@ We welcome contributions to CodeWhisper! Please read our [CONTRIBUTING.md](CONTR
 - [Commander.js](https://github.com/tj/commander.js/) for CLI support
 - [fast-glob](https://github.com/mrmlnc/fast-glob) for file matching
 - [Inquirer.js](https://github.com/SBoudrias/Inquirer.js/) for interactive prompts
+- [Vercel AI SDK](https://sdk.vercel.ai/docs/introduction) for the great AI SDK
 
 ## 📬 Contact
 

diff --git a/USAGE.md b/USAGE.md
@@ -53,8 +53,8 @@ codewhisper task [options]
 | `-c, --context <paths...>`            | Specify files or directories to include in the task context. Can be file paths, directory paths, or glob patterns. Multiple entries should be space-separated.                                                                                                                                                                                                                |
 | `--github-issue`                      | Use GitHub issue for task input                                                                                                                                                                                                                                                                                                                                               |
 | `--github-issue-filters <filters>`    | Use these filters when fetching issues. Format: comma-separated key:value pairs. Example: labels:p1,assignee:abc Note: see "query parameters" at https://docs.github.com/en/rest/issues/issues?apiVersion=2022-11-28#list-repository-issues--parameters for all options.                                                                                                      |
-| `-df, --diff`                         | Use diff format for file updates instead of full file content (default: false). Please note that this is still experimental and may not work with all models. It is currently only recommended with Claude 3.5 Sonnet.                                                                                                                                                        |
-| `--no-diff`                           | Disable diff mode (default: true)                                                                                                                                                                                                                                                                                                                                             |
+| `-df, --diff`                         | Override the default diff mode for the model.                                                                                                                                                                                                                                                                                                                                 |
+| `--no-diff`                           | Override the default diff mode for the model.                                                                                                                                                                                                                                                                                                                                 |
 | `--plan`                              | Use the planning mode, this generates an intermediate plan, which can be modified. Useful for complex tasks. (default: true)                                                                                                                                                                                                                                                  |
 | `--no-plan`                           | Disable the planning mode. Useful for simple tasks (default: false)                                                                                                                                                                                                                                                                                                           |
 | `-g, --gitignore <path>`              | Path to .gitignore file (default: .gitignore)                                                                                                                                                                                                                                                                                                                                 |

diff --git a/package.json b/package.json
@@ -92,16 +92,15 @@
     "prepare": "lefthook install"
   },
   "dependencies": {
-    "@ai-sdk/anthropic": "0.0.35",
-    "@ai-sdk/openai": "0.0.40",
+    "@ai-sdk/anthropic": "0.0.39",
+    "@ai-sdk/openai": "0.0.44",
     "@anthropic-ai/sdk": "0.25.0",
-    "@inquirer/prompts": "5.3.7",
+    "@inquirer/prompts": "5.3.8",
     "@octokit/rest": "21.0.1",
-    "@types/diff": "5.2.1",
-    "ai": "3.3.0",
+    "@types/uuid": "10.0.0",
+    "ai": "3.3.4",
     "chalk": "5.3.0",
     "commander": "12.1.0",
-    "diff": "5.2.0",
     "dotenv": "16.4.5",
     "fast-glob": "3.3.2",
     "fs-extra": "11.2.0",
@@ -112,12 +111,13 @@
     "inquirer-file-tree-selection-prompt": "2.0.5",
     "isbinaryfile": "5.0.2",
     "micromatch": "4.0.7",
-    "ollama-ai-provider": "0.11.0",
+    "ollama-ai-provider": "0.12.0",
     "ora": "8.0.1",
     "piscina": "4.6.1",
     "simple-git": "3.25.0",
     "strip-comments": "2.0.1",
-    "winston": "3.13.1"
+    "uuid": "10.0.0",
+    "winston": "3.14.1"
   },
   "devDependencies": {
     "@biomejs/biome": "1.8.3",
@@ -135,7 +135,7 @@
     "semantic-release": "24.0.0",
     "tsup": "8.2.4",
     "typescript": "5.5.4",
-    "vite": "5.3.5",
+    "vite": "5.4.0",
     "vitest": "2.0.5"
   },
   "packageManager": "[email protected]",