Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: new diff mode #86

Merged
merged 56 commits into from
Aug 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
e3e87c1
add: templates with new improved diff formatting
gmickel Aug 6, 2024
f25db88
add: new parse codegen response logic
gmickel Aug 6, 2024
38bc03f
Merge branch 'main' into feature/new-diff-mode
gmickel Aug 6, 2024
35f4380
add: new apply-changes logic for new diff format
gmickel Aug 6, 2024
33c8c0b
test: test the new diff mode implementation
gmickel Aug 6, 2024
4e60b2b
fix: task-workflow and its corresponding test
gmickel Aug 6, 2024
a00a1bc
wip: new diff mode
gmickel Aug 6, 2024
f8b5345
feat: new diff mode
gmickel Aug 6, 2024
68f7ef4
remove extraenous import
gmickel Aug 6, 2024
8cd9335
improve apply-changes reliability
gmickel Aug 6, 2024
a5d9cc0
fix: make prompt provide unique search blocks
gmickel Aug 6, 2024
f94b99d
prompt engineering time
gmickel Aug 6, 2024
f209c3e
Merge branch 'main' into feature/new-diff-mode
gmickel Aug 6, 2024
097d342
more prompt improvements
gmickel Aug 6, 2024
91c50f1
add new gpt-4o model version, adjust temperature
gmickel Aug 7, 2024
4590871
feat: use the diff mode specified in the model config
gmickel Aug 7, 2024
56d5c6c
add diff mode and other new properties to model config
gmickel Aug 7, 2024
756929a
feat: add deepseek api support
gmickel Aug 7, 2024
9c46e6d
tests: update tests
gmickel Aug 7, 2024
64866ca
docs: update Docs
gmickel Aug 7, 2024
b7bc545
docs: update Acknowledgments
gmickel Aug 7, 2024
a3bb5fd
fix: redo-task should also use the configured diff mode unless overwr…
gmickel Aug 7, 2024
9c4d2bd
fix: make flexible whitespace matching more robust
gmickel Aug 7, 2024
1b8fd4d
test: update redo-task tests
gmickel Aug 7, 2024
846874f
fix: template improvements
gmickel Aug 7, 2024
21a69d1
chore: template improvements
gmickel Aug 7, 2024
26f6bd6
allow multiple identical changes per file again
gmickel Aug 7, 2024
febab4d
more prompt tweaking
gmickel Aug 7, 2024
7b17d70
more prompt tweaks
gmickel Aug 7, 2024
d622b56
try with system prompt
gmickel Aug 7, 2024
6cb438b
prompt tweaking
gmickel Aug 7, 2024
7f6dd89
feat: a new approach
gmickel Aug 7, 2024
35f864c
fix parser
gmickel Aug 7, 2024
254a5d1
remove extraneous console.log
gmickel Aug 7, 2024
5add5e2
prompt improvements
gmickel Aug 7, 2024
9aa81f5
we'll get there
gmickel Aug 7, 2024
8ed3815
improve prompt templates
gmickel Aug 7, 2024
292858b
remove file.diff
gmickel Aug 7, 2024
cbe4126
add preprocess function
gmickel Aug 7, 2024
0b41793
implement more reliable patching
gmickel Aug 7, 2024
e4d6a9a
refactor
gmickel Aug 7, 2024
4755041
test: adapt tests to new file structure and apply changes functionality
gmickel Aug 7, 2024
2f790f8
test: fix cross platform path resolution
gmickel Aug 7, 2024
58729ed
missed one
gmickel Aug 7, 2024
1e60612
improve change application
gmickel Aug 7, 2024
98fe7df
fix code modification application
gmickel Aug 7, 2024
d01b111
fix code modification application
gmickel Aug 7, 2024
65b1772
improving code application
gmickel Aug 7, 2024
43cc6e1
add: integration for the application of changes
gmickel Aug 8, 2024
d659ba9
test: more assertions
gmickel Aug 8, 2024
7b4c7f1
add: new more robust search and replace
gmickel Aug 8, 2024
1386b8b
feat: new search and replace logic
gmickel Aug 8, 2024
f525e8d
chore: update dependencies
gmickel Aug 8, 2024
865fc1d
chore: remove last vestiges of old diff mode
gmickel Aug 8, 2024
ea5a671
fix: test
gmickel Aug 8, 2024
904c3fa
docs: update evaluation table
gmickel Aug 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env.template
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GROQ_API_KEY=
DEEPSEEK_API_KEY=
MODEL=claude-3-5-sonnet-20240620
32 changes: 23 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,14 +165,21 @@ codewhisper list-models
# CodeWhisper will prompt you to select a model from the list of available models
codewhisper task

# To enable the new diff-based code modifications, you can use the following command:
# This will use diff-based code modifications. Please note that this is still experimental and may not work with all models. It is currently only recommended with Claude 3.5 Sonnet.
# The mode of operation for generating code modifications is set automatically based on the model.
# You can override this by using the --diff or --no-diff option.
codewhisper task --diff
codewhisper task --no-diff

# You can also specify a model directly
# Claude-3.5 Sonnet
codewhisper task -m claude-3-5-sonnet-20240620

# GPT-4o
codewhisper task -m gpt-4o-2024-08-06

# DeepSeek Coder
codewhisper task -m deepseek-coder

# Or use a local Ollama model (not recommended as it will be slow and inaccurate for comprehensive feature implementation tasks)
codewhisper task -m ollama:llama3.1:70b --context-window 131072 --max-tokens 8192

Expand All @@ -184,21 +191,27 @@ codewhisper task --undo
codewhisper task --redo
```

> Note: If you are using CodeWhisper's LLM integration with `codewhisper task`, you will need to set the respective environment variable for the model you want to use (e.g., `export ANTHROPIC_API_KEY=your_api_key` or `export OPENAI_API_KEY=your_api_key` or `export GROQ_API_KEY=your_api_key`).
> Note: If you are using CodeWhisper's LLM integration with `codewhisper task`, you will need to set the respective environment variable for the model you want to use (e.g., `export ANTHROPIC_API_KEY=your_api_key` or `export OPENAI_API_KEY=your_api_key` or `export GROQ_API_KEY=your_api_key` or `export DEEPSEEK_API_KEY=your_api_key` ).

For more detailed instructions, see the [Installation](#-installation) and [Usage](#-usage) sections.

### Supported Models and Current Recommendations

While CodeWhisper supports a variety of providers and models, our current recommendations are based on extensive testing and real-world usage. Here's an overview of the current status:

#### Recommended Models
#### Model Evaluation

This section is still under development. We are actively testing and evaluating models.

| Model | Provider | Recommendation | Editing Mode | Plan Quality | Code Quality | Edit Precision | Notes |
| ----------------- | --------- | -------------- | ------------ | ------------ | ------------ | -------------- | ----------------------------------------------------------------------------------- |
| Claude-3.5-Sonnet | Anthropic | Highest | Diff | Excellent | Excellent | High | Generates exceptional quality plans and results |
| GPT-4o | OpenAI | Excellent | Diff | Very Good | Good | Medium | Produces high-quality plans and good results, long max output length (16384 tokens) |
| GPT-4o-mini | OpenAI | Strong | Diff | Good | Good | Medium | Good quality plans and results, long max output length (16384 tokens) |
| GPT-4o-mini | OpenAI | Strong | Whole\* | Good | Very Good | High | Improved code quality and precision in whole-file edit mode |
| DeepSeek Coder | DeepSeek | Good | Diff | Good | Good | Medium | Good quality plans and results, long max output length (16384 tokens) |

| Model | Provider | Recommendation | Notes |
| ----------------- | --------- | -------------- | --------------------------------------------------------------------- |
| Claude-3.5-Sonnet | Anthropic | Highest | Generates exceptional quality plans and results |
| GPT-4o | OpenAI | Excellent | Produces high-quality plans and good results |
| GPT-4o-mini | OpenAI | Strong | Good quality plans and results, long max output length (16384 tokens) |
\* Whole-file edit mode is generally more precise but may lead to issues with maximum output token length, potentially limiting the ability to process larger files or multiple files simultaneously. It can also result in incomplete outputs for very large files, with the model resorting to placeholders like "// other functions here" instead of providing full implementations.

#### Experimental Support

Expand Down Expand Up @@ -410,6 +423,7 @@ We welcome contributions to CodeWhisper! Please read our [CONTRIBUTING.md](CONTR
- [Commander.js](https://github.com/tj/commander.js/) for CLI support
- [fast-glob](https://github.com/mrmlnc/fast-glob) for file matching
- [Inquirer.js](https://github.com/SBoudrias/Inquirer.js/) for interactive prompts
- [Vercel AI SDK](https://sdk.vercel.ai/docs/introduction) for the great AI SDK

## 📬 Contact

Expand Down
4 changes: 2 additions & 2 deletions USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ codewhisper task [options]
| `-c, --context <paths...>` | Specify files or directories to include in the task context. Can be file paths, directory paths, or glob patterns. Multiple entries should be space-separated. |
| `--github-issue` | Use GitHub issue for task input |
| `--github-issue-filters <filters>` | Use these filters when fetching issues. Format: comma-separated key:value pairs. Example: labels:p1,assignee:abc Note: see "query parameters" at https://docs.github.com/en/rest/issues/issues?apiVersion=2022-11-28#list-repository-issues--parameters for all options. |
| `-df, --diff` | Use diff format for file updates instead of full file content (default: false). Please note that this is still experimental and may not work with all models. It is currently only recommended with Claude 3.5 Sonnet. |
| `--no-diff` | Disable diff mode (default: true) |
| `-df, --diff` | Override the default diff mode for the model. |
| `--no-diff` | Override the default diff mode for the model. |
| `--plan` | Use the planning mode, this generates an intermediate plan, which can be modified. Useful for complex tasks. (default: true) |
| `--no-plan` | Disable the planning mode. Useful for simple tasks (default: false) |
| `-g, --gitignore <path>` | Path to .gitignore file (default: .gitignore) |
Expand Down
18 changes: 9 additions & 9 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -92,16 +92,15 @@
"prepare": "lefthook install"
},
"dependencies": {
"@ai-sdk/anthropic": "0.0.35",
"@ai-sdk/openai": "0.0.40",
"@ai-sdk/anthropic": "0.0.39",
"@ai-sdk/openai": "0.0.44",
"@anthropic-ai/sdk": "0.25.0",
"@inquirer/prompts": "5.3.7",
"@inquirer/prompts": "5.3.8",
"@octokit/rest": "21.0.1",
"@types/diff": "5.2.1",
"ai": "3.3.0",
"@types/uuid": "10.0.0",
"ai": "3.3.4",
"chalk": "5.3.0",
"commander": "12.1.0",
"diff": "5.2.0",
"dotenv": "16.4.5",
"fast-glob": "3.3.2",
"fs-extra": "11.2.0",
Expand All @@ -112,12 +111,13 @@
"inquirer-file-tree-selection-prompt": "2.0.5",
"isbinaryfile": "5.0.2",
"micromatch": "4.0.7",
"ollama-ai-provider": "0.11.0",
"ollama-ai-provider": "0.12.0",
"ora": "8.0.1",
"piscina": "4.6.1",
"simple-git": "3.25.0",
"strip-comments": "2.0.1",
"winston": "3.13.1"
"uuid": "10.0.0",
"winston": "3.14.1"
},
"devDependencies": {
"@biomejs/biome": "1.8.3",
Expand All @@ -135,7 +135,7 @@
"semantic-release": "24.0.0",
"tsup": "8.2.4",
"typescript": "5.5.4",
"vite": "5.3.5",
"vite": "5.4.0",
"vitest": "2.0.5"
},
"packageManager": "[email protected]",
Expand Down
Loading