-
Notifications
You must be signed in to change notification settings - Fork 768
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[READY] Improve filename completer #1104
Conversation
9d47ad2
to
bae480a
Compare
Codecov Report
@@ Coverage Diff @@
## master #1104 +/- ##
==========================================
+ Coverage 97.64% 97.67% +0.03%
==========================================
Files 90 90
Lines 7037 7064 +27
==========================================
+ Hits 6871 6900 +29
+ Misses 166 164 -2 |
d870002
to
159ccc1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 7 of 9 files at r1, 1 of 3 files at r3.
Reviewable status: 0 of 2 LGTMs obtained
ycmd/completers/cpp/include_cache.py, line 39 at r1 (raw file):
import logging _logger = logging.getLogger( __name__ )
Logging is not used in this file any more.
ycmd/completers/general/filename_completer.py, line 48 at r1 (raw file):
HEAD_PATH_PATTERN_UNIX = '\.{1,2}|~|\$[^$]+' HEAD_PATH_PATTERN_WINDOWS = '[A-z]+:|\.{1,2}|~|\$[^$]+|%[^%]+%'
Last time I checked, Windows could only use [A-Z]
and exactly once for the drive letter. Even if this is no longer the case, there are characters between Z
and a
.
ycmd/tests/filename_completer_test.py, line 86 at r3 (raw file):
( '//', {}, () ),
This is a valid root path. You can have repeated, consecutive slashes. This is why the tests are failing on Travis.
159ccc1
to
a970a7c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a flaw in the implementation: if the line contained exactly one path separator, the identifier completer was disabled for the rest of the line because the separator was always matched. To work around that, we now fall back to the identifier completer if the filename completer returns nothing.
Reviewed 9 of 9 files at r1, 4 of 4 files at r2, 3 of 3 files at r3, 2 of 2 files at r4.
Reviewable status: 0 of 2 LGTMs obtained
ycmd/completers/cpp/include_cache.py, line 39 at r1 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
Logging is not used in this file any more.
Good catch. Removed.
ycmd/completers/general/filename_completer.py, line 48 at r1 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
Last time I checked, Windows could only use
[A-Z]
and exactly once for the drive letter. Even if this is no longer the case, there are characters betweenZ
anda
.
The drive letter is case-insensitive so we should match [A-Za-z]
but you are right for the rest. Updated.
ycmd/tests/filename_completer_test.py, line 86 at r3 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
This is a valid root path. You can have repeated, consecutive slashes. This is why the tests are failing on Travis.
The current behavior is to not complete such path so I am trying to preserve it. I think it's working now.
a970a7c
to
16213ae
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Simplified the general completer. The code in ShouldUseNow
method is not needed because the identifier, filename, and Ultisnips completers call their own ShouldUseNow
in ComputeCandidates
. In fact, that code caused each completer to call this method twice when returning True
.
Reviewed 2 of 2 files at r5.
Reviewable status: 0 of 2 LGTMs obtained
16213ae
to
a962bd7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 2 LGTMs obtained (and 1 stale)
ycmd/utils.py, line 516 at r6 (raw file):
# Path must be a unicode string to get unicode strings out of listdir.
Instead of commenting this, why not just enforce it by using our utility functions to convert to a unicode string?
ycmd/completers/general/filename_completer.py, line 111 at r6 (raw file):
def DetectPath( self, request_data ):
This function looks rather complex; let's make sure we have good test coverage for it.
a4016f7
to
96452ad
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replaced _GetMacClangVersionList
with ListDirectory
and simplified the macOS flag tests.
Reviewed 1 of 1 files at r6, 1 of 1 files at r7, 1 of 1 files at r8, 2 of 2 files at r9, 1 of 1 files at r10.
Reviewable status: 0 of 2 LGTMs obtained (and 1 stale)
ycmd/utils.py, line 516 at r6 (raw file):
Previously, Valloric (Val Markovic) wrote…
Instead of commenting this, why not just enforce it by using our utility functions to convert to a unicode string?
Done.
ycmd/completers/general/filename_completer.py, line 111 at r6 (raw file):
Previously, Valloric (Val Markovic) wrote…
This function looks rather complex; let's make sure we have good test coverage for it.
I think we do with 31 tests and 15 additional tests for Windows. According to codecov, the function is fully covered.
96452ad
to
3aa8055
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 4 files at r2, 1 of 2 files at r4, 1 of 2 files at r5, 4 of 4 files at r11.
Reviewable status: 0 of 2 LGTMs obtained (and 1 stale)
ycmd/completers/general/filename_completer.py, line 47 at r6 (raw file):
HEAD_PATH_PATTERN_UNIX = '\.{1,2}|~|\$[^$]+' HEAD_PATH_PATTERN_WINDOWS = '[A-Za-z]:|\.{1,2}|~|\$[^$]+|%[^%]+%'
Why is PATH_SEPARATORS_PATTERN
a raw string while HEAD_PATH_PATTERN_*
aren't?
Also, why \$[^$]+
in HEAD_PATH_PATTERN_UNIX
and %[^%]+%
in Windows version? I get that they are meant to match variables, but are we sure we want that?
ycmd/completers/general/filename_completer.py, line 111 at r6 (raw file):
Previously, micbou wrote…
I think we do with 31 tests and 15 additional tests for Windows. According to codecov, the function is fully covered.
The function also returns a tuple, but it doesn't say what do the tuple elements mean. We should have a comment.
ycmd/completers/general/filename_completer.py, line 112 at r11 (raw file):
def DetectPath( self, request_data ): current_line = request_data[ 'prefix' ]
Where does 'prefix'
come from? Should we document it in our ycmd API, since it isn't documented?
ycmd/completers/general/filename_completer.py, line 132 at r11 (raw file):
path = os.path.join( working_dir, path ) if os.path.exists( path ): return path, last_match_start + 2
Why last_match_start + 2
?
ycmd/completers/general/filename_completer.py, line 148 at r11 (raw file):
# TODO: completion on a single "/" or "\" is not really desirable in # languages where such characters are part of special constructs like # comments in C/C++ or closing tags in HTML. This behavior could be improved
At least for C comments this isn't a problem, because you'll probably type //
fast enough to only see a split second flicker. It's not pretty, but oh well. HTML tags are different.
ycmd/tests/filename_completer_test.py, line 86 at r3 (raw file):
Previously, micbou wrote…
The current behavior is to not complete such path so I am trying to preserve it. I think it's working now.
Is there a reason why we wouldn't want to complete such path? I have not yet looked into the actual code, but it could be hard to handle ///home/bstaletic///Downloads
.
☔ The latest upstream changes (presumably #1103) made this pull request unmergeable. Please resolve the merge conflicts. |
826ff8e
to
37a4f53
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed the conflicts and added some comments to the SearchPath
method (renamed from DetectPath
).
Reviewed 4 of 4 files at r11, 6 of 6 files at r12.
Reviewable status: 0 of 2 LGTMs obtained (and 1 stale)
ycmd/completers/general/filename_completer.py, line 47 at r6 (raw file):
Why is PATH_SEPARATORS_PATTERN a raw string while HEAD_PATH_PATTERN_* aren't?
It was to escape the slashes in PATH_SEPARATORS_PATTERN
but it's not needed anymore since the slashes have been replaced with a placeholder.
Also, why
$[^$ ]+ in HEAD_PATH_PATTERN_UNIX and %[^%]+% in Windows version? I get that they are meant to match variables, but are we sure we want that?
We want to match a path that start with an environment variable i.e. $var
on all platforms as well as %var%
on Windows. It's something that we already support so we should still support it.
ycmd/completers/general/filename_completer.py, line 111 at r6 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
The function also returns a tuple, but it doesn't say what do the tuple elements mean. We should have a comment.
There is this comment at the top of those tests:
# A serie of tests represented by tuples whose elements are:
# - the line to complete;
# - the environment variables;
# - the expected completions.
Do you feel it's not enough?
ycmd/completers/general/filename_completer.py, line 112 at r11 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
Where does
'prefix'
come from? Should we document it in our ycmd API, since it isn't documented?
This field is internal so it would be wrong to mention it in the API.
ycmd/completers/general/filename_completer.py, line 132 at r11 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
Why
last_match_start + 2
?
We want to return the position just after the latest path separator. Since last_match_start
is the position just before and is 0-indexed, we add 1 for the path separator and 1 for obtaining a 1-indexed column.
ycmd/completers/general/filename_completer.py, line 148 at r11 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
At least for C comments this isn't a problem, because you'll probably type
//
fast enough to only see a split second flicker. It's not pretty, but oh well. HTML tags are different.
Yes, the issue with HTML tags is more severe though it's slightly improved with the proposed changes because we now fall back to the identifier completer if there are no candidates, e.g.
<article>
</art
art
is not going to match a directory in the root folder so the article
identifier will be suggested.
ycmd/tests/filename_completer_test.py, line 86 at r3 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
Is there a reason why we wouldn't want to complete such path? I have not yet looked into the actual code, but it could be hard to handle
///home/bstaletic///Downloads
.
This is only if the path is exactly two or more separators i.e. //
, ///
, ////
, etc. ///home/bstaletic///Downloads
will be completed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While all discussions have been resolved, I'd like to give this a try before giving LGTM.
Reviewed 5 of 6 files at r12, 1 of 1 files at r13.
Reviewable status: 0 of 2 LGTMs obtained (and 1 stale)
ycmd/completers/general/filename_completer.py, line 47 at r6 (raw file):
It's something that we already support so we should still support it.
I guess I have never tried to complete that kind of path before.
ycmd/completers/general/filename_completer.py, line 111 at r6 (raw file):
Previously, micbou wrote…
There is this comment at the top of those tests:
# A serie of tests represented by tuples whose elements are: # - the line to complete; # - the environment variables; # - the expected completions.Do you feel it's not enough?
That's fine. I was talking about DetectPath
(now SearchPath
) not having a comment. Sorry for being unclear.
ycmd/completers/general/filename_completer.py, line 148 at r11 (raw file):
Previously, micbou wrote…
Yes, the issue with HTML tags is more severe though it's slightly improved with the proposed changes because we now fall back to the identifier completer if there are no candidates, e.g.
<article> </art
art
is not going to match a directory in the root folder so thearticle
identifier will be suggested.
Unless someone has some path that starts with /art
but doesn't contain article
.
Either way, this is fine if you ask me. There's always <C-y><C-x><C-p>
. *cough*vim specific*cough*
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 1 files at r13.
Reviewable status: 0 of 2 LGTMs obtained (and 1 stale)
ycmd/completers/general/filename_completer.py, line 148 at r11 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
Unless someone has some path that starts with
/art
but doesn't containarticle
.
Either way, this is fine if you ask me. There's always<C-y><C-x><C-p>
. *cough*vim specific*cough*
A simple solution would be to not trigger completion if there is a <
character before a slash in tag-based languages (HTML, XML, etc.). If we implement that, we could stop blacklisting the filepath completer in HTML, XML, and JSX files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 2 LGTMs obtained (and 2 stale)
ycmd/completers/general/filename_completer.py, line 148 at r11 (raw file):
Previously, micbou wrote…
A simple solution would be to not trigger completion if there is a
<
character before a slash in tag-based languages (HTML, XML, etc.). If we implement that, we could stop blacklisting the filepath completer in HTML, XML, and JSX files.
That doesn't sound like a bad idea.
Hi, Not sure this is the right place but I have an issue for path completion when the character '@' is starting a directory name. The path completion allows the directory name to be added but then restarts at the root instead of showing the subsequent files or subdirectories. vim 8.1.436 Thank you. |
@erouvin Are you experiencing the issue with the changes in that PR? |
Sorry, I am new in this. How can I tell if I get the right release ? |
37a4f53
to
b0dc7c2
Compare
To try these changes, navigate to the ycmd folder and fetch the PR: cd ~/.vim/bundle/YouCompleteMe/third_party/ycmd
git fetch origin pull/1104/head:filepath-completer-improvements
git checkout filepath-completer-improvements then start Vim and check if completion works with paths starting with git checkout master |
@micbou . This solves the issue. Thanks a lot. |
b0dc7c2
to
c1bc916
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 3 of 6 files at r12, 1 of 1 files at r13.
Reviewable status: 1 of 2 LGTMs obtained (and 1 stale)
ycmd/completers/general/filename_completer.py, line 111 at r6 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
That's fine. I was talking about
DetectPath
(nowSearchPath
) not having a comment. Sorry for being unclear.
fwiw series is both singular and plural ("a series", not "a serie")
ycmd/completers/general/filename_completer.py, line 54 at r13 (raw file):
PATH_SEPARATORS_PATTERN = '([{seps}][^{seps}]*|[{seps}]$)' HEAD_PATH_PATTERN_UNIX = '\.{1,2}|~|\$[^$]+'
i feel like the previous regex which had comments was easier to follow. Could we use the extended style and reinstate things like the comment # 'D:/'-like token
?
ycmd/completers/general/filename_completer.py, line 132 at r13 (raw file):
return None, None working_dir = self.GetWorkingDirectory( request_data )
do you think it is worth cacheing the compiled regex for the WD when we get the OnFileReadyToParse event? That should speed this up the first time we get a completion request that doesn't hit semantic? i think that we can just call GetCompiledHeadRegexForDirectory
in OnFileReadyToParse
. Might not be necessary, but filesystem metadata is notoriously slow in corporate environments.
ycmd/completers/general/filename_completer.py, line 156 at r13 (raw file):
if os.path.exists( path ): # +2 because last_match_start is the 0-indexed position just before # the latest path separator.
and we assume that the separator is exactly 1 "character" ? Should we prefer len( os.path.sep )? I realise that in practice, it's always 1
ycmd/completers/general/filename_completer.py, line 178 at r13 (raw file):
# languages where such characters are part of special constructs like # comments in C/C++ or closing tags in HTML. This behavior could be improved # by using rules that depend on the filetype.
or at least, heuristically, Such as if the character immediately prior is <
then don't. At least that would work for XML. LaTeX would be harder to deal with. Anyway, different PR...
ycmd/completers/general/general_completer_store.py, line 63 at r13 (raw file):
def ShouldUseNow( self, request_data ):
I'm not seeing how this ShouldUseNow
is replaced. Did I miss something?
ycmd/completers/general/general_completer_store.py, line 67 at r13 (raw file):
return candidates # The filename completer may have changed the start column. Reset it.
I don't think the filename completer should change the start column if it didn't return anything. that seems like it is being naughty. When would this happen?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 5 of 9 files at r1, 1 of 4 files at r2, 1 of 4 files at r11, 2 of 6 files at r12.
Reviewable status: 1 of 2 LGTMs obtained (and 1 stale)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 1 of 2 LGTMs obtained (and 1 stale)
ycmd/completers/general/filename_completer.py, line 156 at r13 (raw file):
Previously, puremourning (Ben Jackson) wrote…
and we assume that the separator is exactly 1 "character" ? Should we prefer len( os.path.sep )? I realise that in practice, it's always 1
There's also os.path.altsep
if you want to be even more nipicky, but I did call it nitpicky for a reason.
ycmd/completers/general/general_completer_store.py, line 67 at r13 (raw file):
Previously, puremourning (Ben Jackson) wrote…
I don't think the filename completer should change the start column if it didn't return anything. that seems like it is being naughty. When would this happen?
Filename completer always sets the start column, provided ShouldUseNow
returns True
. We just aren't sure if it changed the value.
c1bc916
to
0053e82
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 4 files at r14.
Reviewable status: 0 of 2 LGTMs obtained (and 2 stale)
ycmd/completers/general/filename_completer.py, line 111 at r6 (raw file):
Previously, puremourning (Ben Jackson) wrote…
fwiw series is both singular and plural ("a series", not "a serie")
I blame my native language (or should I blame yours?). Fixed.
ycmd/completers/general/filename_completer.py, line 54 at r13 (raw file):
Previously, puremourning (Ben Jackson) wrote…
i feel like the previous regex which had comments was easier to follow. Could we use the extended style and reinstate things like the comment
# 'D:/'-like token
?
Done.
ycmd/completers/general/filename_completer.py, line 132 at r13 (raw file):
Previously, puremourning (Ben Jackson) wrote…
do you think it is worth cacheing the compiled regex for the WD when we get the OnFileReadyToParse event? That should speed this up the first time we get a completion request that doesn't hit semantic? i think that we can just call
GetCompiledHeadRegexForDirectory
inOnFileReadyToParse
. Might not be necessary, but filesystem metadata is notoriously slow in corporate environments.
I am not sure. The issue is that the FileReadyToParse
event is not triggered when changing the working directory in the client so there is a good chance the compiled regex won't be cached until completion.
ycmd/completers/general/filename_completer.py, line 156 at r13 (raw file):
Previously, bstaletic (Boris Staletic) wrote…
There's also
os.path.altsep
if you want to be even more nipicky, but I did call it nitpicky for a reason.
The comment now mentions that the length of path separators is always 1. Does that work for you?
ycmd/completers/general/filename_completer.py, line 178 at r13 (raw file):
Such as if the character immediately prior is < then don't
That's my idea for tag-based languages.
LaTeX would be harder to deal with.
\
should be ignored for TeX languages. It's a reserved character and cannot be escaped.
ycmd/completers/general/general_completer_store.py, line 63 at r13 (raw file):
Previously, puremourning (Ben Jackson) wrote…
I'm not seeing how this
ShouldUseNow
is replaced. Did I miss something?
It's never called with the changes in ComputeCandidates
.
ycmd/completers/general/general_completer_store.py, line 67 at r13 (raw file):
I don't think the filename completer should change the start column if it didn't return anything. that seems like it is being naughty. When would this happen?
This happens when the query does not match any candidates. It's something we can't know beforehand. What we can do though is resetting the start column directly in the filename completer when there are no candidates.
Add the following improvements to the filename completer: - allow completion of paths containing spaces; - allow completion of multiple paths on the same line; - allow completion of relative paths not starting with ".", "..", or "./"; - allow completion of Windows environment variables (e.g. %USERPROFILE%).
0053e82
to
ddecdca
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 0 of 2 LGTMs obtained (and 3 stale)
ycmd/completers/general/filename_completer.py, line 111 at r6 (raw file):
Previously, micbou wrote…
I blame my native language (or should I blame yours?). Fixed.
You can blame English :) We don't have a whole académie to regulate it :)
ycmd/completers/general/filename_completer.py, line 132 at r13 (raw file):
Previously, micbou wrote…
I am not sure. The issue is that the
FileReadyToParse
event is not triggered when changing the working directory in the client so there is a good chance the compiled regex won't be cached until completion.
Fair point. Let's call it yagni and if there are real problems reported, we can think about it.
ycmd/completers/general/filename_completer.py, line 156 at r13 (raw file):
Previously, micbou wrote…
The comment now mentions that the length of path separators is always 1. Does that work for you?
Yup
@zzbot r+ |
📌 Commit ddecdca has been approved by |
…etic [READY] Improve filename completer This PR brings the following improvements to the filename completer: - allow completion of paths containing spaces; - allow completion of multiple paths on the same line; - allow completion of relative paths not starting with `.`, `..`, or `./`; - allow completion of Windows environment variables (e.g. `%USERPROFILE%`). The first three improvements are achieved by assuming that if a user types `[something]/` and `[something]/` is an existing path (absolute or relative to the working directory) then he want to complete that path. <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/ycmd/1104) <!-- Reviewable:end -->
💔 Test failed - status-travis |
@zzbot retry |
…etic [READY] Improve filename completer This PR brings the following improvements to the filename completer: - allow completion of paths containing spaces; - allow completion of multiple paths on the same line; - allow completion of relative paths not starting with `.`, `..`, or `./`; - allow completion of Windows environment variables (e.g. `%USERPROFILE%`). The first three improvements are achieved by assuming that if a user types `[something]/` and `[something]/` is an existing path (absolute or relative to the working directory) then he want to complete that path. <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/ycmd/1104) <!-- Reviewable:end -->
☀️ Test successful - status-appveyor, status-travis |
[READY] Update ycmd Include the following changes: - PR ycm-core/ycmd#1080: replace Boost canonical function with our own implementation; - PR ycm-core/ycmd#1104: improve filename completer; - PR ycm-core/ycmd#1121: support completion FixIts for C-family languages; - PR ycm-core/ycmd#1122: update jdt.ls to 0.26.0; - PR ycm-core/ycmd#1123: install fixed version of TypeScript in third-party folder; - PR ycm-core/ycmd#1124: only add the necessary directories to Python path. Fixes #3173. <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/3174) <!-- Reviewable:end -->
This PR brings the following improvements to the filename completer:
.
,..
, or./
;%USERPROFILE%
).The first three improvements are achieved by assuming that if a user types
[something]/
and[something]/
is an existing path (absolute or relative to the working directory) then he want to complete that path.This change is