os.makedirs fails on long-path UNC-paths if it is the first sub-folder

[Safihre]

BPO	41705
Nosy	@pfmoore, @tjguk, @zware, @eryksun, @zooba, @Safihre

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-09-03.19:28:37.918>
labels = ['type-bug', '3.8', 'OS-windows']
title = 'os.makedirs fails on long-path UNC-paths if it is the first sub-folder'
updated_at = <Date 2020-09-03.22:01:32.759>
user = 'https://github.com/Safihre'

bugs.python.org fields:

activity = <Date 2020-09-03.22:01:32.759>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Windows']
creation = <Date 2020-09-03.19:28:37.918>
creator = 'Safihre'
dependencies = []
files = []
hgrepos = []
issue_num = 41705
keywords = []
message_count = 2.0
messages = ['376312', '376326']
nosy_count = 6.0
nosy_names = ['paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'Safihre']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41705'
versions = ['Python 3.8']

[Safihre]

It consistently fails on the first directory in a long-path UNC notation server-folder.

>>> os.makedirs(r"\\?\UNC\DiskStation\already_exists", exist_ok=True)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\UNC\\'

>>> os.makedirs(r"\\?\UNC\DiskStation\already_exists")
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '\\\\?\\UNC\\'

The second level directory is working correctly as expected:

>> os.makedirs(r"\\?\UNC\DiskStation\already_exists\new")

>>> os.makedirs(r"\\?\UNC\DiskStation\already_exists\new")
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\safihre\AppData\Local\Programs\Python\Python38\lib\os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '\\\\?\\UNC\\DiskStation\\test_get2\\test2'

Inspecting the code, I think the problem is in the os.path.exists function that is called from within os.makedirs, the line in os.makedirs says:
if head and tail and not path.exists(head):

But:
>>> head, tail = path.split(r"\\?\UNC\DiskStation\already_exists")
('\\\\?\\UNC\\DiskStation', 'already_exists')

>>> os.path.exists(r'\\?\UNC\DiskStation')
False
>>> os.path.exists(r'\\DiskStation')
False

So it wrongly goes ahead and tries to create \\?\UNC\DiskStation

[eryksun]

This behavior is due to bpo-37609, i.e. ntpath.splitdrive fails for "UNC" device paths.

    >>> ntpath.splitdrive('//?/UNC/server/share')
    ('//?/UNC', '/server/share')

The result should be "//?/UNC/server/share". A path on the "UNC" device requires a share component, on which is mounted a local or remote filesystem directory. It's functionally part of the 'drive' path. Using just the root path or a server path on the "UNC" device is malformed in the context of a normal file API open. The former fails as ERROR_INVALID_NAME (123), and the latter fails as ERROR_BAD_PATHNAME (161).

The incorrect splitdrive result in turn makes ntpath.split misbehave:

    >>> ntpath.split('//?/UNC/server/share')
    ('//?/UNC/server', 'share')
    >>> ntpath.split('//?/UNC/server')
    ('//?/UNC/', 'server')

The correct result should be ('//?/UNC/server/share', '').

[Safihre]

@barneygale can you confirm if this is also resolved with your patch? The source of the problem is indicated by @eryksun to be in splitdrive.
#85871 (comment)

[picnixz]

@barneygale friendly ping for confirmation (otherwise, we can close this issue)