Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crash when parsing / splitting property, or using stdlib base clase #27

Merged
merged 1 commit into from
Dec 20, 2020

Conversation

tony
Copy link
Contributor

@tony tony commented Aug 2, 2020

class RepoLoggingAdapter(logging.LoggerAdapter):

    """Adapter for adding Repo related content to logger.

    - [`RepoLoggingAdapter.bin_name`](RepoLoggingAdapter.bin_name) -> ``repo_vcs``
    - [`RepoLoggingAdapter.name`](RepoLoggingAdapter.name) -> ``repo_name``
    """

Full file (since diff may be rebased):

# -*- coding: utf-8 -*-
"""Utility functions for libvcs."""
from __future__ import absolute_import, print_function, unicode_literals

import datetime
import errno
import logging
import os
import subprocess

from . import exc
from ._compat import console_to_str

logger = logging.getLogger(__name__)


def which(
    exe=None, default_paths=['/bin', '/sbin', '/usr/bin', '/usr/sbin', '/usr/local/bin']
):
    """Return path of bin. Python clone of /usr/bin/which.

    from salt.util - https://www.github.com/saltstack/salt - license apache

    Parameters
    ----------
    exe : str
        Application to search PATHs for.
    default_path : list
        Application to search PATHs for.

    Returns
    -------
    str :
        Path to binary
    """

    def _is_executable_file_or_link(exe):
        # check for os.X_OK doesn't suffice because directory may executable
        return os.access(exe, os.X_OK) and (os.path.isfile(exe) or os.path.islink(exe))

    if _is_executable_file_or_link(exe):
        # executable in cwd or fullpath
        return exe

    # Enhance POSIX path for the reliability at some environments, when
    # $PATH is changing. This also keeps order, where 'first came, first
    # win' for cases to find optional alternatives
    search_path = (
        os.environ.get('PATH') and os.environ['PATH'].split(os.pathsep) or list()
    )
    for default_path in default_paths:
        if default_path not in search_path:
            search_path.append(default_path)
    os.environ['PATH'] = os.pathsep.join(search_path)
    for path in search_path:
        full_path = os.path.join(path, exe)
        if _is_executable_file_or_link(full_path):
            return full_path
    logger.info(
        '\'{0}\' could not be found in the following search path: '
        '\'{1}\''.format(exe, search_path)
    )

    return None


def mkdir_p(path):
    """Make directories recursively.

    Parameters
    ----------
    path : str
        path to create
    """
    try:
        os.makedirs(path)
    except OSError as exc:  # Python >2.5
        if exc.errno == errno.EEXIST and os.path.isdir(path):
            pass
        else:
            raise Exception('Could not create directory %s' % path)


class RepoLoggingAdapter(logging.LoggerAdapter):

    """Adapter for adding Repo related content to logger.

    Extends `logging.LoggerAdapter`'s functionality.

    The standard library :py:mod:`logging` facility is pretty complex, so this
    warrants and explanation of what's happening.

    Any class that subclasses this will have its class attributes for:

    - [`RepoLoggingAdapter.bin_name`](RepoLoggingAdapter.bin_name) -> ``repo_vcs``
    - [`RepoLoggingAdapter.name`](RepoLoggingAdapter.name) -> ``repo_name``

    Added to a dictionary of context information in :py:meth:`
    logging.LoggerAdapter.process()` to be made use of when the user of this
    library wishes to use a custom :class:`logging.Formatter` to output
    results.

    """

    def __init__(self, *args, **kwargs):
        logging.LoggerAdapter.__init__(self, *args, **kwargs)

    def process(self, msg, kwargs):
        """Add additional context information for loggers."""
        prefixed_dict = {}
        prefixed_dict['repo_vcs'] = self.bin_name
        prefixed_dict['repo_name'] = self.name

        kwargs["extra"] = prefixed_dict

        return msg, kwargs


def run(
    cmd,
    shell=False,
    cwd=None,
    log_in_real_time=True,
    check_returncode=True,
    callback=None,
):
    """ Run 'cmd' in a shell and return the combined contents of stdout and
    stderr (Blocking).  Throws an exception if the command exits non-zero.

    Parameters
    ----------
    cmd : list or str, or single str, if shell=True
       the command to run

    shell : boolean
        boolean indicating whether we are using advanced shell
        features. Use only when absolutely necessary, since this allows a lot
        more freedom which could be exploited by malicious code. See the
        warning here:
        http://docs.python.org/library/subprocess.html#popen-constructor

    cwd : str
        dir command is run from. Defaults to ``path``.

    log_in_real_time : boolean
        boolean indicating whether to read stdout from the
        subprocess in real time instead of when the process finishes.

    check_returncode : bool
        Indicate whether a `libvcs.exc.CommandError` should be raised if return code is
        different from 0.

    callback : callable
        callback to return output as a command executes, accepts a function signature
        of `(output, timestamp)`. Example usage:

        def progress_cb(output, timestamp):
            sys.stdout.write(output)
            sys.stdout.flush()
        run(['git', 'pull'], callback=progrses_cb)
    """
    proc = subprocess.Popen(
        cmd, shell=shell, stderr=subprocess.PIPE, stdout=subprocess.PIPE, cwd=cwd,
    )

    all_output = []
    code = None
    line = None
    while code is None:
        code = proc.poll()

        # output = console_to_str(proc.stdout.readline())
        # all_output.append(output)
        if callback and callable(callback):
            line = console_to_str(proc.stderr.read(128))
            if line:
                callback(output=line, timestamp=datetime.datetime.now())
    if callback and callable(callback):
        callback(output='\r', timestamp=datetime.datetime.now())

    lines = filter(None, (line.strip() for line in proc.stdout.readlines()))
    all_output = console_to_str(b'\n'.join(lines))
    if code:
        stderr_lines = filter(None, (line.strip() for line in proc.stderr.readlines()))
        all_output = console_to_str(b''.join(stderr_lines))
    output = ''.join(all_output)
    if code != 0 and check_returncode:
        raise exc.CommandError(output=output, returncode=code, cmd=cmd)
    return output

Modificatons to print() / debug mkapi/core/object.py:

def get_fullname(obj: Any, name: str) -> str:
    """Reutrns an object full name specified by `name`.

    Args:
        obj: Object that has a module.
        name: Object name in the module.

    Examples:
        >>> obj = get_object('mkapi.core.base.Item')
        >>> get_fullname(obj, 'Section')
        'mkapi.core.base.Section'
        >>> get_fullname(obj, 'preprocess')
        'mkapi.core.preprocess'
        >>> get_fullname(obj, 'abc')
        ''
    """
    if not hasattr(obj, "__module__"):
        return ""
    obj = importlib.import_module(obj.__module__)
    names = name.split(".")

    for name in names:
        if not hasattr(obj, name):
            return ""
        obj = getattr(obj, name)

    if isinstance(obj, property):
        print('found a property, this will crash, names:', names)
        # return ""

    return ".".join(split_prefix_and_name(obj))


def split_prefix_and_name(obj: Any) -> Tuple[str, str]:
    """Splits an object full name into prefix and name.

    Args:
        obj: Object that has a module.

    Examples:
        >>> import inspect
        >>> obj = get_object('mkapi.core')
        >>> split_prefix_and_name(obj)
        ('mkapi', 'core')
        >>> obj = get_object('mkapi.core.base')
        >>> split_prefix_and_name(obj)
        ('mkapi.core', 'base')
        >>> obj = get_object('mkapi.core.node.Node')
        >>> split_prefix_and_name(obj)
        ('mkapi.core.node', 'Node')
        >>> obj = get_object('mkapi.core.node.Node.get_markdown')
        >>> split_prefix_and_name(obj)
        ('mkapi.core.node.Node', 'get_markdown')
    """
    if inspect.ismodule(obj):
        prefix, _, name = obj.__name__.rpartition(".")
    else:
        print(obj, type(obj), str(obj), obj, inspect.getdoc(obj))
        module = obj.__module__
        qualname = obj.__qualname__
        if "." not in qualname:
            prefix, name = module, qualname
        else:
            prefix, _, name = qualname.rpartition(".")
            prefix = ".".join([module, prefix])
        if prefix == "__main__":
            prefix = ""
    return prefix, name
<function GitRepo.update_repo at 0x7f964355f8c0> <class 'function'> <function GitRepo.update_repo at 0x7f964355f8c0> <function GitRepo.update_repo at 0x7f964355f8c0> None
<function LoggerAdapter.warn at 0x7f9645855290> <class 'function'> <function LoggerAdapter.warn at 0x7f9645855290> <function LoggerAdapter.warn at 0x7f9645855290> None
<function LoggerAdapter.warning at 0x7f9645855200> <class 'function'> <function LoggerAdapter.warning at 0x7f9645855200> <function LoggerAdapter.warning at 0x7f9645855200> Delegate a warning call to the underlying
 logger.
<function extract_status at 0x7f964355f560> <class 'function'> <function extract_status at 0x7f964355f560> <function extract_status at 0x7f964355f560> Returns ``git status -sb --porcelain=2`` extracted to a dict

Returns
-------
dict :
    Dictionary of git repo's status
<class 'libvcs.util.RepoLoggingAdapter'> <class 'type'> <class 'libvcs.util.RepoLoggingAdapter'> <class 'libvcs.util.RepoLoggingAdapter'> Adapter for adding Repo related content to logger.

Extends `logging.LoggerAdapter`'s functionality.

The standard library :py:mod:`logging` facility is pretty complex, so this
warrants and explanation of what's happening.

Any class that subclasses this will have its class attributes for:

- [`RepoLoggingAdapter.bin_name`](RepoLoggingAdapter.bin_name) -> ``repo_vcs``
- [`RepoLoggingAdapter.name`](RepoLoggingAdapter.name) -> ``repo_name``
Added to a dictionary of context information in :py:meth:`
logging.LoggerAdapter.process()` to be made use of when the user of thislibrary wishes to use a custom :class:`logging.Formatter` to output
results.
found a property, this will crash, names: ['RepoLoggingAdapter', 'name']<property object at 0x7f9645845dd0> <class 'property'> <property object at 0x7f9645845dd0> <property object at 0x7f9645845dd0> None
Traceback (most recent call last):  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/bin/mkdocs", line 10, in <module>
    sys.exit(cli())
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/click/core.py", line 1066, in invoke    return ctx.invoke(self.callback, **ctx.params)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/click/core.py", line 610, in invoke    return callback(*args, **kwargs)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkdocs/__main__.py", line 152, in build_command    build.build(config.load_config(**kwargs), dirty=not clean)  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkdocs/commands/build.py", line 236, in build    config = config['plugins'].run_event('config', config)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkdocs/plugins.py", line 94, in run_event    result = method(item, **kwargs)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/plugins/mkdocs.py", line 51, in on_config    config, self.config["filters"]
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/plugins/api.py", line 24, in create_nav    value, docs_dir, config_dir, global_filters
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/plugins/api.py", line 47, in collect    module = get_module(package_path)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/module.py", line 104, in get_module    module = Module(obj)  File "<string>", line 4, in __init__
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/module.py", line 27, in __post_init__    super().__post_init__()
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/structure.py", line 106, in __post_init__    self.members = self.get_members()
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/module.py", line 48, in get_members    return get_members(self.obj)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/module.py", line 81, in get_members
    module = get_module(name)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/module.py", line 104, in get_module
    module = Module(obj)
  File "<string>", line 4, in __init__
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/module.py", line 28, in __post_init__
    self.node = get_node(self.obj)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/node.py", line 229, in get_node
    return Node(obj, sourcefile_index)
  File "<string>", line 5, in __init__
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/node.py", line 31, in __post_init__
    super().__post_init__()
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/structure.py", line 106, in __post_init__
    self.members = self.get_members()
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/node.py", line 82, in get_members
    return get_members(self.obj)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/node.py", line 209, in get_members
    member = get_node(obj, sourcefile_index)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/node.py", line 229, in get_node
    return Node(obj, sourcefile_index)
  File "<string>", line 5, in __init__
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/node.py", line 31, in __post_init__
    super().__post_init__()
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/structure.py", line 104, in __post_init__
    self.docstring = get_docstring(obj)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/docstring.py", line 295, in get_docstring
    postprocess(docstring, obj)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/docstring.py", line 272, in postprocess
    base.markdown = replace_link(obj, base.markdown)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/linker.py", line 209, in replace_link
    return re.sub(REPLACE_LINK_PATTERN, replace, markdown)
  File "/home/t/.zinit/plugins/pyenv/versions/3.7.7/lib/python3.7/re.py", line 192, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/linker.py", line 201, in replace
    fullname = get_fullname(obj, name)
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/object.py", line 72, in get_fullname
    return ".".join(split_prefix_and_name(obj))
  File "/home/t/.cache/pypoetry/virtualenvs/libvcs-c8x14mkR-py3.7/lib/python3.7/site-packages/mkapi/core/object.py", line 100, in split_prefix_and_name
    module = obj.__module__
AttributeError: 'property' object has no attribute '__module__'
make[1]: *** [Makefile:26: build_docs] Error 1
make[1]: Leaving directory '/home/t/work/python/libvcs'

```python
class RepoLoggingAdapter(logging.LoggerAdapter):

    """Adapter for adding Repo related content to logger.

    - [`RepoLoggingAdapter.bin_name`](RepoLoggingAdapter.bin_name) -> ``repo_vcs``
    - [`RepoLoggingAdapter.name`](RepoLoggingAdapter.name) -> ``repo_name``
    """
```
@tony tony mentioned this pull request Aug 2, 2020
3 tasks
@tony tony changed the title Fix crash when parsing / splitting property Fix crash when parsing / splitting property, or using stdlib base clase Aug 2, 2020
@tony
Copy link
Contributor Author

tony commented Aug 2, 2020

Possibly related to #26 - since it's trying to access a value inherited from logging

tony added a commit to tony/mkapi that referenced this pull request Aug 2, 2020
…d base class

```python
class RepoLoggingAdapter(logging.LoggerAdapter):

    """Adapter for adding Repo related content to logger.

    - [`RepoLoggingAdapter.bin_name`](RepoLoggingAdapter.bin_name) -> ``repo_vcs``
    - [`RepoLoggingAdapter.name`](RepoLoggingAdapter.name) -> ``repo_name``
    """
```

See also:
- daizutabi#27
- Possibly related to daizutabi#26
tony added a commit to vcs-python/libvcs that referenced this pull request Aug 2, 2020
tony added a commit to tony/mkapi that referenced this pull request Aug 3, 2020
…d base class

```python
class RepoLoggingAdapter(logging.LoggerAdapter):

    """Adapter for adding Repo related content to logger.

    - [`RepoLoggingAdapter.bin_name`](RepoLoggingAdapter.bin_name) -> ``repo_vcs``
    - [`RepoLoggingAdapter.name`](RepoLoggingAdapter.name) -> ``repo_name``
    """
```

See also:
- daizutabi#27
- Possibly related to daizutabi#26
tony added a commit to vcs-python/libvcs that referenced this pull request Aug 6, 2020
tony added a commit to vcs-python/libvcs that referenced this pull request Aug 16, 2020
@daizutabi daizutabi merged commit de4313a into daizutabi:master Dec 20, 2020
@tony tony deleted the fix-splitting-property-crash branch January 14, 2024 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants