Skip to content

Problem with contextlib.nullcontext #78248

Open

Description

@serhiy-storchaka
BPO 34067
Nosy @ncoghlan, @ericvsmith, @serhiy-storchaka, @miss-islington, @iritkatriel
PRs
  • bpo-34067: Include a more easily understood example for nullcontext #8158
  • [3.7] bpo-34067: Include a more easily understood example for nullcontext (GH-8158) #8199
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2018-07-07.17:33:25.936>
    labels = ['3.8', '3.9', '3.10', 'performance']
    title = 'Problem with contextlib.nullcontext'
    updated_at = <Date 2020-11-02.14:13:49.172>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2020-11-02.14:13:49.172>
    actor = 'iritkatriel'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = []
    creation = <Date 2018-07-07.17:33:25.936>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 34067
    keywords = ['patch']
    message_count = 8.0
    messages = ['321226', '321254', '321319', '321322', '321461', '321463', '321472', '378332']
    nosy_count = 5.0
    nosy_names = ['ncoghlan', 'eric.smith', 'serhiy.storchaka', 'miss-islington', 'iritkatriel']
    pr_nums = ['8158', '8199']
    priority = 'normal'
    resolution = None
    stage = 'needs patch'
    status = 'open'
    superseder = None
    type = 'resource usage'
    url = 'https://bugs.python.org/issue34067'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    Activity

    1. serhiy-storchaka commented on Jul 7, 2018

      @serhiy-storchaka
      MemberAuthor

      There is a flaw in the example in the documentation for contextlib.nullcontext.

            def process_file(file_or_path):
                if isinstance(file_or_path, str):
                    # If string, open file
                    cm = open(file_or_path)
                else:
                    # Caller is responsible for closing file
                    cm = nullcontext(file_or_path)
      
                with cm as file:
                    # Perform processing on the file

      The execution can be interrupted by Ctrl-C between opening a file and entering a 'with' block. There is the same problem with the simple "with open(path) as file:", but it can be easily solved (see bpo-34066). It is harder to do for this example, because there are more opcodes between calling open() and entering 'with'. I afraid that this problem can't be solved in general case. Maybe contextlib.nullcontext has a flaw similar to the flaw in contextlib.nested().

    2. ncoghlan commented on Jul 8, 2018

      @ncoghlan
      Contributor

      That's certainly similar to the problems with contextlib.nested, but I don't think it's as inherently flawed as nested was. What I'd suggest we change the existing example to is this:

            from functools import partial
            from os import fspath
      
            def process_file(file_or_path):
                try:
                    as_fspath = os.fspath(file_or_path)
                except TypeError:
                    # If it's a file, caller is responsible for closing it
                    make_cm = partial(nullcontext, file_or_path)
                else:
                    # If it's a path, open file when the context is entered
                    make_cm = partial(open, as_fspath)
      
                with make_cm() as file:
                    # Perform processing on the file

      Optionally, we could also present a cleaner example where a pre-created context manager is passed in and we're just coping with the fact it may be None:

          def update_resource(resource, updates, resource_lock=None):
              if resource_lock is None:
                  resource_lock = nullcontext()
              with resource_lock:
                  resource.apply_updates(updates)

      (I'll also note that ExitStack is *far* from being immune to the Ctrl-C problem, as it's implemented in Python itself, which allows its __exit__ method to be interrupted, as well as for interrupt to occur between a resource being created or acquired, and it being registered with the stack)

    3. ncoghlan commented on Jul 9, 2018

      @ncoghlan
      Contributor

      New changeset c287545 by Nick Coghlan (Daniel Porteous) in branch 'master':
      bpo-34067: Include a more easily understood example for nullcontext (GH-8158)
      c287545

    4. miss-islington commented on Jul 9, 2018

      @miss-islington
      Contributor

      New changeset 1b6e217 by Miss Islington (bot) in branch '3.7':
      bpo-34067: Include a more easily understood example for nullcontext (GH-8158)
      1b6e217

    5. vstinner commented on Jul 11, 2018

      @vstinner
      Member

      Can we close this issue?

    6. serhiy-storchaka commented on Jul 11, 2018

      @serhiy-storchaka
      MemberAuthor

      No, the merged PR is not directly related to the original issue.

    7. ncoghlan commented on Jul 11, 2018

      @ncoghlan
      Contributor

      Right, the merged PR adds a simpler example that handles "ignore exceptions or not" by selecting between suppress() and nullcontext().

      The documentation still needs updating to improve or remove the file-based example, and point out that for resources that get acquired on construction, you either need to delay construction to the start of the with statement, or else use ExitStack instead of nullcontext.

    8. transferred this issue fromon Apr 10, 2022
    9. added
      docsDocumentation in the Doc dir
      and removed
      performancePerformance or resource usage
      on Aug 11, 2022
    10. gpshead commented on Apr 24, 2023

      @gpshead
      Member

      triage: removed the docs and easy tags as the remaining questions weren't right to call clear or "easy" for a new person in a sprint. :)

    11. added
      stdlibStandard Library Python modules in the Lib/ directory
      on Nov 23, 2023
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      stdlibStandard Library Python modules in the Lib/ directory

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions