Skip to content

twisted.web._flatten._getSlotValue copies slotData on every call. #11834

@dreid

Description

@dreid

Describe the incorrect behavior you saw

_getSlotValue attempts retrieve a value for a named slot from the slotData stack by using slice syntax to reverse it and then iterate over it.

This also causes it create a copy of the list.

In templates with large amounts of data such those used by pydoctor's summary pages. This copying ends up being a significant amount of time spent generating summary templates.

Describe how to cause this behavior

I discovered this issue while running pydoctor under the austin profiler.

I can reproduce it on twisted's own apidocs.

You can view the results of this profiling in speedscope: https://www.speedscope.app/#profileURL=https%3A%2F%2Fgist.githubusercontent.com%2Fdreid%2F197566471f39a96523f5065d19d0bf7f%2Fraw%2F3e7ec92a17bc82d40acceb1e2efcaa3ef7c8ef07%2Ftwisted-austin-trunk.speedscope

In the above profile _getSlotValue accounts for ~55s of the total runtime which was 509.42.

Describe the correct behavior you'd like to see
slotData should not need to be copied on every call to _getSlotValue simply to iterate over it in reverse.

Initial profiling using reversed instead resulted in reduced runtime to 357.36s

speedscope profile of using reversed is here: https://www.speedscope.app/#profileURL=https%3A%2F%2Fgist.githubusercontent.com%2Fdreid%2F197566471f39a96523f5065d19d0bf7f%2Fraw%2F3e7ec92a17bc82d40acceb1e2efcaa3ef7c8ef07%2Ftwisted-pydoctor-reversed.speedscope

Testing environment

  • Operating System and Version; paste the output of these commands:
    • on Linux, uname -a ; cat /etc/lsb-release
    • on Windows, systeminfo | Findstr /i "OS"
    • on macOS, sw_vers
  • Twisted version [e.g. 22.2.0]
    • please paste the output of twist --version and pip --freeze
  • Reactor [e.g. select, iocp]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions