A full build of all languages / versions has been taking somewhere between 24 and 50 hours (#169, but python/cpython#123113 should cut about a third).
We update the CPython repo once at the start, then loop each language/version combo:
|
cpython_repo.update() |
|
while todo: |
|
version, language = todo.pop() |
|
logging.root.handlers[0].setFormatter( |
|
logging.Formatter( |
|
f"%(asctime)s %(levelname)s {language.tag}/{version.name}: %(message)s" |
|
) |
|
) |
|
if sentry_sdk: |
|
with sentry_sdk.configure_scope() as scope: |
|
scope.set_tag("version", version.name) |
|
scope.set_tag("language", language.tag) |
|
builder = DocBuilder( |
|
version, versions, language, languages, cpython_repo, **vars(args) |
|
) |
|
all_built_successfully &= builder.run() |
This means builds near the end of the loop will be using a Git commit which could be a day or two old.
For example, looking at the current logs:
15502:2024-08-24 16:07:01,574 DEBUG: Run: 'git -C /srv/docsbuild/cpython fetch'
It's currently 2024-08-25 10:45, meaning current builds are using an 18-hour-old commit, and we're about half way through a full build.
So let's instead update the CPython repo before each language/version, perhaps by moving the update inside the while loop:
cpython_repo = Repository(
"https://github.com/python/cpython.git", args.build_root / "cpython"
)
- cpython_repo.update()
while todo:
version, language = todo.pop()
logging.root.handlers[0].setFormatter(
f"%(asctime)s %(levelname)s {language.tag}/{version.name}: %(message)s"
)
)
if sentry_sdk:
with sentry_sdk.configure_scope() as scope:
scope.set_tag("version", version.name)
scope.set_tag("language", language.tag)
+ cpython_repo.update()
builder = DocBuilder(
version, versions, language, languages, cpython_repo, **vars(args)
)
A full build of all languages / versions has been taking somewhere between 24 and 50 hours (#169, but python/cpython#123113 should cut about a third).
We update the CPython repo once at the start, then loop each language/version combo:
docsbuild-scripts/build_docs.py
Lines 1120 to 1135 in 56d72d4
This means builds near the end of the loop will be using a Git commit which could be a day or two old.
For example, looking at the current logs:
It's currently 2024-08-25 10:45, meaning current builds are using an 18-hour-old commit, and we're about half way through a full build.
So let's instead update the CPython repo before each language/version, perhaps by moving the update inside the
whileloop:cpython_repo = Repository( "https://github.com/python/cpython.git", args.build_root / "cpython" ) - cpython_repo.update() while todo: version, language = todo.pop() logging.root.handlers[0].setFormatter( f"%(asctime)s %(levelname)s {language.tag}/{version.name}: %(message)s" ) ) if sentry_sdk: with sentry_sdk.configure_scope() as scope: scope.set_tag("version", version.name) scope.set_tag("language", language.tag) + cpython_repo.update() builder = DocBuilder( version, versions, language, languages, cpython_repo, **vars(args) )