Fewer github api calls for performance#27
Conversation
* For a user look up all their user repos & organization repos and cache results. * When there are a large number of jobs/repositories, this is much more performant than asking each repository for its list of members which is what it was previously doing. * There was also a bug with caching if the first person to ask about a private repository had no access (it incorrectly cached [] as the list of members)
|
plugins » github-oauth-plugin #55 SUCCESS |
|
Created jira issue for this: JENKINS-24069 (https://issues.jenkins-ci.org/browse/JENKINS-24069) |
|
@alexrothenberg Assuming this fix is for when you are using Github authorization . When does the cache expire? Looks like user would have access even if he is removed from the github org until there is a jenkins restart? |
|
Yes this is for when we are using github authorization and letting the repo permission determine the job permission. It is intended as a fix to PR #17 I submitted several months back. I think the cache expires after an hour |
|
I just looked it up (https://code.google.com/p/guava-libraries/wiki/CachesExplained#Timed_Eviction) and yes it should expire after 1 hour.
|
|
Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests |
|
@alexrothenberg Got it. thanks for the link. That makes sense. I didn't look at the code, I just tested it functionally. I guess, 1 hr lag is not bad for most people. This PR works as intended based on my (limited) functional testing. Ideal solution would be to listen on |
|
Ey guys, any new about this being merged? I'm experiencing the same problem too |
|
@skottler I saw this is assigned to you in jira JENKINS-24069. Anything I can do to help move this forward? |
|
👍 Having timeout issues as well due to this; please merge! |
|
Will anyone merge this? |
…-performance Fewer github api calls for performance
|
Thanks, this will be included in the next release. |
|
Thanks @skottler for merging & maintaining this plugin! |
|
Could we make a new release for this? We are having the issue right now. |
|
+1 still having issue |
On jenkins instances with a large number of jobs the previous implementation got very slow because it made a github api call per job & did not do a great job of caching. This pull reduces the chatter with the github api by asking which repos the user has access to and caching the results.
There are 2 big changes in this pull
The algorithm now builds and caches a list of "my repositories"
It makes these api calls to github:
/api/v3/user/repos/api/v3/user/orgs. For each organization/api/v3/orgs/XXX/reposBecause of pagination it sometimes makes several calls to get the full list of repos.
Cache list of public repositories for read-only access.
For this it still makes an api call per repository but caches the results for all users.
On my jenkins instance with ~400 projects this has improved performance from unusable (minutes to load the home page) to pretty fast (~4 seconds as it builds the cache and faster after that).