You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Is it possible to change the proxyUrl with the beforeRequest hook? #77
As the title suggests, is there any way to set the proxyUrl through the beforeRequest hook?
Ideally I'd like to do this on the beforeRetry hook but we cannot access the options object from there.
It should sort of act like a proxy rotation system - if the request fails, call a function which returns a new proxy and set it as the new proxyUrl.
Setting error.options.context.proxyUrl to undefined, null or an empty string in the beforeRetry hook still makes a retry request with the originally set proxyUrl.
@szmarczak We'll appreciate any suggestions or unit test results from you. Thanks.
as I understand, if we extends the got-scraping client our hooks will be called after got-scraping ones, so the proxy hook is called before we can change the proxyUrl
Activity
szmarczak commentedon Sep 11, 2022
It should be possible to do that, I'll add a test for this. You should be able to override
error.options.context.proxyUrl.VoidMonk commentedon Jul 7, 2023
Any way to remove the set proxy before a retry?
Setting
error.options.context.proxyUrlto undefined, null or an empty string in thebeforeRetryhook still makes a retry request with the originally setproxyUrl.@szmarczak We'll appreciate any suggestions or unit test results from you. Thanks.
yovanoc commentedon Sep 23, 2023
Would like to know this too, thanks
yovanoc commentedon Jan 18, 2024
as I understand, if we extends the got-scraping client our hooks will be called after got-scraping ones, so the proxy hook is called before we can change the proxyUrl