Repository navigation
Add support for remote-import API Endpoint #2260
Description
Description of the problem, including code/CLI snippet
Hello Together,
GitLab supports the import from a remote storage, additionally to the export to a remote storage. This is really helpful, if you migrate a lot of big repositories to another GitLab Instance, it's increase the speed and also reduce the API calls and scripts complexity.
https://docs.gitlab.com/ee/api/project_import_export.html#import-a-file-from-a-remote-object-storage
Expected Behavior
It would be great if gitlab-python would support this endpoint, so additionally to the import_project function, we could also use the import_project_from_remote function.
Actual Behavior
At the moment, it is not possible to import a repository from a remote storage via import_project function.
Specifications
- python-gitlab version: 3.8.1
- API version you are using (v3/v4): v4
- Gitlab server version (or gitlab.com): 15.2.3
Activity
abhishekmsingh commentedon Oct 29, 2022
@nejch @JohnVillalovos trying to implement this. Can you please assign this to me?
I am thinking to create an FileImportMixin which will be used by
I can add the unit tests too. Just worried about functionally testing these esp S3 one. If you have any ideas on how to test, please let me know. Will most probably create a mock server to mock file responses.
nejch commentedon Oct 30, 2022
@abhiandthetruth thanks for looking into this! Sorry I was away and still catching up on PRs.
I wouldn't worry too much about functional tests for this. I'd just add unit tests for each method as needed in
tests/unit/with the mocked responses. The remote imports do a lot less heavy lifting on our side so we just want to make sure the request parameters are sent correctly by using one of the matchers in responses (https://github.com/getsentry/responses#matching-requests). We have some similar use in our code. It's probably overkill to add a local MinIO just to test this ;)I guess a
ProjectImportMixinor something similar might make sense if you can get enough common code in, although it will be reused by the methods in the same class, rather than as a real mixin.One more thing, we now usually try to match method names to the API URLs closely. Although in this case GitLab decided to use kebab-case for some reason, the closest method names would be
remote_importandremote_import_s3.abhishekmsingh commentedon Oct 31, 2022
@nejch thanks for the elaborate comment!
I have written unit tests and non mixin code in the PR I have raised for remote-import. Need to do the same to remote-import-s3. Will fix the naming of function according to your recommendation, thanks.
On the topic of functional testing, I am exploring on asserting import_errors field with certain value which we may receive on sending a non accessible URL for remote import. If the error message clearly denotes that Gitlab tried to fetch the resource, and we can assert this, I think it is enough of a functional test, because the fetching the resource successfully and instantiating the project is then Gitlab's code not ours. Will update with findings.
nejch commentedon Oct 31, 2022
Ok @abhiandthetruth thanks a lot! And just ping when ready for review. That's a simple approach as well with the exceptions, yes. You can use
pytest.raises()there and match on the message.abhishekmsingh commentedon Oct 31, 2022
@nejch thanks used pytest.raises to test the exception. Raised PR