-
Notifications
You must be signed in to change notification settings - Fork 4
[LIB-837] - add custom sockets functionality to the LIB. #228
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
12cda74
460006f
5eca906
c0f75cb
e108b5a
5eab464
1afcd9f
48eb08e
4e2ee33
fe52e7a
7bc974a
bc71ad1
327a3fc
d50344b
c2ffd3a
42e4373
04c5ef6
9f5adba
69f3a11
3d1f4dd
d982fd0
8662e42
0864163
9246b98
31c51b9
14f6ff7
2372986
515e50b
16795dd
b8c252f
3cf2465
ef39be6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
…r file, add possibility to recheck and update custom socket; add possibility to remove endpoints and dependencies;
- Loading branch information
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,214 @@ | ||
| .. _custom-sockets: | ||
|
|
||
| ========================= | ||
| Custom Sockets in Syncano | ||
| ========================= | ||
|
|
||
| ``Syncano`` provides possibility of creating the custom sockets. It means that there's a possibility | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. creating |
||
| to define a very specific endpoints in syncano application and use them as normal api calls. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we should use API not api to be more correct here (with it being an acronym and all). Same for a few other uses below. |
||
| Currently custom sockets allow only one dependency - script. This mean that on the backend side | ||
| each time the api is called - the script is executed and result from this script is returned as a result of the | ||
| api call. | ||
|
|
||
| Creating a custom socket | ||
| ------------------------ | ||
|
|
||
| There are two methods of creating the custom socket. First: use the helpers objects defined in Python Libray. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's still the one and only method. Library simply provides some helpers for it. Indicating that it is separate somehow and is overall a different method - can lead to some confusion. Even I'm confused after reading this ;) especially when it says about describing raw format below and then goes on to using python helpers. I think we can just skip this paragraph and move on to an example about how to actually create it |
||
| Second: use the raw format - this is described below. | ||
|
|
||
| To create a custom socket follow the steps:: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. follow |
||
|
|
||
| import syncano | ||
| from syncano.models import CustomSocket, Endpoint, ScriptCall, ScriptDependency, RuntimeChoices | ||
| from syncano.connection import Connection | ||
|
|
||
| custom_socket = CustomSocket(name='my_custom_socket') # this will create an object in place (do api call) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe add comments about actual steps numbered. |
||
|
|
||
| # define endpoints | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| my_endpoint = Endpoint(name='my_endpoint') # again - no api call here | ||
| my_endpoint.add_call(ScriptCall(name='custom_script'), methods=['GET']) | ||
| my_endpoint.add_call(ScriptCall(name='another_custom_script'), methods=['POST']) | ||
|
|
||
| # explanation for the above lines: | ||
| # The endpoint will be seen under `my_endpoint` name: | ||
| # On this syncano api endpoint the above endpoint will be called (after custom socket creation) | ||
| # <host>://<api_version>/instances/<instance_name>/endpoints/sockets/my_endpoint/ | ||
| # On this syncano api endpoint the details of the defined endpoint will be returned | ||
| # <host>://<api_version>/instances/<instance_name>/sockets/my_custom_socket/endpoints/my_endpoint/ | ||
| # For the above endpoint - the two calls are defined, one uses GET method - the custom_script will be executed | ||
| # there, second uses the POST method and then the another_custom_script will be called; | ||
| # Currently only script are available for calls; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't really understand above comments. Could someone rewrite them in other words in here?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Every script consists of dependencies (which are simply checked and created when needed) and endpoints. we create one endpoint called my_endpoint and we define that: when someone does GET my_endpoint -> call script with endpoint=custom_script. When someone does POST my_endpoint, call script with endpoint=another_custom_script. So we can define separate logic/bindings based on a HTTP method used. Now endpoints live in two places in API. Second place defines API for custom sockets in general. It's a little verbose but hopefully I explained it enough? |
||
|
|
||
| # After the creation of the endpoint, add them to custom_socket: | ||
| custom_socket.add_endpoint(my_endpoint) | ||
|
|
||
| # define dependency now; | ||
| # using a new script - defining new source code; | ||
| custom_socket.add_dependency( | ||
| ScriptDependency( | ||
| Script( | ||
| label='custom_script', | ||
| runtime_name=RuntimeChoices.PYTHON_V5_0, | ||
| source='print("custom_script")' | ||
| ) | ||
| ) | ||
| ) | ||
| # using a existing script: | ||
| another_custom_script = Script.please.get(id=2) | ||
| custom_socket.add_dependency( | ||
| ScriptDependency( | ||
| another_custom_script | ||
| ) | ||
| ) | ||
|
|
||
| # now it is time to publish custom_socket; | ||
| custom_socket.publish() # this will do an api call and will create script; | ||
|
|
||
| Some time is needed to setup the environment for this custom socket. | ||
| There is possibility to check the custom socket status:: | ||
|
|
||
| print(custom_socket.status) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it will always be "checking" after creation/update so maybe here first .reload() then print |
||
| # and | ||
| print(custom_socket.status_info) | ||
|
|
||
| # to reload object (read it again from syncano api) use: | ||
| custom_socket.reload() | ||
|
|
||
|
|
||
|
|
||
| Updating the custom socket | ||
| -------------------------- | ||
|
|
||
| To update custom socket, use:: | ||
|
|
||
| custom_socket = CustomSocket.please.get(name='my_custom_socket') | ||
|
|
||
| custom_socket.remove_endpoint(endpoint_name='my_endpoint') | ||
| custom_socket.remove_dependency(dependency_name='custom_script') | ||
|
|
||
| # or add new: | ||
|
|
||
| custom_socket.add_endpoint(new_endpoint) # see above code for endpoint examples; | ||
| custom_socket.add_dependency(new_dependency) # see above code for dependency examples; | ||
|
|
||
| custom_socket.update() | ||
|
|
||
|
|
||
| Running the custom socket | ||
| ------------------------- | ||
|
|
||
| To run custom socket use:: | ||
|
|
||
| # this will run the my_endpoint - and call the custom_script (method is GET); | ||
| result = custom_socket.run(method='GET', endpoint_name='my_endpoint') | ||
|
|
||
|
|
||
| Read all endpoints | ||
| ------------------ | ||
|
|
||
| To get the all defined endpoints in custom socket run:: | ||
|
|
||
| endpoints = custom_socket.get_endpoints() | ||
|
|
||
| for endpoint in endpoints: | ||
| print(endpoint.name) | ||
| print(endpoint.calls) | ||
|
|
||
| To run particular endpoint:: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To run a particular endpoint |
||
|
|
||
| endpoint.run(method='GET') | ||
| # or: | ||
| endpoint.run(method='POST', data={'name': 'test_name'}) | ||
|
|
||
| The data will be passed to the api call in the request body. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What about getting list of all socket endpoints (not bound to specific socket)? It would also be useful if user could run some custom endpoint without getting anything at all. This may end up as how it will be used the most in apps.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ++ |
||
|
|
||
| Custom sockets endpoints | ||
| ------------------------ | ||
|
|
||
| Each custom socket is created from at least one endpoint. The endpoint is characterized by name and | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not from. Also, instead of characterized in these whole docs, defined may be a little more commonly used and fitting |
||
| defined calls. Calls is characterized by name and methods. The name is a identification for dependency, eg. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. and a list of calls. Each call is defined by a name and a list of methods. |
||
| if it's equal to 'my_script' - the Script with label 'my_script' will be used (if exist and the source match), | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not exactly. |
||
| or new one will be created. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. or a new one |
||
| There's a special wildcard method: `methods=['*']` - this mean that any request with | ||
| any method will be executed in this endpoint. | ||
|
|
||
| To add endpoint to the custom_socket use:: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To add an endpoint |
||
|
|
||
| my_endpoint = Endpoint(name='my_endpoint') # again - no api call here | ||
| my_endpoint.add_call(ScriptCall(name='custom_script'), methods=['GET']) | ||
| my_endpoint.add_call(ScriptCall(name='another_custom_script'), methods=['POST']) | ||
|
|
||
| custom_socket.add_endpoint(my_endpoint) | ||
|
|
||
| Custom socket dependency | ||
| ------------------------ | ||
|
|
||
| Each custom socket has dependency - this is a meta information for endpoint: which resource | ||
| should be used to return the api call results. The dependencies are bind to the endpoints call objects. | ||
| Currently supported dependency in only script. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ordering is strange ;)
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yoda style ;) |
||
|
|
||
| **Using new script** | ||
|
|
||
| :: | ||
|
|
||
| custom_socket.add_dependency( | ||
| ScriptDependency( | ||
| Script( | ||
| label='custom_script', | ||
| runtime_name=RuntimeChoices.PYTHON_V5_0, | ||
| source='print("custom_script")' | ||
| ) | ||
| ) | ||
| ) | ||
|
|
||
|
|
||
| **Using defined script** | ||
|
|
||
| :: | ||
|
|
||
| another_custom_script = Script.please.get(id=2) | ||
| custom_socket.add_dependency( | ||
| ScriptDependency( | ||
| another_custom_script | ||
| ) | ||
| ) | ||
|
|
||
|
|
||
| Custom socket recheck | ||
| --------------------- | ||
|
|
||
| The creation of the socket can fail - this happen, eg. when endpoint name is already taken by another | ||
| custom socket. To check the statuses use:: | ||
|
|
||
| print(custom_socket.status) | ||
| print(custom_socket.status_info) | ||
|
|
||
| There is a possibility to re-check socket - this mean that if conditions are met - the socket will be | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also, dependencies can be mistakenly deleted. So re-check is useful here to well, "reinstall" everything that is missing. The socket itself is not actually created again as it's already there. It just checks endpoints and dependencies again. |
||
| `created` again and available to use - if not the error will be returned in status field. | ||
|
|
||
| Custom socket - raw format | ||
| -------------------------- | ||
|
|
||
| There is a possibility to create a custom socket from the raw JSON format:: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe something like: |
||
|
|
||
| CustomSocket.please.create( | ||
| name='my_custom_socket_3', | ||
| endpoints={ | ||
| "my_endpoint_3": { | ||
| "calls": | ||
| [ | ||
| {"type": "script", "name": "my_script_3", "methods": ["POST"]} | ||
| ] | ||
| } | ||
| }, | ||
| dependencies=[ | ||
| { | ||
| "type": "script", | ||
| "runtime_name": "python_library_v5.0", | ||
| "name": "my_script_3", | ||
| "source": "print(3)" | ||
| } | ||
| ] | ||
| ) | ||
|
|
||
| The disadvantage of this method is that - the JSON internal structure must be known by developer. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,6 +19,7 @@ Contents: | |
|
|
||
| getting_started | ||
| interacting | ||
| custom_sockets | ||
| refs/syncano | ||
|
|
||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| syncano.models.custom_sockets | ||
| ============================= | ||
|
|
||
| .. automodule:: syncano.models.custom_sockets | ||
| :members: | ||
| :undoc-members: | ||
| :show-inheritance: |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| syncano.models.custom_sockets_utils | ||
| =================================== | ||
|
|
||
| .. automodule:: syncano.models.custom_sockets_utils | ||
| :members: | ||
| :undoc-members: | ||
| :show-inheritance: |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| syncano.models.geo | ||
| ================== | ||
|
|
||
| .. automodule:: syncano.models.geo | ||
| :members: | ||
| :undoc-members: | ||
| :show-inheritance: |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| syncano.models.hosting | ||
| ====================== | ||
|
|
||
| .. automodule:: syncano.models.hosting | ||
| :members: | ||
| :undoc-members: | ||
| :show-inheritance: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lifcio We need help from native here ;)