Skip to content

Improve status check for pulling an image through docker-swarm - #337

Closed
hawky-4s- wants to merge 1 commit into
docker-java:masterfrom
hawky-4s-:swarm-pull
Closed

Improve status check for pulling an image through docker-swarm#337
hawky-4s- wants to merge 1 commit into
docker-java:masterfrom
hawky-4s-:swarm-pull

Conversation

@hawky-4s-

@hawky-4s- hawky-4s- commented Oct 6, 2015

Copy link
Copy Markdown

This improves the PullImageCmd for docker-swarm because docker-swarm returns a different status line than the normal docker client.
This fix only checks if the last status line returns successfully when pulling through docker-swarm, which means that the pull on the last node was successful. It does not mean the image pull on all swarm nodes went ok.
I do not know how to check this, because to be fully swarm compatible it would be needed to read the last x lines where x is the number of swarm nodes and check them all if the result is successful.


This change is Reviewable

@marcuslinke

Copy link
Copy Markdown
Contributor

I do not know how to check this, because to be fully swarm compatible it would be needed to read the last x lines where x is the number of swarm nodes and check them all if the result is successful.

This should be possible by inherit PullImageResultCallback and override awaitSuccess() then. WDYT?

@hawky-4s-

Copy link
Copy Markdown
Author

I followed your suggestion but only changed the implementation of the PullImageResultCallback. It is now able to differentiate between Docker Swarm and Docker Client and adjust it's behavior accordingly. I did this so it is transparent to the user of the api if he's talking to a swarm or client without knowing it in advance.

WDYT?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How this should work. The first item that doesn't indicate a succesfull pull will lead to pull error.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swarm works like so. When you issue a pull cmd, it returns following responses as PullResponseItem for each node in the swarm:
21:38:57.269 DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4...,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci7.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
When a pull is successfully for a swarm node following PullResponseItem is returned:
21:39:04.513 DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4... : downloaded,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci3.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
There are only these two responses for each swarm node when a pull request is successfull.

My code will first collect all PullResponseItems ordered by node id as key. So only the latest PullResponseItem of each node will be stored, previous ones overwritten.
When the PullImageCmd is finished and awaitSuccess method continues to evaluate the success of the pull, it will check the last result of each node. When one of those results indicates a failed pull, it will throw an exception with all nodes where the pull failed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Now i got it. Thanks!

@KostyaSha

Copy link
Copy Markdown
Member

Interesting, how this all can be tested... DinD?

@marcuslinke

Copy link
Copy Markdown
Contributor

Shouldn't it be possible to start a swarm manager and a swarm node as containers at the docker host and handle this single docker host as a swarm? This would be the easiest solution...

@KostyaSha

Copy link
Copy Markdown
Member

Doesn't swarm do pull on two hosts at once?

@hawky-4s-

Copy link
Copy Markdown
Author

You can start a swarm manager and point it to the local docker server like
docker run -it --rm -p 2377:2375 swarm manage nodes://<node_ip>:2375
Then configure <node_ip>:2377 as endpoint for docker-java.

@hawky-4s-

Copy link
Copy Markdown
Author

@KostyaSha
Swarm pulls on as many nodes as you have concurrently.

@KostyaSha

Copy link
Copy Markdown
Member

Swarm pulls on as many nodes as you have concurrently.

It what i mean about tests complexity. (Just curious)

@hawky-4s-

Copy link
Copy Markdown
Author

Yes, but you could also just start a single swarm master configured to use the same node where it runs as swarm member with the command I provided above.

@hawky-4s-

Copy link
Copy Markdown
Author

Example debug output from running the pull cmd against a swarm using docker-java:

DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4...,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci7.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4...,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci3.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4...,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci6.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4...,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci4.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4... : downloaded,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci4.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4... : downloaded,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci3.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4... : downloaded,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci7.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]
DEBUG c.g.d.c.c.PullImageResultCallback - PullResponseItem[stream=<null>,status=Pulling ci1.camunda.loc:5000/camunda-ci-postgresql:9.4... : downloaded,progressDetail=ResponseItem.ProgressDetail[current=0,total=0,start=0],progress=<null>,id=ci6.camunda.loc,from=<null>,time=0,errorDetail=<null>,error=<null>]

@KostyaSha

Copy link
Copy Markdown
Member

#711 on the way, then your changes should be picked.

@KostyaSha

Copy link
Copy Markdown
Member

Trying to pick into swarm branch. Help is welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants