Changing amount of replicas for swarm produces more changes than expected

General Information

docker-java version: 3.1.0-SNAPSHOT
docker-compose version: docker-compose version 1.17.1, build 6d101fb
docker version: Docker version 17.09.0-ce, build afdb6d4

Problem

Iterating over my services received via DockerClient#listServicesCmd() and then scaling them up with

/* some example code to explain - variable "service" is a service from the iteration and "dockerClient" is an instance of  DockerClient */
ServiceSpec serviceSpec = service.getSpec();
ServiceModeConfig serviceModeConfig = service.getSpec().getMode();
ServiceReplicatedModeOptions serviceReplicatedModeOptions = service.getSpec().getMode().getReplicated().withReplicas(2);
serviceModeConfig = serviceModeConfig.withReplicated(serviceReplicatedModeOptions);
serviceSpec = serviceSpec.withMode(serviceModeConfig);
dockerClient.updateServiceCmd(service.getId(), serviceSpec).withVersion(service.getVersion().getIndex()).exec();

produces an interesting diff:

  "Spec": {
    "Name": "<NAME>",
    "Labels": {
      "com.docker.stack.image": "<IMAGE_NAMESPACE>/<IMAGE_NAME>",
      "com.docker.stack.namespace": "<NAMESPACE>"
    },
    "TaskTemplate": {
      "ContainerSpec": {
        "Image": "<IMAGE_NAMESPACE>/<IMAGE_NAME>:latest",
        "Labels": {
          "com.docker.stack.namespace": "<NAMESPACE>"
        },
        "Env": [],
-       "Privileges": {
-         "CredentialSpec": null,
-         "SELinuxContext": null
-        },
        "TTY": true,
-       "OpenStdin": true,
-       "StopGracePeriod": 60000000000
+       "StopGracePeriod": 10000000000,
+       "DNSConfig": {}
        },
        "Resources": {
          "Limits": {
            "MemoryBytes": 536870912
          }
        },
        "RestartPolicy": {
          "Condition": "none",
          "Delay": 5000000000,
          "MaxAttempts": 3,
          "Window": 20000000000
        },
        "Placement": {},
-       "Networks": [{
-         "Target": "fzmfcwseizy9l7xtbbeq8fuh8",
-         "Aliases": [
-           "hub"
-         ]
-       }],
        "ForceUpdate": 0,
        "Runtime": "container"
      },
      "Mode": {
        "Replicated": {
-         "Replicas": 0
+         "Replicas": 2
        }
+     },
+     "UpdateConfig": {
+       "Parallelism": 1,
+       "FailureAction": "pause",
+       "Monitor": 5000000000,
+       "MaxFailureRatio": 0,
+       "Order": "stop-first"
+     },
+     "RollbackConfig": {
+       "Parallelism": 1,
+       "FailureAction": "pause",
+       "Monitor": 5000000000,
+       "MaxFailureRatio": 0,
+       "Order": "stop-first"
      },
      "EndpointSpec": {
       "Mode": "vip"
      }
    }
-   "Endpoint": {
-     "Spec": {
-        "Mode": "vip"
-     }
-   }

I thought, just the amount of replicas ("Replicas": 0 to "Replicas": 2) would change. Why are there so many other changes? For me, especially the missing network part is a problem. Maybe someone has an idea why this is happening or has proper working code for scaling the amount of a service's replicas.

I suspect the ServiceSpec serviceSpec = service.getSpec(); can you try use a clean spec for update ServiceSpec serviceSpec = new ServiceSpec(); ?

Just using new ServiceSpec(); as following

ServiceSpec serviceSpec = new ServiceSpec();
ServiceModeConfig serviceModeConfig = service.getSpec().getMode();
ServiceReplicatedModeOptions serviceReplicatedModeOptions = service.getSpec().getMode().getReplicated().withReplicas(2);
serviceModeConfig = serviceModeConfig.withReplicated(serviceReplicatedModeOptions);
serviceSpec = serviceSpec.withMode(serviceModeConfig);
dockerClient.updateServiceCmd(service.getId(), serviceSpec).withVersion(service.getVersion().getIndex()).exec();

results in a com.github.dockerjava.api.exception.BadRequestException: {"message":"rpc error: code = InvalidArgument desc = TaskSpec: missing runtime"}.

General question in between: How does updating a service work? Is just the data being sent updated or is it completely overriding those service setting (resulting in the need to include everything like those network settings in this ServiceSpec)?

Do you have a quick code snipped for me how to update the replica amount of a service keeping its original settings?

Actually it is not documented https://docs.docker.com/engine/api/v1.30/#operation/ServiceUpdate
you has to decipher it yourself.

Edit /etc/docker/daemon.json to add "debug": true, then update service using docker cli, and monitor /var/log/message for the request body, sample
I tried
docker service update --replicate 3 worker and got below request

Dec  5 10:52:55 x dockerd: time="2017-12-05T10:52:55.307104030-05:00" level=debug msg="form data: {\"EndpointSpec\":{\"Mode\":\"vip\"},\"Labels\":{},\"Mode\":{\"Replicated\":{\"Replicas\":3}},
\"Name\":\"worker\",\"TaskTemplate\":{\"ContainerSpec\":{\"DNSConfig\":{},\"Image\":\"busybox:latest@sha256:bbc3a03235220b170ba48a157dd097dd1379299370e1ed99ce976df0355d24f0\"},\"ForceUpdate\":0,\"Placement\":{\"Platforms\":[{\"Architecture\":\"amd64\",\"OS\":\"linux\"},{\"OS\":\"linux\"},{\"OS\":\"linux\"},{\"OS\":\"linux\"},{\"Architecture\":\"arm64\",\"OS\":\"linux\"},{\"Architecture\":\"386\",\"OS\":\"linux\"},{\"Architecture\":\"ppc64le\",\"OS\":\"linux\"},{\"Architecture\":\"s390x\",\"OS\":\"linux\"}]},\"Resources\":{\"Limits\":{},\"Reservations\":{}},\"Runtime\":\"container\"}}"

I just did some further testing - especially trying to get rid of the the com.github.dockerjava.api.exception.BadRequestException: {"message":"rpc error: code = InvalidArgument desc = TaskSpec: missing runtime"} error mentioned above to be able to use a clean ServiceSpec and found out that the "Runtime": "container" part is completely missing in the ContainerSpec model.
Is one of you guys able to add this to the model?

it seems runtime was added in v1.30, I think it is optional, tested with below code

        ServiceSpec serviceSpec = new ServiceSpec();
        TaskSpec taskSpec = new TaskSpec().withContainerSpec(
                new ContainerSpec().withImage("busybox").withArgs(Arrays.asList("sleep", "3600")));
        serviceSpec.withMode(new ServiceModeConfig()
                .withReplicated(new ServiceReplicatedModeOptions()
                        .withReplicas(5))).withTaskTemplate(taskSpec).withName("worker");
        getClient().updateServiceCmd("worker", serviceSpec)
                .withVersion(4464L).exec();

you need image/command/name and version for updating

Thanks, this works for replication with a clean ServiceSpec. But still those configuration changes occur.

Seems like a lot of things are getting nulled, when using docker-java compared to CLI...

Here is the debug log output when using your code snippet… Any ideas on this?

Dec 14 09:11:28 htznr1 dockerd[665]: time="2017-12-14T09:11:28.751373103+01:00" level=debug msg="form data: {\"EndpointSpec\":null,\"Labels\":null,\"Mode\":{\"Global\":null,\"Replicated\":{\"Replicas\":5},\"mode\":\"REPLICATED\"},\"Name\":\"worker\",\"Networks\":null,\"TaskTemplate\":{\"ContainerSpec\":{\"Args\":[\"sleep\",\"3600\"],\"Command\":null,\"Dir\":null,\"Duration\":null,\"Env\":null,\"Groups\":null,\"Image\":\"busybox\",\"Labels\":null,\"Mounts\":null,\"TTY\":null,\"User\":null},\"ForceUpdate\":null,\"LogDriver\":null,\"Placement\":null,\"Resources\":null,\"RestartPolicy\":null,\"Runtime\":null},\"UpdateConfig\":null}"

I fixed the null string issue by adding Include.NON_NULL annotation (#966)

Thanks for changing this.

I just had some closer look on the code and found out that ContainerSpec Model is missing following values:

Hostname
OpenStdin
Readonly
StopSignal
StopGracePeriod
Healthcheck
Hosts
DNSConfig
Secrets
Configs

TaskSpec Model is missing

Networks (which might be the solution for my original issue)

ServiceSpec Model is missing

RollbackConfig

Include.NON_NULL annotation still missing on:

Thanks for the finding, I will update the data model when I am free, maybe next week.

@fengxx

@fengxx That would be awesome 👍

@fengxx

Hey @fengxx
Hope you had a nice Christmas time. Any update on this issue?

@fengxx

Works perfectly, @fengxx - thanks so much!

@sebastianbinder

@sebastianbinder I encountered the same question as you working with the docker API. Do I have to specify everything as upon service creation or is it possible updating only a single thing of the service, for example the name?

Navigation Menu

Repository navigation

Changing amount of replicas for swarm produces more changes than expected #958

General Information

Problem

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Changing amount of replicas for swarm produces more changes than expected #958

Description

General Information

Problem

Activity

fengxx commented on Dec 5, 2017

sebastianbinder commented on Dec 5, 2017

fengxx commented on Dec 5, 2017

sebastianbinder commented on Dec 8, 2017

fengxx commented on Dec 9, 2017

sebastianbinder commented on Dec 14, 2017

fengxx commented on Dec 15, 2017

sebastianbinder commented on Dec 15, 2017

fengxx commented on Dec 19, 2017

sebastianbinder commented on Dec 19, 2017

sebastianbinder commented on Jan 2, 2018

sebastianbinder commented on Jan 9, 2018

jaschaio commented on Apr 19, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions