Skip to content

Commit dc88d19

Browse files
olim7tolim7t
committed
Merge branch '2.1' into 3.0
2 parents dba1d6a + b43a084 commit dc88d19

7 files changed

Lines changed: 351 additions & 57 deletions

File tree

‎driver-core/src/main/java/com/datastax/driver/core/policies/LatencyAwarePolicy.java‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -537,18 +537,18 @@ public TimestampedAverage getCurrentAverage() {
537537
*/
538538
public static class Builder {
539539

540-
private static final double DEFAULT_EXCLUSION_THRESHOLD = 2.0;
541-
private static final long DEFAULT_SCALE = TimeUnit.MILLISECONDS.toNanos(100);
542-
private static final long DEFAULT_RETRY_PERIOD = TimeUnit.SECONDS.toNanos(10);
543-
private static final long DEFAULT_UPDATE_RATE = TimeUnit.MILLISECONDS.toNanos(100);
544-
private static final int DEFAULT_MIN_MEASURE = 50;
540+
public static final double DEFAULT_EXCLUSION_THRESHOLD = 2.0;
541+
public static final long DEFAULT_SCALE_NANOS = TimeUnit.MILLISECONDS.toNanos(100);
542+
public static final long DEFAULT_RETRY_PERIOD_NANOS = TimeUnit.SECONDS.toNanos(10);
543+
public static final long DEFAULT_UPDATE_RATE_NANOS = TimeUnit.MILLISECONDS.toNanos(100);
544+
public static final int DEFAULT_MIN_MEASURE = 50;
545545

546546
private final LoadBalancingPolicy childPolicy;
547547

548548
private double exclusionThreshold = DEFAULT_EXCLUSION_THRESHOLD;
549-
private long scale = DEFAULT_SCALE;
550-
private long retryPeriod = DEFAULT_RETRY_PERIOD;
551-
private long updateRate = DEFAULT_UPDATE_RATE;
549+
private long scale = DEFAULT_SCALE_NANOS;
550+
private long retryPeriod = DEFAULT_RETRY_PERIOD_NANOS;
551+
private long updateRate = DEFAULT_UPDATE_RATE_NANOS;
552552
private int minMeasure = DEFAULT_MIN_MEASURE;
553553

554554
/**

‎manual/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ The other aspects that you can configure on the `Cluster` are:
7474
* [query options][QueryOptions];
7575
* [reconnections](reconnection/);
7676
* [retries](retries/);
77-
* [socket options][SocketOptions];
77+
* [socket options](socket_options/);
7878
* [SSL](ssl/);
7979
* [speculative executions](speculative_execution/);
8080
* [query timestamps](query_timestamps/).

‎manual/socket_options/README.md‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
## Socket options
2+
3+
[SocketOptions] controls various low-level parameters related to TCP connections between the driver and Cassandra.
4+
5+
You can provide these when initializing the cluster:
6+
7+
```java
8+
Cluster cluster = Cluster.builder()
9+
.addContactPoint("127.0.0.1")
10+
.withSocketOptions(
11+
new SocketOptions()
12+
.setConnectTimeoutMillis(2000))
13+
.build();
14+
```
15+
16+
If you don't, the driver uses a default configuration (see the javadocs of the setter methods of [SocketOptions] for the
17+
default values).
18+
19+
You can retrieve and change the options at runtime:
20+
21+
```java
22+
SocketOptions socketOptions = cluster.getConfiguration().getSocketOptions();
23+
socketOptions.setConnectTimeoutMillis(3000);
24+
```
25+
26+
* changes to the [read timeout](#driver-read-timeout) will be taken into account for future request executions;
27+
* changes to any other option will be taken into account for future connections (connections that were already opened at
28+
the time of the change are unaffected, they keep the old values).
29+
30+
### TCP options
31+
32+
[setConnectTimeoutMillis] defines how long the driver waits to establish a new connection to a Cassandra node before
33+
giving up.
34+
35+
Other options control the usual low-level TCP parameters (refer to their individual javadoc for details):
36+
[setKeepAlive], [setReceiveBufferSize], [setReuseAddress], [setSendBufferSize], [setSoLinger], [setTcpNoDelay]. Most of
37+
these options are not set explicitly by the driver, so they get the default value from the underlying TCP transport.
38+
One exception is `setTcpNoDelay`, which is forced to `true` (meaning that Nagle's algorithm is *disabled* for driver
39+
connections).
40+
41+
### Driver read timeout
42+
43+
[setReadTimeoutMillis] controls how long the driver waits for a response *from a given Cassandra node* before
44+
considering it unresponsive.
45+
46+
Cassandra normally provides a guaranteed response time for each type of query, as shown by these parameters in
47+
`cassandra.yaml` (here with their default values):
48+
49+
```yaml
50+
counter_write_request_timeout_in_ms: 5000
51+
range_request_timeout_in_ms: 10000
52+
read_request_timeout_in_ms: 5000
53+
request_timeout_in_ms: 10000
54+
truncate_request_timeout_in_ms: 60000
55+
write_request_timeout_in_ms: 2000
56+
```
57+
58+
The goal of `setReadTimeoutMillis` is to give up on a node if it took longer than these thresholds to reply, on the
59+
assumption that there's probably something wrong with it. Therefore it should be set **higher than the server-side
60+
timeouts**.
61+
62+
The default value is **12 seconds**. For truncate queries (where the server timeout is 60 seconds) or aggregate queries
63+
(where `read_request_timeout_in_ms` applies per page of processed data, not to the whole query), you can increase the
64+
timeout on a specific statement:
65+
66+
```java
67+
session.execute(
68+
new SimpleStatement("TRUNCATE tmp").setReadTimeoutMillis(65000));
69+
```
70+
71+
Do not set the read timeout too low, or the driver might give up on requests that had a chance of succeeding.
72+
73+
If the timeout is reached, the driver will receive an [OperationTimedOutException], and invoke [onRequestError] on the
74+
[retry policy](../retries/) to decide what to do (the default is to retry on the next node in the
75+
[query plan](../load_balancing/#query-plan)).
76+
77+
#### Limiting overall query time
78+
79+
It should be clear by now that `setReadTimeoutMillis` is *per node*, not per query. If the driver retries on 4 different
80+
nodes, the overall execution time could theoretically be up to 4 times the read timeout. If you want a per query timeout,
81+
use the following pattern:
82+
83+
```java
84+
import com.google.common.base.Throwables;
85+
86+
ResultSet execute(Statement statement, long timeout, TimeUnit unit)
87+
throws InterruptedException, TimeoutException {
88+
ResultSetFuture future = session.executeAsync(statement);
89+
try {
90+
return future.get(timeout, unit);
91+
} catch (ExecutionException e) {
92+
throw Throwables.propagate(e.getCause());
93+
} catch (TimeoutException e) {
94+
future.cancel(true);
95+
throw e;
96+
}
97+
}
98+
```
99+
100+
A complementary approach is to enable [speculative executions](../speculative_execution/), to have the driver query
101+
multiple nodes in parallel. This way you won't have to wait for the full timeout if the first node is unresponsive.
102+
103+
#### Driver read timeout vs. server read timeout
104+
105+
Unfortunately, the term "read timeout" clashes with another concept that is not directly related: a Cassandra node may
106+
reply with a `Read_timeout` error when it didn't hear back from enough replicas during a read query.
107+
108+
To clarify:
109+
110+
* **driver read timeout:** the driver did not receive any response from the current coordinator within
111+
`SocketOptions.setReadTimeoutMillis`. It invokes [onRequestError] on the [retry policy](../retries/) with an
112+
[OperationTimedOutException] to decide what to do.
113+
* **server read timeout:** the driver *did* receive a response, but that response indicates that the coordinator timed
114+
out while waiting for other replicas. It invokes [onReadTimeout] on the [retry policy](../retries/) to decide what to
115+
do.
116+
117+
We might rename `SocketOptions.setReadTimeoutMillis` in a future version to clear up any confusion.
118+
119+
[SocketOptions]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html
120+
[setReadTimeoutMillis]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html#setReadTimeoutMillis-int-
121+
[setConnectTimeoutMillis]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html#setConnectTimeoutMillis-int-
122+
[setKeepAlive]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html#setKeepAlive-boolean-
123+
[setReceiveBufferSize]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html#setReceiveBufferSize-int-
124+
[setReuseAddress]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html#setReuseAddress-boolean-
125+
[setSendBufferSize]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html#setSendBufferSize-int-
126+
[setSoLinger]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html#setSoLinger-int-
127+
[setTcpNoDelay]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/SocketOptions.html#setTcpNoDelay-boolean-
128+
[onReadTimeout]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/policies/RetryPolicy.html#onReadTimeout-com.datastax.driver.core.Statement-com.datastax.driver.core.ConsistencyLevel-int-int-boolean-int-
129+
[onRequestError]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/policies/RetryPolicy.html#onRequestError-com.datastax.driver.core.Statement-com.datastax.driver.core.ConsistencyLevel-com.datastax.driver.core.exceptions.DriverException-int-
130+
[OperationTimedOutException]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/exceptions/OperationTimedOutException.html

‎manual/statements/.nav‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
simple
2+
prepared
3+
built
4+
batch

‎manual/statements/README.md‎

Lines changed: 9 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
## Statements
22

3-
To execute a query, you create a [Statement] instance and pass it to
4-
`Session#execute()`. The driver provides various implementations:
3+
To execute a query, you create a [Statement] instance and pass it to [Session#execute()][execute] or
4+
[Session#executeAsync][executeAsync]. The driver provides various implementations:
55

66
* [SimpleStatement](simple/): a simple implementation built directly from a
77
character string. Typically used for queries that are executed only
@@ -14,16 +14,6 @@ To execute a query, you create a [Statement] instance and pass it to
1414
* [BatchStatement](batch/): a statement that groups multiple statements to be
1515
executed as a batch.
1616

17-
`Session` also has a shortcut to build and execute a simple statement in
18-
a single call:
19-
20-
```java
21-
session.execute("select release_version from system.local");
22-
23-
// Is equivalent to:
24-
Statement s = new SimpleStatement("select release_version from system.local");
25-
session.execute(s);
26-
```
2717

2818
### Customizing execution
2919

@@ -42,9 +32,11 @@ If you use custom policies ([RetryPolicy], [LoadBalancingPolicy],
4232
properties that influence statement execution. To achieve this, you can
4333
wrap your statements in a custom [StatementWrapper] implementation.
4434

45-
[Statement]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/Statement.html
46-
[QueryBuilder]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/querybuilder/QueryBuilder.html
47-
[StatementWrapper]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/StatementWrapper.html
48-
[RetryPolicy]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/policies/RetryPolicy.html
49-
[LoadBalancingPolicy]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/policies/LoadBalancingPolicy.html
35+
[Statement]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/Statement.html
36+
[QueryBuilder]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/querybuilder/QueryBuilder.html
37+
[StatementWrapper]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/StatementWrapper.html
38+
[RetryPolicy]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/policies/RetryPolicy.html
39+
[LoadBalancingPolicy]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/policies/LoadBalancingPolicy.html
5040
[SpeculativeExecutionPolicy]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/policies/SpeculativeExecutionPolicy.html
41+
[execute]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/Session.html#execute-com.datastax.driver.core.Statement-
42+
[executeAsync]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/Session.html#executeAsync-com.datastax.driver.core.Statement-

‎manual/statements/prepared/README.md‎

Lines changed: 72 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,68 @@
11
## Prepared statements
22

3-
When Cassandra executes a query, the first thing it does is parse the
4-
query string to an internal representation. If the same query is used
5-
often, you can use a prepared statement, which allows Cassandra to cache
6-
that representation, and save time and resources for subsequent
7-
executions. Prepared statements are typically parameterized, using
8-
different values for each execution:
3+
Use [PreparedStatement] for queries that are executed multiple times in your application:
94

105
```java
116
PreparedStatement prepared = session.prepare(
127
"insert into product (sku, description) values (?, ?)");
138

14-
BoundStatement bound;
15-
16-
bound = prepared.bind("234827", "Mouse");
9+
BoundStatement bound = prepared.bind("234827", "Mouse");
1710
session.execute(bound);
1811

19-
bound = prepared.bind("987274", "Keyboard");
20-
session.execute(bound);
12+
session.execute(prepared.bind("987274", "Keyboard"));
2113
```
2214

23-
Statements should be be prepared only once. If you call `prepare`
24-
multiple times with the same query string, the driver will log a
25-
warning. So your application should cache the `PreparedStatement` object
26-
once it's been created (this can be as simple as storing it as a field
27-
in a DAO).
15+
When you prepare the statement, Cassandra will parse the query string, cache the result and return a unique identifier
16+
(the `PreparedStatement` object keeps an internal reference to that identifier):
17+
18+
```ditaa
19+
client driver Cassandra
20+
--+------------------------+----------------+------
21+
| | |
22+
| session.prepare(query) | |
23+
|----------------------->| |
24+
| | PREPARE(query) |
25+
| |--------------->|
26+
| | |
27+
| | |
28+
| | | - compute id
29+
| | | - parse query string
30+
| | | - cache (id, parsed)
31+
| | |
32+
| | PREPARED(id) |
33+
| |<---------------|
34+
| PreparedStatement(id) | |
35+
|<-----------------------| |
36+
```
37+
38+
When you bind and execute a prepared statement, the driver will only send the identifier, which allows Cassandra to
39+
skip the parsing phase:
40+
41+
```ditaa
42+
client driver Cassandra
43+
--+---------------------------------+---------------------+------
44+
| | |
45+
| session.execute(BoundStatement) | |
46+
|-------------------------------->| |
47+
| | EXECUTE(id, values) |
48+
| |-------------------->|
49+
| | |
50+
| | |
51+
| | | - get cache(id)
52+
| | | - execute query
53+
| | |
54+
| | ROWS |
55+
| |<--------------------|
56+
| | |
57+
|<--------------------------------| |
58+
```
59+
60+
61+
You should prepare only once, and cache the `PreparedStatement` in your application (it is thread-safe). If you call
62+
`prepare` multiple times with the same query string, the driver will log a warning.
63+
64+
If you execute a query only once, a prepared statement is inefficient because it requires two roundtrips. Consider a
65+
[simple statement](../simple/) instead.
2866

2967
### Parameters and binding
3068

@@ -36,9 +74,8 @@ ps1 = session.prepare("insert into product (sku, description) values (?, ?)");
3674
ps2 = session.prepare("insert into product (sku, description) values (:s, :d)");
3775
```
3876

39-
To turn the statement into its executable form, you need to *bind* it
40-
and provide values for the parameters. As shown previously, there is a
41-
shorthand form to do it all in one call:
77+
To turn the statement into its executable form, you need to *bind* it to create a [BoundStatement]. As shown previously,
78+
there is a shorthand to provide the parameters in the same call:
4279

4380
```java
4481
BoundStatement bound = ps1.bind("324378", "LCD screen");
@@ -74,7 +111,7 @@ for example: `select * from sales where sku = ? and date > ? and date <
74111

75112
For native protocol V3 or below, all variables must be bound. With native
76113
protocol V4 or above, variables can be left unset, in which case they
77-
will be ignored server side (no tombstones will be generated). If you're
114+
will be ignored server side (no tombstones will be generated). If you're
78115
reusing a bound statement you can use the `unset` method to unset variables
79116
that were previously set:
80117

@@ -95,21 +132,19 @@ A bound statement also has getters to retrieve the values. Note that
95132
this has a small performance overhead since values are stored in their
96133
serialized form.
97134

98-
### How the driver handles prepared statements
135+
`BoundStatement` is **not thread-safe**. You can reuse an instance multiple times with different parameters, but only
136+
from a single thread, and only if you use the synchronous API ([Session#execute][execute] is fine,
137+
[Session#executeAsync][executeAsync] is not).
138+
Also, make sure you don't accidentally reuse parameters from previous executions.
99139

100-
When the driver prepares a statement, it sends the query string to
101-
Cassandra, which caches the statement and returns an identifier. Later,
102-
when the driver needs to execute the statement, it just sends the
103-
identifier and parameter values. Note that the identifier is
104-
deterministic, so it will always be the same for all nodes (it's a
105-
actually a hash of the query string).
140+
### Preparing on multiple nodes
106141

107-
Prepared statements are not replicated across the cluster. It is the
142+
Cassandra does not replicate prepared statements across the cluster. It is the
108143
driver's responsibility to ensure that each node's cache is up to
109144
date. It uses a number of strategies to achieve this:
110145

111146
1. When a statement is initially prepared, it is first sent to a single
112-
node in the cluster (this prevents from hitting all nodes in case
147+
node in the cluster (this avoids hitting all nodes in case
113148
the query string is wrong). Once that node replies successfully, the
114149
driver re-prepares on all remaining nodes:
115150

@@ -135,6 +170,9 @@ date. It uses a number of strategies to achieve this:
135170
|<-----------------------| | | |
136171
```
137172
173+
The prepared statement identifier is deterministic (it's a hash of the query string), so it is the same
174+
for all nodes.
175+
138176
2. if a node crashes, it loses all of its prepared statements. So the
139177
driver keeps a client-side cache; anytime a node is marked back up,
140178
the driver re-prepares all statements on it;
@@ -184,5 +222,9 @@ Changing the driver's defaults should be done with care and only in
184222
specific situations; read each method's Javadoc for detailed
185223
explanations.
186224
225+
[PreparedStatement]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/PreparedStatement.html
226+
[BoundStatement]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/BoundStatement.html
187227
[setPrepareOnAllHosts]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/QueryOptions.html#setPrepareOnAllHosts-boolean-
188-
[setReprepareOnUp]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/QueryOptions.html#setReprepareOnUp-boolean-
228+
[setReprepareOnUp]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/QueryOptions.html#setReprepareOnUp-boolean-
229+
[execute]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/Session.html#execute-com.datastax.driver.core.Statement-
230+
[executeAsync]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/core/Session.html#executeAsync-com.datastax.driver.core.Statement-

0 commit comments

Comments
 (0)