You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/big-data-cluster/deploy-big-data-tools.md
+19Lines changed: 19 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -25,6 +25,7 @@ The following table lists common big data cluster tools and how to install them:
25
25
26
26
| Tool | Required | Description | Installation |
27
27
|---|---|---|---|
28
+
|**python**| Yes | Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Many parts of big data clusters for SQL Server use python. |[Install python](#python)|
28
29
|**azdata**| Yes | Command-line tool for installing and managing a big data cluster. |[Install](deploy-install-mssqlctl.md)|
29
30
|**kubectl**<sup>1</sup> | Yes | Command-line tool for monitoring the underlying Kuberentes cluster ([More info](https://kubernetes.io/docs/tasks/tools/install-kubectl/)). |[Windows](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-with-powershell-from-psgallery)\|[Linux](https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-using-native-package-management)|
30
31
|**Azure Data Studio (insiders)**| Yes | Cross-platform graphical tool for querying SQL Server ([More info](https://docs.microsoft.com/sql/azure-data-studio/what-is?view=sql-server-ver15)). |[Install](https://aka.ms/azdata-insiders)|
@@ -58,6 +59,24 @@ The previous table provides all of the common tools that are used with big data
58
59
59
60
The remaining tools are only required in certain scenarios. **Azure CLI** can be used to manage Azure services associated with AKS deployments. **mssql-cli** is an optional but useful tool that allows you to connect to the SQL Server master instance in the cluster and run queries from the command line. And **sqlcmd** and **curl** are required if you plan to install sample data with the GitHub script.
60
61
62
+
### <a id="python"></a> Install python offline
63
+
64
+
1. On a machine with internet access, download one of the following compressed files containing Python:
65
+
66
+
| Operating system | Download |
67
+
|---|---|
68
+
| Windows | [https://go.microsoft.com/fwlink/?linkid=2074021](https://go.microsoft.com/fwlink/?linkid=2074021) |
69
+
| Linux | [https://go.microsoft.com/fwlink/?linkid=2065975](https://go.microsoft.com/fwlink/?linkid=2065975) |
After configuring the tools, deploy a SQL Server 2019 big data cluster to Kubernetes in the Cloud or on-premises. For more information, see the following deployment articles:
Copy file name to clipboardExpand all lines: docs/sql-server/what-s-new-in-sql-server-ver15-prerelease.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,8 +26,8 @@ For the current information, see [What's New in SQL Server 2019](what-s-new-in-s
26
26
27
27
| New feature or update | Details |
28
28
|:---|:---|
29
-
|`azdata` command changes |`azdata cluster` commands have been renamed to `azdata bdc`. For more information, see the [`azdata` reference](../big-data-cluster/reference-mssqlctl.md). |
30
-
|New status commands for `mssqlsctl`|`azdata` adds new commands to complement existing monitoring commands. These replace the cluster administration portal - which is removed in this release.|
29
+
|`mssqlctl` command changes |`mssqlctl cluster` commands have been renamed to `mssqlctl bdc`. For more information, see the [`mssqlctl` reference](../big-data-cluster/reference-mssqlctl.md). |
30
+
|New status commands for `mssqlsctl`|`mssqlctl` adds new commands to complement existing monitoring commands. These replace the cluster administration portal - which is removed in this release.|
31
31
| Spark compute pools | Create additional nodes to increase Spark compute power without having to scale up storage. In addition, you can start storage pool nodes that aren't used for Spark. Spark and storage are decoupled. For more information, see [Configure storage without spark](../big-data-cluster/deployment-custom-configuration.md#sparkstorage). |
32
32
| MSSQL Spark connector | Support for read/write to data pool external tables. Previous releases supported read/write to MASTER instance tables only. For more information, see [How to read and write to SQL Server from Spark using the MSSQL Spark Connector](../big-data-cluster/spark-mssql-connector.md). |
33
33
| Machine Learning using MLeap |[Train an MLeap machine learning model in Spark and score it in SQL Server using the Java language extension](../big-data-cluster/spark-create-machine-learning-model.md). |
@@ -57,7 +57,7 @@ For the current information, see [What's New in SQL Server 2019](what-s-new-in-s
57
57
58
58
| New feature or update | Details |
59
59
|:---|:---|
60
-
|**azdata** updates | Several **azdata**[command and parameter updates](../big-data-cluster/reference-mssqlctl.md). This includes an update to the **azdata login** command, which now targets the controller username and endpoint. |
60
+
|**mssqlctl** updates | Several **mssqlctl**[command and parameter updates](../big-data-cluster/reference-mssqlctl.md). This includes an update to the **mssqlctl login** command, which now targets the controller username and endpoint. |
61
61
| Storage enhancements | Support for different storage configurations for logs and data. Also, the number of persistent volume claims for a big data cluster has been reduced. |
62
62
| Multiple compute pool instances | Support for multiple compute pool instances. |
63
63
| New pool behavior and capabilities | The compute pool is now used by default for storage pool and data pool operations in a **ROUND_ROBIN** distribution only. The data pool can now use a new **REPLICATED** distribution type, which means that the same data is present on all the data pool instances. |
@@ -108,9 +108,9 @@ For the current information, see [What's New in SQL Server 2019](what-s-new-in-s
108
108
| New feature or update | Details |
109
109
|:---|:---|
110
110
| Deployment profiles | Use default and customized [deployment configuration JSON files](../big-data-cluster/deployment-guidance.md#configfile) for big data cluster deployments instead of environment variables. |
111
-
| Prompted deployments |`azdata cluster create` now prompts for any necessary settings for default deployments. |
111
+
| Prompted deployments |`mssqlctl cluster create` now prompts for any necessary settings for default deployments. |
112
112
| Service endpoint and pod name changes | For more information, see the [big data cluster release notes](../big-data-cluster/release-notes-big-data-cluster.md). |
113
-
|**azdata** improvements | Use **azdata** to [list external endpoints](../big-data-cluster/deployment-guidance.md#endpoints) and check the version of **azdata** with the `--version` parameter. |
113
+
|**mssqlctl** improvements | Use **mssqlctl** to [list external endpoints](../big-data-cluster/deployment-guidance.md#endpoints) and check the version of **mssqlctl** with the `--version` parameter. |
114
114
| Offline install |[Guidance for offline big data cluster deployments](../big-data-cluster/deploy-offline.md). |
115
115
| HDFS tiering improvements | HDFS tiering against Amazon S3 storage. OAuth support for ADLS Gen2. Caching functionality for better performance. For more information, see [HSDFS Tiering](../big-data-cluster/hdfs-tiering.md)|
116
116
| Spark to SQL Server connector |[Read and write to SQL Server from Spark using the MSSQL JDBC Connector](../big-data-cluster/spark-mssql-connector.md)|
@@ -174,11 +174,11 @@ For the current information, see [What's New in SQL Server 2019](what-s-new-in-s
174
174
| Submit Spark jobs on big data clusters in IntelliJ. |[Submit Spark jobs on SQL Server big data clusters in IntelliJ](../big-data-cluster/spark-submit-job-intellij-tool-plugin.md)|
175
175
| Common CLI for application deployment and cluster management. |[How to deploy an app on SQL Server 2019 big data cluster (preview)](../big-data-cluster/big-data-cluster-create-apps.md)|
176
176
| VS Code extension to deploy applications to a big data cluster. |[How to use VS Code to deploy applications to SQL Server big data clusters](../big-data-cluster/app-deployment-extension.md)|
177
-
| Changes to the **azdata** tool command usage. | For more details see the [known issues for azdata](../big-data-cluster/release-notes-big-data-cluster.md#azdatactp23). |
177
+
| Changes to the **mssqlctl** tool command usage. | For more details see the [known issues for mssqlctl](../big-data-cluster/release-notes-big-data-cluster.md#mssqlctlctp23). |
178
178
| Use Sparklyr in big data cluster. |[Use Sparklyr in SQL Server 2019 big data cluster](../big-data-cluster/sparklyr-from-RStudio.md)|
179
179
| Mount external HDFS-compatible storage into big data cluster with **HDFS tiering**. | See [HDFS tiering](../big-data-cluster/hdfs-tiering.md). |
180
180
| New unified connection experience for the SQL Server master instance and the HDFS/Spark Gateway. | See [SQL Server master instance and the HDFS/Spark Gateway](../big-data-cluster/connect-to-big-data-cluster.md). |
181
-
| Deleting a cluster with **azdata cluster delete** now deletes only the objects in the namespace that were part of the big data cluster. | The namespace is not deleted. However, in earlier releases this command did delete the entire namespace. |
181
+
| Deleting a cluster with **mssqlctl cluster delete** now deletes only the objects in the namespace that were part of the big data cluster. | The namespace is not deleted. However, in earlier releases this command did delete the entire namespace. |
182
182
|_Security_ endpoint names have been changed and consolidated. |**service-security-lb** and **service-security-nodeport** have been consolidated into the **endpoint-security** endpoint. |
183
183
|_Proxy_ endpoint names have been changed and consolidated. |**service-proxy-lb** and **service-proxy-nodeport** have been consolidated into the **endpoint-service-proxy** endpoint. |
184
184
|_Controller_ endpoint names have been changed and consolidated. |**service-mssql-controller-lb** and **service-mssql-controller-nodeport** have been consolidated into the **endpoint-controller** endpoint. |
@@ -224,7 +224,7 @@ For the current information, see [What's New in SQL Server 2019](what-s-new-in-s
224
224
| New feature or update | Details |
225
225
|:-----|:-----|
226
226
|Use SparkR from Azure Data Studio on a big data cluster. ||
227
-
|Deploy Python and R apps.|[Deploy applications using azdata](../big-data-cluster/big-data-cluster-create-apps.md)|
227
+
|Deploy Python and R apps.|[Deploy applications using mssqlctl](../big-data-cluster/big-data-cluster-create-apps.md)|
Copy file name to clipboardExpand all lines: docs/sql-server/what-s-new-in-sql-server-ver15.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,8 +29,8 @@ Community technology preview (CTP) 3.2 is the latest public release of [!INCLUDE
29
29
30
30
|New feature or update | Details |
31
31
|:---|:---|
32
-
|Public preview |Prior to CTP 3.2, SQL Server big data cluster was available to registered early adopters. This release allows anyone to experience the features of SQL Server Big data clusters. |
33
-
|Notebook based troubleshooting|CTP 3.2 introduces Jupyter notebooks to assist with discovery, diagnosis, and troubleshooting for components in a SQL Server big data cluster. |
32
+
|Public preview |Prior to CTP 3.2, SQL Server big data cluster was available to registered early adopters. This release allows anyone to experience the features of SQL Server Big data clusters. <br/><br/> See [Get started with SQL Server big data clusters](../big-data-cluster/deploy-get-started.md).|
33
+
|Notebook based troubleshooting.|CTP 3.2 introduces Jupyter notebooks to assist with discovery, diagnosis, and troubleshooting for components in a SQL Server big data cluster. |
34
34
|`azdata`|CTP 3.2 introduces `azdata` - a command-line utility written in Python that enables cluster administrators to bootstrap and manage the big data cluster via REST APIs. `azdata` replaces `mssqlctl`. |
35
35
|HDFS tiering refresh |Introducing refresh functionality for HDFS tiering so that an existing mount can be refreshed for the latest snapshot of the remote data. |
36
36
| | |
@@ -66,8 +66,8 @@ For more details, see [What are SQL Server big data clusters](../big-data-cluste
66
66
|:---|:---|
67
67
|Feature restrictions| Prevent some forms of SQL injection from leaking information about the database, even when the SQL injection is successful. See [Feature restrictions](../relational-databases/security/feature-restrictions.md)|
68
68
|Index encrypted columns|Create indexes on columns encrypted using randomized encryption and enclave-enabled keys, to improve the performance of rich queries (using `LIKE` and comparison operators). See [Always Encrypted with Secure Enclaves](../relational-databases/security/encryption/always-encrypted-enclaves.md).
69
-
|Transparent Data Encryption (TDE) scan - suspend and resume|[Transparent Data Encryption (TDE) scan - suspend and resume](../relational-databases/security/encryption/transparent-data-encryption.md#scan-suspend-resume)|
70
-
|Certificate management in SQL Server Configuration Manager|[Certificate Management (SQL Server Configuration Manager)](../database-engine/configure-windows/manage-certificates.md)
69
+
|Suspend and resume initial scan for Transparent Data Encryption (TDE)|See [Transparent Data Encryption (TDE) scan - suspend and resume](../relational-databases/security/encryption/transparent-data-encryption.md#scan-suspend-resume)|
70
+
|Certificate management in SQL Server Configuration Manager|See [Certificate Management (SQL Server Configuration Manager)](../database-engine/configure-windows/manage-certificates.md)
0 commit comments