Skip to content

Commit a172458

Browse files
authored
Merge pull request #14185 from MicrosoftDocs/master
3/17 PM Publish
2 parents b27437b + 2c82adf commit a172458

25 files changed

Lines changed: 270 additions & 208 deletions

docs/2014/includes/media/yes.png

170 Bytes
Loading

docs/big-data-cluster/deploy-active-directory.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -73,19 +73,19 @@ The BDC domain service account (DSA) needs to be able to create users, groups, a
7373

7474
![image15](./media/deploy-active-directory/image15.png)
7575

76-
1. Click **Add...** and add the **[!INCLUDE[big-data-clusters](../includes/ssbigdataclusters-nover.md)]DSA** user
76+
1. Click **Add...** and add the **bdcDSA** user
7777

7878
![image16](./media/deploy-active-directory/image16.png)
7979

8080
![image17](./media/deploy-active-directory/image17.png)
8181

82-
1. Select the **[!INCLUDE[big-data-clusters](../includes/ssbigdataclusters-nover.md)]DSA** user and clear all permissions, then click **Advanced**
82+
1. Select the **bdcDSA** user and clear all permissions, then click **Advanced**
8383

8484
1. Click **Add**
8585

8686
![image18](./media/deploy-active-directory/image18.png)
8787

88-
- Click **Select a Principal**, insert **[!INCLUDE[big-data-clusters](../includes/ssbigdataclusters-nover.md)]DSA**, and click Ok
88+
- Click **Select a Principal**, insert **bdcDSA**, and click Ok
8989

9090
- Set **Type** to **Allow**
9191

@@ -109,7 +109,7 @@ The BDC domain service account (DSA) needs to be able to create users, groups, a
109109

110110
- Click **Add**
111111

112-
- Click **Select a Principal**, insert **[!INCLUDE[big-data-clusters](../includes/ssbigdataclusters-nover.md)]DSA**, and click Ok
112+
- Click **Select a Principal**, insert **bdcDSA**, and click Ok
113113

114114
- Set **Type** to **Allow**
115115

@@ -123,7 +123,7 @@ The BDC domain service account (DSA) needs to be able to create users, groups, a
123123

124124
- Click **Add**
125125

126-
- Click **Select a Principal**, insert **[!INCLUDE[big-data-clusters](../includes/ssbigdataclusters-nover.md)]DSA**, and click Ok
126+
- Click **Select a Principal**, insert **bdcDSA**, and click Ok
127127

128128
- Set **Type** to **Allow**
129129

@@ -162,7 +162,7 @@ AD integration requires the following parameters. Add these parameters to the `c
162162

163163
- `security.activeDirectory.ouDistinguishedName`: distinguished name of an organizational unit (OU) where all AD accounts created by cluster deployment will be added. If the domain is called `contoso.local`, the OU distinguished name is: `OU=BDC,DC=contoso,DC=local`.
164164

165-
- `security.activeDirectory.dnsIpAddresses`: list of IP addresses of domain controllers
165+
- `security.activeDirectory.dnsIpAddresses`: contains the list of domain’s DNS servers IP addresses.
166166

167167
- `security.activeDirectory.domainControllerFullyQualifiedDns`: List of FQDN of domain controller. The FQDN contains the machine/host name of the domain controller. If you have multiple domain controllers, you can provide a list here. Example: `HOSTNAME.CONTOSO.LOCAL`
168168

@@ -246,7 +246,7 @@ You can find an example script here for [deploying a SQL Server big data cluster
246246

247247
You should now have set all the required parameters for a deployment of BDC with Active Directory integration.
248248

249-
For full documentation of how to deploy [!INCLUDE[big-data-clusters](../includes/ssbigdataclusters-nover.md)], please visit the [official documentation](deployment-guidance.md).
249+
You can now deploy the BDC cluster integrated with Active Directory using the `azdata` command and the kubeadm-prod deployment profile. For full documentation of how to deploy [!INCLUDE[big-data-clusters](../includes/ssbigdataclusters-nover.md)], please visit the [How to deploy SQL Server Big Data Clusters on Kubernetes](deployment-guidance.md).
250250

251251
## Verify reverse DNS entry for domain controller
252252

@@ -321,3 +321,5 @@ curl -k -v --negotiate -u : https://<Gateway DNS name>:30443/gateway/default/web
321321
- Only one BDC per domain (Active Directory) is allowed at this time. Enabling multiple BDCs per domain is planned for a future release.
322322

323323
- None of the AD groups specified in security configurations can be DomainLocal scoped. You can check the scope of an AD group by following [these instructions](https://docs.microsoft.com/powershell/module/activedirectory/get-adgroup?view=winserver2012-ps&viewFallbackFrom=winserver2012r2-ps).
324+
325+
- AD account that can be used to login into BDC are allowed from the same domain that was configured for BDC, Enabling logins from other trusted domain is planned for a future release

docs/big-data-cluster/spark-mssql-connector.md

Lines changed: 51 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -15,22 +15,51 @@ ms.technology: big-data-cluster
1515

1616
A key big data usage pattern is high volume data processing in Spark, followed by writing the data to SQL Server for access to line-of-business applications. These usage patterns benefit from a connector that utilizes key SQL optimizations and provides an efficient write mechanism.
1717

18-
This article provides an example of how to use the MSSQL Spark connector to read and write to the following locations within a big data cluster:
19-
18+
This article provides an overview of the MSSQL Spark connector interface and instantiating it for use with non-AD mode and AD-mode. Then, provides an example of how to use the MSSQL Spark connector to read and write to the following locations within a big data cluster:
2019
1. The SQL Server master instance
2120
1. The SQL Server data pool
2221

2322
![MSSQL Spark connector diagram](./media/spark-mssql-connector/mssql-spark-connector-diagram.png)
2423

25-
The sample performs the following tasks:
26-
27-
- Read a file from HDFS and do some basic processing.
28-
- Write the dataframe to a SQL Server master instance as a SQL table and then read the table to a dataframe.
29-
- Write the dataframe to a SQL Server data pool as a SQL external table and then read the external table to a dataframe.
30-
3124
## MSSQL Spark Connector Interface
3225

33-
SQL Server 2019 provides the **MSSQL Spark connector** for big data clusters that uses SQL Server bulk write APIs for Spark to SQL writes. MSSQL Spark Connector is based on Spark data source APIs and provides a familiar Spark JDBC connector interface. For interface parameters refer [Apache Spark documentation](http://spark.apache.org/docs/latest/sql-data-sources-jdbc.html). The MSSQL Spark connector is referenced by the name **com.microsoft.sqlserver.jdbc.spark**.
26+
SQL Server 2019 provides the **MSSQL Spark connector** for big data clusters that uses SQL Server bulk write APIs for Spark to SQL writes. MSSQL Spark Connector is based on Spark data source APIs and provides a familiar Spark JDBC connector interface. For interface parameters refer [Apache Spark documentation](http://spark.apache.org/docs/latest/sql-data-sources-jdbc.html). The MSSQL Spark connector is referenced by the name **com.microsoft.sqlserver.jdbc.spark**. The MSSQL Spark connector supports two security modes to connect with SQL Server, non-Active Directory mode and Active Directory(AD)-mode:
27+
### Non-AD Mode:
28+
In non-AD mode security, each user has a username and password which need to be provided as parameters during the connector instantiation to perform read and/or writes.
29+
An example connector instantiation for non-AD mode is below:
30+
```python
31+
# Note: '?' is a placeholder for a necessary user-specified value
32+
connector_type = "com.microsoft.sqlserver.jdbc.spark"
33+
34+
url = "jdbc:sqlserver://master-p-svc;databaseName=?;"
35+
writer = df.write \
36+
.format(connector_type)\
37+
.mode("overwrite")
38+
.option("url", url) \
39+
.option("user", ?) \
40+
.option("password",?)
41+
writer.save()
42+
```
43+
### AD Mode:
44+
In AD mode security, after a user has generated a key tab file, the user needs to provide the `principal` and `keytab` as parameters during the connector instantiation.
45+
46+
In this mode, the driver loads the keytab file to the respective executor containers. Then, the executors use the principal name and keytab to generate a token that is used to create a JDBC connector for read/write.
47+
48+
An example connector instantiation for AD mode is below:
49+
```python
50+
# Note: '?' is a placeholder for a necessary user-specified value
51+
connector_type = "com.microsoft.sqlserver.jdbc.spark"
52+
53+
url = "jdbc:sqlserver://master-p-svc;databaseName=?;integratedSecurity=true;authenticationScheme=JavaKerberos;"
54+
writer = df.write \
55+
.format(connector_type)\
56+
.mode("overwrite")
57+
.option("url", url) \
58+
.option("principal", ?) \
59+
.option("keytab", ?)
60+
61+
writer.save()
62+
```
3463

3564
The following table describes interface parameters that have changed or are new:
3665

@@ -40,13 +69,19 @@ The following table describes interface parameters that have changed or are new:
4069

4170
The connector uses SQL Server Bulk write APIs. Any bulk write parameters can be passed as optional parameters by the user and are passed as-is by the connector to the underlying API. For more information about bulk write operations, see [SQLServerBulkCopyOptions]( ../connect/jdbc/using-bulk-copy-with-the-jdbc-driver.md#sqlserverbulkcopyoptions).
4271

43-
## Prerequisites
72+
## MSSQL Spark Connector Sample
73+
The sample performs the following tasks:
74+
75+
- Read a file from HDFS and do some basic processing.
76+
- Write the dataframe to a SQL Server master instance as a SQL table and then read the table to a dataframe.
77+
- Write the dataframe to a SQL Server data pool as a SQL external table and then read the external table to a dataframe.
78+
### Prerequisites
4479

4580
- A [SQL Server big data cluster](deploy-get-started.md).
4681

4782
- [Azure Data Studio](https://aka.ms/getazuredatastudio).
4883

49-
## Create the target database
84+
### Create the target database
5085

5186
1. Open Azure Data Studio, and [connect to the SQL Server master instance of your big data cluster](connect-to-big-data-cluster.md).
5287

@@ -57,7 +92,7 @@ The connector uses SQL Server Bulk write APIs. Any bulk write parameters can be
5792
GO
5893
```
5994

60-
## Load sample data into HDFS
95+
### Load sample data into HDFS
6196

6297
1. Download [AdultCensusIncome.csv](https://amldockerdatasets.azureedge.net/AdultCensusIncome.csv) to your local machine.
6398

@@ -69,9 +104,9 @@ The connector uses SQL Server Bulk write APIs. Any bulk write parameters can be
69104

70105
![AdultCensusIncome CSV file](./media/spark-mssql-connector/spark_data.png)
71106

72-
## Run the sample notebook
107+
### Run the sample notebook
73108

74-
To demonstrate the use of the MSSQL Spark Connector with this data, you can download a sample notebook, open it in Azure Data Studio, and run each code block. For more information about working with notebooks, see [How to use notebooks in SQL Server](notebooks-guidance.md).
109+
To demonstrate the use of the MSSQL Spark Connector with this data in non-AD mode, you can download a sample notebook, open it in Azure Data Studio, and run each code block. For more information about working with notebooks, see [How to use notebooks in SQL Server](notebooks-guidance.md).
75110

76111
1. From a PowerShell or bash command line, run the following command to download the **mssql_spark_connector_non_ad_pyspark.ipynb** sample notebook:
77112

@@ -86,3 +121,5 @@ To demonstrate the use of the MSSQL Spark Connector with this data, you can down
86121
## Next steps
87122

88123
For more information about big data clusters, see [How to deploy [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)] on Kubernetes](deployment-guidance.md)
124+
125+
Have feedback or feature recommendations for SQL Server Big Data Clusters? [Leave us a note at SQL Server Big Data Clusters Feedback](https://aka.ms/sql-server-bdc-feedback).

docs/includes/media/yes.png

170 Bytes
Loading

docs/linux/quickstart-install-connect-suse.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ In this quickstart, you install SQL Server 2017 or SQL Server 2019 on SUSE Linux
2525
In this quickstart, you install SQL Server 2019 on SUSE Linux Enterprise Server (SLES) v12. You then connect with **sqlcmd** to create your first database and run queries.
2626

2727
> [!IMPORTANT]
28-
> SQL Server 2019 is supported on SUSE Enterprise Linux Server v12 SP2, SP3, or SP4.
28+
> SQL Server 2019 is supported on SUSE Enterprise Linux Server v12 SP2, SP3, SP4 or SP5.
2929
3030
::: moniker-end
3131

@@ -44,7 +44,7 @@ You must have a SLES v12 SP2 machine with **at least 2 GB** of memory. The file
4444
<!--SQL Server 2019 on Linux-->
4545
::: moniker range=">= sql-server-linux-ver15 || >= sql-server-ver15 || =sqlallproducts-allversions"
4646

47-
You must have a SLES v12 SP2, SP3, or SP4 machine with **at least 2 GB** of memory. The file system must be **XFS** or **EXT4**. Other file systems, such as **BTRFS**, are unsupported.
47+
You must have a SLES v12 SP2, SP3, SP4 or SP5 machine with **at least 2 GB** of memory. The file system must be **XFS** or **EXT4**. Other file systems, such as **BTRFS**, are unsupported.
4848

4949
::: moniker-end
5050

docs/linux/sample-unattended-install-ubuntu.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,9 @@ This sample Bash script installs SQL Server 2017 on Ubuntu 16.04 without interac
2626

2727
## Sample script
2828

29+
> [!NOTE]
30+
> The script might fail if SQL Server is slow to start. That's because the script will exit with a non-zero status. Removing the `-e` switch on the first line may resolve this issue.
31+
2932
```bash
3033
#!/bin/bash -e
3134

docs/relational-databases/extended-events/quick-start-extended-events-in-sql-server.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
title: "Quickstart: Extended events in SQL Server"
3+
description: This quickstart helps you use Extended Events, a lightweight performance monitoring system, to collect data to monitor and troubleshoot problems in SQL Server.
34
ms.date: "05/28/2019"
45
ms.prod: sql
56
ms.prod_service: "database-engine, sql-database"

docs/relational-databases/extended-events/selects-and-joins-from-system-views-for-extended-events-in-sql-server.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
title: "SELECTs and JOINs from system views for extended events"
3+
description: There are system views of extended events in SQL Server and Azure SQL Database. Learn how event session information is represented in different perspectives.
34
ms.date: "08/02/2016"
45
ms.prod: sql
56
ms.prod_service: "database-engine, sql-database"

docs/relational-databases/extended-events/sql-server-extended-events-engine.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
title: "SQL Server Extended Events Engine"
3+
description: The SQL Server Extended Events engine enables the definition of events and processing of event data, manages Extended Events, and maintains a list of sessions.
34
ms.date: "03/14/2017"
45
ms.prod: sql
56
ms.prod_service: "database-engine, sql-database"

docs/relational-databases/extended-events/sql-server-extended-events-packages.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
title: "SQL Server Extended Events Packages"
3+
description: A package is a container for SQL Server Extended Events objects. This article describes the objects a package can contain.
34
ms.date: "03/14/2017"
45
ms.prod: sql
56
ms.prod_service: "database-engine, sql-database"

0 commit comments

Comments
 (0)