Skip to content

Commit 77aa6e9

Browse files
TrishaMcTrishaMc
authored
Merge pull request #5941 from yanancai/master
Update docs of ADLA task for Azure Feature Pack
2 parents 289091b + a9401f0 commit 77aa6e9

4 files changed

Lines changed: 200 additions & 1 deletion

File tree

‎docs/integration-services/connection-manager/TOC.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
# [Analysis Services Connection Manager](analysis-services-connection-manager.md)
66
## [Add Analysis Services Connection Manager Dialog Box UI Reference](add-analysis-services-connection-manager-dialog-box-ui-reference.md)
77
# [Azure Data Lake Store Connection Manager](azure-data-lake-store-connection-manager.md)
8+
# [Azure Data Lake Analytics Connection Manager](azure-data-lake-analytics-connection-manager.md)
89
# [Azure HDInsight Connection Manager](azure-hdinsight-connection-manager.md)
910
# [Azure Resource Manager Connection Manager](azure-resource-manager-connection-manager.md)
1011
# [Azure Storage Connection Manager](azure-storage-connection-manager.md)

‎docs/integration-services/connection-manager/azure-data-lake-analytics-connection-manager.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
title: "Azure Data Lake Analytics Connection Manager | Microsoft Docs"
3+
ms.custom: ""
4+
ms.date: "05/18/2018"
5+
ms.prod: sql
6+
ms.prod_service: "integration-services"
7+
ms.component: "connection-manager"
8+
ms.reviewer: ""
9+
ms.suite: "sql"
10+
ms.technology:
11+
- "integration-services"
12+
ms.tgt_pltfrm: ""
13+
ms.topic: conceptual
14+
f1_keywords:
15+
- "SQL13.DTS.DESIGNER.AFPADLSCM.F1"
16+
- "sql14.dts.designer.afpadlscm.f1"
17+
ms.assetid: f4c44553-0f08-4731-ac47-7534990b8c8d
18+
caps.latest.revision: 7
19+
author: "yanancai"
20+
ms.author: "yanacai"
21+
ms.reviewer: "douglasl"
22+
manager: craigg
23+
---
24+
25+
# Azure Data Lake Analytics Connection Manager
26+
27+
A SQL Server Integration Services (SSIS) package can use the Azure Data Lake Analytics connection manager to connect to an Azure Data Lake Analytics account with one of the two following authentication types:
28+
- Azure AD User Identity
29+
- Azure AD Service Identity
30+
31+
The Azure Data Lake Analytics connection manager is a component of the [SQL Server Integration Services (SSIS) Feature Pack for Azure](../../integration-services/azure-feature-pack-for-integration-services-ssis.md).
32+
33+
## Configure the Azure Data Lake Analytics Connection Manager
34+
35+
1. In the **Add SSIS Connection Manager** dialog box, select **AzureDataLakeAnalytics**, and then select **Add**. The **Azure Data Lake Analytics Connection Manager Editor** dialog box opens.
36+
37+
2. In the **Azure Data Lake Analytics Connection Manager Editor** dialog box, in the **ADLA Account Name** field, provide the Azure Data Lake Analytics account name. For example: myadlaaccountname.
38+
39+
3. In the **Authentication** field, choose the appropriate authentication type to access the data in Azure Data Lake Analytics.
40+
41+
1. If you select the **Azure AD User Identity** authentication option, do the following things:
42+
1. Provide values for the **User Name** and **Password** fields.
43+
44+
2. To test the connection, select **Test Connection**. If you or the tenant administrator didn't previously consent to allow SSIS to access your Azure Data Lake Analytics account, select **Accept** when prompted. For more information about this consent experience, see [Integrating applications with Azure Active Directory](https://docs.microsoft.com/azure/active-directory/active-directory-integrating-applications#updating-an-application).
45+
46+
> [!NOTE]
47+
> When you select the **Azure AD User Identity** authentication option, multi-factor authentication and Microsoft account authentication are not supported.
48+
49+
2. If you select the **Azure AD Service Identity** authentication option, do the following things:
50+
1. Create an Azure Active Directory (AAD) application and service principal to access the Azure Data Lake Analytics account. For more information about this authentication option, see [Use portal to create Active Directory application and service principal that can access resources](https://docs.microsoft.com/azure/azure-resource-manager/resource-group-create-service-principal-portal).
51+
52+
2. Assign appropriate permissions to let this AAD application accesses your Azure Data Lake Analytics account. Learn how to grant permissions to your Azure Data Lake Analytics account [using Add User Wizard](https://docs.microsoft.com/azure/data-lake-analytics/data-lake-analytics-manage-use-portal#add-a-new-user).
53+
54+
3. Provide values for the **Application ID**, **Authentication Key**, and **Tenant ID** fields.
55+
56+
4. To test the connection, select **Test Connection**.
57+
58+
4. Select **OK** to close the **Azure Data Lake Analytics Connection Manager Editor** dialog box.
59+
60+
## View the properties of the connection manager
61+
You can see the properties of the connection manager you created in the **Properties** window.
62+
63+

‎docs/integration-services/control-flow/TOC.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,8 @@
1616
## [Azure HDInsight Create Cluster Task](azure-hdinsight-create-cluster-task.md)
1717
## [Azure HDInsight Delete Cluster Task](azure-hdinsight-delete-cluster-task.md)
1818
## [Azure SQL DW Upload Task](azure-sql-dw-upload-task.md)
19-
## [Azure Data Lake Store File System Task](azure-data-lake-store-file-system-task.md)
19+
## [Azure Data Lake Store File System Task](azure-data-lake-store-file-system-task.md)
20+
## [Azure Data Lake Analytics Task](azure-data-lake-analytics-task.md)
2021
## [Back Up Database Task](back-up-database-task.md)
2122
## [Bulk Insert Task](bulk-insert-task.md)
2223
## [CDC Control Task](cdc-control-task.md)

‎docs/integration-services/control-flow/azure-data-lake-analytics-task.md‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
---
2+
title: "Azure Data Lake Analytics Task | Microsoft Docs"
3+
ms.custom: ""
4+
ms.date: "05/18/2018"
5+
ms.prod: sql
6+
ms.prod_service: "integration-services"
7+
ms.component: "control-flow"
8+
ms.reviewer: ""
9+
ms.suite: "sql"
10+
ms.technology:
11+
- "integration-services"
12+
ms.tgt_pltfrm: ""
13+
ms.topic: conceptual
14+
f1_keywords:
15+
- "SQL13.DTS.DESIGNER.AFPADLSTASK.F1"
16+
- "SQL14.DTS.DESIGNER.AFPADLSTASK.F1"
17+
author: "yanancai"
18+
ms.author: "yanacai"
19+
ms.reviewer: "douglasl"
20+
manager: craigg
21+
---
22+
23+
# Azure Data Lake Analytics Task
24+
25+
The Azure Data Lake Analytics Task lets users submit U-SQL jobs to Azure Data Lake Analytics service. [Azure Data Lake Analytics (ADLA)](https://azure.microsoft.com/services/data-lake-analytics/).
26+
27+
The Azure Data Lake Analytics Task is a component of the [SQL Server Integration Services (SSIS) Feature Pack for Azure](../../integration-services/azure-feature-pack-for-integration-services-ssis.md).
28+
29+
## Configure the Azure Data Lake Analytics Task
30+
31+
To add an Azure Data Lake Analytics Task to a package, drag it from SSIS Toolbox to the designer canvas. Then double-click the task, or right-click the task and select **Edit**, to open the **Azure Data Lake Analytics Task Editor** dialog box. You can set properties through SSIS Designer or programmatically.
32+
33+
## General Page Configuration
34+
35+
Use the **General** page to configure the Azure Data Lake Analytics Task and provide the U-SQL script that the task submits. To learn more about the U-SQL language, see [U-SQL Language Reference](https://msdn.microsoft.com/azure/data-lake-analytics/u-sql/u-sql-language-reference).
36+
37+
### Basic Configuration
38+
39+
- **Name:** Specifies the name of the Azure Data Lake Analytics task.
40+
- **Description:** Specifies the description of the Azure Data Lake Analytics task.
41+
42+
### U-SQL Configuration
43+
44+
U-SQL configuration has two settings: **SourceType** and dynamic options based on **SourceType** value.
45+
46+
- **SourceType:** Specifies the source of the U-SQL script. The script will be submitted to an Azure Data Lake Analytics account during SSIS package execution. This property has 3 options listed in the following table.
47+
48+
|Value|Description|
49+
|-----------|-----------------|
50+
|**DirectInput**|Specifies the U-SQL script through the inline editor. Selecting this value displays the dynamic option, **USQLStatement**.|
51+
|**FileConnection**|Specifies a local .usql file that contains the U-SQL script. Selecting this option displays the dynamic option, **FileConnection**.|
52+
|**Variable**|Specifies an SSIS variable that contains the U-SQL script. Selecting this value displays the dynamic option, **SourceVariable**.|
53+
54+
- **SourceType Dynamic Options:** Specifies the script content for the U-SQL query.
55+
56+
|SourceType|Dynamic Options|
57+
|-----------|-----------------|
58+
|**SourceType = DirectInput**|Type the U-SQL query to be submitted in the option box directly, or click the browse button (...) to type the U-SQL query in the **Enter U-SQL Query** dialog box.|
59+
|**SourceType = FileConnection**|Select an existing file connection manager, or click <**New connection...**> to create a new file connection. **Related Article:** [File Connection Manager](../../integration-services/connection-manager/file-connection-manager.md), [File Connection Manager Editor](../../integration-services/connection-manager/file-connection-manager-editor.md)|
60+
|**SourceType = Variable**|Select an existing variable, or click \<**New variable...**> to create a new variable. **Related Article:** [Integration Services &#40;SSIS&#41; Variables](../../integration-services/integration-services-ssis-variables.md), [Add Variable](http://msdn.microsoft.com/library/d09b5d31-433f-4f7c-8c68-9df3a97785d5)|
61+
62+
63+
### Job Configuration
64+
Job configuration specifies U-SQL job submission properties.
65+
66+
- **AzureDataLakeAnalyticsConnection:** Specifies the Azure Data Lake Analytics account where the U-SQL script will be submitted. Choose the connection from a list of defined connection managers. To create a new connection, select <**New connection**>. Related article: [Azure Data Lake Analytics Connection Manager](../../integration-services/connection-manager/azure-data-lake-analytics-connection-manager.md).
67+
68+
- **JobName:** Specifies the name of the U-SQL job.
69+
- **AnalyticsUnits:** Specifies the Analytics Unit count of the U-SQL job.
70+
- **Priority:** Specifies the Priority of the U-SQL job. Priority can be set from 0 to 1000, the lower the number, the higher the priority.
71+
- **RuntimeVersion:** Specifies the Azure Data Lake Analytics runtime version of the U-SQL job. It is set to "default" by default. Usually you don't need to change this property.
72+
- **Synchronous:** A Boolean value specifies if the task waits the job execution completes or not. Set to Ture, the task will be marked as succeed after job completes. Set to False, the task will be marked as succeed after job passes the Preparation phase.
73+
74+
|Value|Description|
75+
|-----------|-----------------|
76+
|True|The task result is based on the U-SQL job execution result. Job succeeds --> Task succeeds; Job fails --> Task fails; Task succeeds or fails --> Task completes.|
77+
|False|The task result is based on the U-SQL job submission and preparation result. Job submission succeeds and passes the Preparation phase --> Task succeeds; Job submission fails or job fails at the Preparation phase --> Task fails; Task succeeds or fails --> Task completes.|
78+
79+
- **TimeOut:** Specifies a time-out time in seconds for job execution. The job will be cancelled and the task will be marked as failed after the job is time-out. TimeOut property is not available if Synchronous is set to False. TimeOut property is not available if **Synchronous** is set to **false**.
80+
81+
## Parameter Mapping Page Configuration
82+
83+
Use the **Parameter Mapping** page of the **Azure Data Lake Analytics Task Editor** dialog box to map variables to parameters(U-SQL variables) in U-SQL script.
84+
85+
- **Variable Name:** After you have added a parameter mapping by clicking **Add**, select a system or user-defined variable from the list or click \<**New variable...**> to add a new variable by using the **Add Variable** dialog box. **Related Topics:** [Integration Services &#40;SSIS&#41; Variables](../../integration-services/integration-services-ssis-variables.md)
86+
87+
- **Parameter Name:** Provide a parameter/variable name in U-SQL script. Make sure the the parameter name starts with the @ sign, like @Param1.
88+
89+
Here is an example of how to pass parameters to U-SQL script.
90+
91+
**Sample U-SQL script**
92+
```
93+
@searchlog =
94+
EXTRACT UserId int,
95+
Start DateTime,
96+
Region string,
97+
Query string,
98+
Duration int,
99+
Urls string,
100+
ClickedUrls string
101+
FROM @in
102+
USING Extractors.Tsv(nullEscape:"#NULL#");
103+
104+
@rs1 =
105+
SELECT Start, Region, Duration
106+
FROM @searchlog
107+
WHERE Region == "en-gb";
108+
109+
@rs1 =
110+
SELECT Start, Region, Duration
111+
FROM @rs1
112+
WHERE Start <= DateTime.Parse("2012/02/19");
113+
114+
OUTPUT @rs1
115+
TO @out
116+
USING Outputters.Tsv(quoting:false, dateTimeFormat:null);
117+
```
118+
119+
In above script example, the input and output paths are defined in **@in** and **@out** parameters. The values for **@in** and **@out** parameters in the U-SQL script are passed dynamically by Parameter Mapping configuration.
120+
121+
|Variable Name|Parameter Name|
122+
|-------------|--------------|
123+
|User: Variable1|@in|
124+
|User: Variable2|@out|
125+
126+
## Expression Page Configuration
127+
128+
All properties in General page configuration can be assigned as a property expression to enable dynamic update of the property at run time. **Related Topics:** [Use Property Expressions in Packages](../../integration-services/expressions/use-property-expressions-in-packages.md)
129+
130+
## See Also
131+
- [Azure Data Lake Analytics Connection Manager](../../integration-services/connection-manager/azure-data-lake-analytics-connection-manager.md)
132+
- [Azure Data Lake Store File System Task](../../integration-services/control-flow/azure-data-lake-store-file-system-task.md)
133+
- [Azure Data Lake Store Connection Manager](../../integration-services/connection-manager/azure-data-lake-store-connection-manager.md)
134+

0 commit comments

Comments
 (0)