Skip to content

Commit 54858e9

Browse files
committed
reordered sections to separate prerequisites from discussion
1 parent 071af31 commit 54858e9

1 file changed

Lines changed: 43 additions & 32 deletions

File tree

docs/advanced-analytics/tutorials/use-python-revoscalepy-to-create-model.md

Lines changed: 43 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -18,33 +18,33 @@ manager: "cgronlund"
1818
# Use Python with revoscalepy to create a model
1919
[!INCLUDE[appliesto-ss-xxxx-xxxx-xxx-md-winonly](../../includes/appliesto-ss-xxxx-xxxx-xxx-md-winonly.md)]
2020

21-
This example demonstrates how you can create a linear regression model in SQL Server, using an algorithm from the **revoscalepy** package. The code in this sample uses data that is in SQL Server, but the code is executed from a remote Python development client.
21+
This example demonstrates how you can run Python code from a remote development client, to create a linear regression model in SQL Server. This sample builds a model using an algorithm from the **revoscalepy** package, with data in SQL Server.
2222

23-
## Prerequisites
23+
## What you'll learn
2424

25-
+ To run Python code in SQL Server, you must have installed SQL Server 2017, and you must install and enable the feature, **Machine Learning Services** with Python. Other versions of SQL Server do not support Python integration.
25+
This sample demonstrates the process of creating a Python model in a remote _compute context_, which lets you work from a client, but choose a remote environment, such as SQL Server, Spark, or Machine Learning Server, where the operations are actually performed. Using compute contexts makes it easier to write code once and deploy it to any supported environment.
2626

27-
If you installed a pre-release version of SQL Server 2017, we strongly recommend that you update to at least the RTM version. Later service releases continue to upgrade and expand on Python functionality, so we recommend getting the latest version whenever possible.
27+
To execute Python code in SQL Server requires the **revoscalepy** package. This is a special Python package provided by Microsoft, similar to the **RevoScaleR** package for the R language. The **revoscalepy** package supports the creation of compute contexts, and provides the infrastructure for passing data and models between a local workstation and a remote server.
2828

29-
+ This sample uses the Airline dataset, which is available in both R and Python. You should create a database for your Python samples, and populate a table with the data before running this code.
29+
In this sample, you use data in SQL Server to train a linear model based on [rx_lin_mod](https://docs.microsoft.com/machine-learning-server/python-reference/revoscalepy/rx-lin-mod), a function in **revoscalepy** that supports regression over very large datasets. For more information, see [What is revoscalepy?](../python/what-is-revoscalepy.md) and the [Python function reference](https://docs.microsoft.com/machine-learning-server/python-reference/introducing-python-package-reference)
3030

31-
This article provides information about the sample datasets, and how you can import the data from a CSV file into SQL Server: [Sample data in RevoScaleR](https://docs.microsoft.com/machine-learning-server/r/sample-built-in-data).
31+
This sample also demonstrates the basics of how to set up and then use a **SQL Server compute context** in Python. For a discussion of how compute contexts work with other platforms, and which compute contexts are supported, see [Compute context for script execution in Machine Learning Server](https://docs.microsoft.com/machine-learning-server/r/concept-what-is-compute-context)
3232

33-
+ The code that is provided uses the database `PyTestDb`. You can change the name of this database, but be sure to update your connection string.
33+
## Prerequisites
3434

35-
## Overview of revoscalepy and compute contexts
35+
+ To run Python code in SQL Server requires SQL Server 2017 or later. Earlier versions of SQL Server do not support Python integration. Moreover, you must explicitly install and then enable the feature, **Machine Learning Services**, choosing the Python language option.
3636

37-
The **revoscalepy** package for Python contains objects, transformations, and algorithms similar to those provided for the **RevoScaleR** package for the R language.
37+
If you installed a pre-release version of SQL Server 2017, you should update to at least the RTM version. Later service releases continue to upgrade and expand on Python functionality. Some features of this tutorial might not work in early pre-release versions.
3838

39-
For more information, see [What is revoscalepy?](../python/what-is-revoscalepy.md) and the [Python function reference](https://docs.microsoft.com/machine-learning-server/python-reference/introducing-python-package-reference)
39+
+ Before trying to run any Python code, set up a database for testing Python samples. The code that is provided in this article uses the database `PyTestDb`. You can change the name of this database, but be sure to update your connection string.
4040

41-
This library also supports the notion of a _compute context_, which lets you work from a client, but specify a remote environment, such as SQL Server, Spark, or Machine Learning Server, where the operations are actually performed. Functions that can be run in remote compute context include moving data between compute contexts, transforming data, and training predictive models using popular algorithms such as logistic and linear regression, decision trees, and neural networks.
41+
+ This sample uses the Airline dataset, which is available in both R and Python. After you have created a database for your Python samples, populate a table with the data.
4242

43-
This sample demonstrates how to set up and then use a SQL Server compute context. For a discussion of how compute contexts work with other platforms, and which compute contexts are supported, see [Compute context for script execution in Machine Learning Server](https://docs.microsoft.com/machine-learning-server/r/concept-what-is-compute-context)
43+
For information about the sample datasets, and how you can import the data from a CSV file into SQL Server, see this article: [Sample data in RevoScaleR](https://docs.microsoft.com/machine-learning-server/r/sample-built-in-data).
4444

4545
## Run the sample code
4646

47-
This code performs the following steps:
47+
After you have prepared the database and have the data ready in a table, open a Python development environment. The code performs the following steps:
4848

4949
1. Imports the required libraries and functions
5050
2. Creates a connection to SQL Server, and creates data source objects for working with the data
@@ -55,16 +55,11 @@ This code performs the following steps:
5555

5656
All operations are performed using an instance of SQL Server as the compute context.
5757

58-
In general, the process of calling Python in a remote compute context is similar to the way you use R in a remote compute context.
59-
You can execute the sample either as a Python script from the command line, or by using a Python development environment that includes the Python integration components provided in this release.
60-
In your code, you create and use a compute context object to indicate where you want specific computations to be performed.
61-
6258
> [!NOTE]
6359
> Be sure to change the database and environment names as appropriate.
6460
>
6561
> For a demonstration of this sample running from the command line, see this video: [SQL Server 2017 Advanced Analytics with Python](https://www.youtube.com/watch?v=FcoY795jTcc)
6662
67-
6863
### Sample code
6964

7065
```python
@@ -114,42 +109,58 @@ def test_linmod_sql():
114109
summary = rx_summary("ArrDelay ~ DayOfWeek", data = data_source, compute_context = sql_compute_context)
115110
```
116111

117-
## Discussion
112+
### Defining a data source vs. defining a compute context
118113

119-
Let's review the code and highlight some key steps.
114+
A data source is different from a compute context. The _data source_ defines the data used in your code. The _compute context_ defines where the code will be executed. However, they use some of the same information:
120115

121-
### Defining a data source and compute context
116+
+ Python variables, such as `sql_query` and `sql_connection_string`, define the source of the data.
122117

123-
A data source is different from a compute context. The _data source_ defines the data used in your code. The _compute context_ defines where the code will be executed.
118+
Pass these variables to the [RxSqlServerData](https://docs.microsoft.com/r-server/python-reference/revoscalepy/rxsqlserverdata) constructor to implement the **data source object** named `data_source`.
124119

125-
1. Create Python variables, such as `sql_query` and `sql_connection_string`, that define the source and the data you want to use. Pass these variables to the [RxSqlServerData](https://docs.microsoft.com/r-server/python-reference/revoscalepy/rxsqlserverdata) constructor to implement the **data source object** named `data_source`.
126-
2. Create a compute context object by using the [RxInSqlServer](https://docs.microsoft.com/r-server/python-reference/revoscalepy/rxinsqlserverdata) constructor. In this example, you pass the same connection string you defined earlier, on the assumption that the data is on the same SQL Server instance that you will be using as the compute context. However, the data source and the compute context could be on different servers. The resulting **compute context object** is named `sql_cc`.
127-
3. Choose the active compute context. By default, operations are run locally, which means that if you don't specify a different compute context, the data will be fetched from the data source, and the model-fitting will run in your current Python environment.
120+
+ You create a **compute context object** by using the [RxInSqlServer](https://docs.microsoft.com/r-server/python-reference/revoscalepy/rxinsqlserverdata) constructor. The resulting **compute context object** is named `sql_cc`.
128121

122+
This example re-uses the same connection string that you used in the data source, on the assumption that the data is on the same SQL Server instance that you will be using as the compute context.
123+
124+
However, the data source and the compute context could be on different servers.
125+
129126
### Changing compute contexts
130127

131-
In this example, you set the compute context by using an argument of the individual **rx** function.
128+
After you define a compute context, you must set the **active compute context**.
129+
130+
By default, most operations are run locally, which means that if you don't specify a different compute context, the data will be fetched from the data source, and the code will run in your current Python environment.
131+
132+
There are two ways to set the active compute context:
133+
134+
+ As an argument of a method or function
135+
136+
In this example, you set the compute context by using an argument of the individual **rx** function.
132137

133-
`linmod = rx_lin_mod_ex("ArrDelay ~ DayOfWeek", data = data, compute_context = sql_compute_context)`
138+
`linmod = rx_lin_mod_ex("ArrDelay ~ DayOfWeek", data = data, compute_context = sql_compute_context)`
139+
140+
In the call to **rxsummary**, the compute context is reused.
141+
142+
`summary = rx_summary("ArrDelay ~ DayOfWeek", data = data_source, compute_context = sql_compute_context)`
134143

135-
The same applies in the call to **rxsummary**, where the compute context is reused.
144+
+ By calling `rx_set_computecontext`
136145

137-
`summary = rx_summary("ArrDelay ~ DayOfWeek", data = data_source, compute_context = sql_compute_context)`
146+
Use the function [rx_set_computecontext](https://docs.microsoft.com/r-server/python-reference/revoscalepy/rx-set-compute-context) to toggle between compute contexts that have already been defined.
138147

139-
You can also use the function [rx_set_computecontext](https://docs.microsoft.com/r-server/python-reference/revoscalepy/rx-set-compute-context) to toggle between compute contexts that have already been defined.
148+
After you have set the active compute context, it remains active until you change it.
140149

141-
### Setting the degree of parallelism
150+
### Using parallel processing and streaming
142151

143152
When you define the compute context, you can also set parameters that control how the data is handled by the compute context. These parameters differ depending on the data source type.
144153

145154
For SQL Server compute contexts, you can set the batch size, or provide hints about the degree of parallelism to use in running tasks.
146155

147-
The sample was run on a computer with four processors, so we set the *num_tasks* parameter to 4. If you set this value to 0, SQL Server uses the default, which is to run as many tasks in parallel as possible, under the current MAXDOP settings for the server.However, even in servers with many processors, the exact number of tasks that might be allocated depends on many other factors, such as server settings, and other jobs that are running.
156+
+ The sample was run on a computer with four processors, so the `num_tasks` parameter is set to 4 to allow maximum use of resources.
157+
+ If you set this value to 0, SQL Server uses the default, which is to run as many tasks in parallel as possible, under the current MAXDOP settings for the server. However, the exact number of tasks that might be allocated depends on many other factors, such as server settings, and other jobs that are running.
148158

149159
## Related samples
150160

151161
See these Python samples and tutorials for advanced tips and end-to-end demos.
152162

163+
+ [Run Python code in T-SQL](run-python-using-t-sql.md)
153164
+ [In-Database Python for SQL developers](sqldev-in-database-python-for-sql-developers.md)
154165
+ [Build a predictive model using Python and SQL Server](https://microsoft.github.io/sql-ml-tutorials/python/rentalprediction/)
155166
+ [Deploy and consume Python Models](../python/publish-consume-python-code.md)

0 commit comments

Comments
 (0)