Skip to content

Commit b0defa3

Browse files
v-donspencerv-donspencer
authored
Merge pull request #19919 from garyericson/08-03-copy-microsoftml-to-sqlml
SQLML: Bulk Update: Copying microsoftml to SQL MLS
2 parents 93d53c5 + 79ccbe0 commit b0defa3

47 files changed

Lines changed: 8529 additions & 46 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/machine-learning/concepts/extension-python.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ Python packages added by Microsoft for parallel and distributed workloads includ
3737
| Library | Description |
3838
|---------|-------------|
3939
| [**revoscalepy**](/machine-learning-server/python-reference/revoscalepy/revoscalepy-package) | Supports data source objects and data exploration, manipulation, transformation, and visualization. It supports creation of remote compute contexts, as well as a various scalable machine learning models, such as **rxLinMod**. For more information, see [revoscalepy module with SQL Server](../python/ref-py-revoscalepy.md). |
40-
| [**microsoftml**](/machine-learning-server/python-reference/microsoftml/microsoftml-package) | Contains machine learning algorithms that have been optimized for speed and accuracy, as well as in-line transformations for working with text and images. For more information, see [microsoftml module with SQL Server](../python/ref-py-microsoftml.md). |
40+
| [**microsoftml**](../python/ref-py-microsoftml.md) | Contains machine learning algorithms that have been optimized for speed and accuracy, as well as in-line transformations for working with text and images. For more information, see [microsoftml module with SQL Server](../python/ref-py-microsoftml.md). |
4141

4242
Microsoftml and revoscalepy are tightly coupled; data sources used in microsoftml are defined as revoscalepy objects. Compute context limitations in revoscalepy transfer to microsoftml. Namely, all functionality is available for local operations, but switching to a remote compute context requires RxInSqlServer.
4343

‎docs/machine-learning/install/sql-pretrained-models-install.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@ To use the pretrained models, call the functions listed in the following table.
2222

2323
| R function (MicrosoftML) | Python function (microsoftml) | Usage |
2424
|--------------------------|-------------------------------|-------|
25-
| [getSentiment](/machine-learning-server/r-reference/microsoftml/getsentiment) | [get_sentiment](/machine-learning-server/python-reference/microsoftml/get-sentiment) | Generates positive-negative sentiment score over text inputs. |
26-
| [featurizeImage](/machine-learning-server/r-reference/microsoftml/featurizeimage) | [featurize_image](/machine-learning-server/python-reference/microsoftml/featurize-image) | Extracts text information from image file inputs. |
25+
| [getSentiment](/machine-learning-server/r-reference/microsoftml/getsentiment) | [get_sentiment](../python/reference/microsoftml/get-sentiment.md) | Generates positive-negative sentiment score over text inputs. |
26+
| [featurizeImage](/machine-learning-server/r-reference/microsoftml/featurizeimage) | [featurize_image](../python/reference/microsoftml/featurize-image.md) | Extracts text information from image file inputs. |
2727

2828
## Prerequisites
2929

@@ -34,7 +34,7 @@ You must have administrator rights on the computer and SQL Server to add pre-tra
3434
External scripts must be enabled and SQL Server LaunchPad service must be running. Installation instructions provide the steps for enabling and verifying these capabilities.
3535

3636
::: moniker range=">=sql-server-2017"
37-
[MicrosoftML R package](/machine-learning-server/r-reference/microsoftml/microsoftml-package) or [microsoftml Python package](/machine-learning-server/python-reference/microsoftml/microsoftml-package) contain the pre-trained models.
37+
[MicrosoftML R package](/machine-learning-server/r-reference/microsoftml/microsoftml-package) or [microsoftml Python package](../python/ref-py-microsoftml.md) contain the pre-trained models.
3838

3939
[SQL Server Machine Learning Services](sql-machine-learning-services-windows-install.md) includes both language versions of the machine learning library, so this prerequisite is met with no further action on your part. Because the libraries are present, you can use the PowerShell script described in this article to add the pre-trained models to these libraries.
4040
::: moniker-end
@@ -211,4 +211,4 @@ For more information about the algorithms used in these deep learning models, an
211211
+ [SQL Server Machine Learning Services](sql-machine-learning-services-windows-install.md)
212212
+ [Upgrade R and Python components in SQL Server instances](../install/upgrade-r-and-python.md)
213213
+ [MicrosoftML package for R](/machine-learning-server/r-reference/microsoftml/microsoftml-package)
214-
+ [microsoftml package for Python](/machine-learning-server/python-reference/microsoftml/microsoftml-package)
214+
+ [microsoftml package for Python](../python/ref-py-microsoftml.md)

‎docs/machine-learning/install/upgrade-r-and-python.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ Microsoft R Open (MRO) over R | R 3.3.3 | R 3.4.3 | R 3.5.2 |
7979
[olapR](/machine-learning-server/r-reference/olapr/olapr) | 1.0 | 1.0 | 1.0 |
8080
Anaconda 4.2 over Python 3.5 | 4.2/3.5.2 | 4.2/3.5.2 |
8181
[revoscalepy](/machine-learning-server/python-reference/revoscalepy/revoscalepy-package) | 9.2 | 9.3| 9.4.7 |
82-
[microsoftml](/machine-learning-server/python-reference/microsoftml/microsoftml-package) | 9.2 | 9.3| 9.4.7 |
82+
[microsoftml](../python/ref-py-microsoftml.md) | 9.2 | 9.3| 9.4.7 |
8383
[pretrained models](/machine-learning-server/install/microsoftml-install-pretrained-models) | 9.2 | 9.3| 9.4.7 |
8484
::: moniker-end
8585

‎docs/machine-learning/package-management/python-package-information.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ The following Microsoft Python packages are installed with SQL Server Machine Le
8484
| Packages | Version | Description |
8585
| ---------|---------|--------------|
8686
| [revoscalepy](/machine-learning-server/python-reference/revoscalepy/revoscalepy-package) | 9.4.7 | Used for remote compute contexts, streaming, parallel execution of rx functions for data import and transformation, modeling, visualization, and analysis. |
87-
| [microsoftml](/machine-learning-server/python-reference/microsoftml/microsoftml-package) | 9.4.7 | Adds machine learning algorithms in Python. |
87+
| [microsoftml](../python/ref-py-microsoftml.md) | 9.4.7 | Adds machine learning algorithms in Python. |
8888

8989
For information on which version of Python is included, see [Python and R versions](../sql-server-machine-learning-services.md#versions).
9090

‎docs/machine-learning/python/ref-py-microsoftml.md‎

Lines changed: 41 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ title: microsoftml Python package
33
description: microsoftml is a Python package from Microsoft that provides high-performance machine learning algorithms. It includes functions for training and transformations, scoring, text and image analysis, and feature extraction for deriving values from existing data. The package is included in SQL Server Machine Learning Services.
44
ms.prod: sql
55
ms.technology: machine-learning-services
6-
ms.date: 07/14/2020
7-
ms.topic: how-to
6+
ms.date: 08/03/2021
7+
ms.topic: reference
88
author: garyericson
99
ms.author: garye
1010
monikerRange: ">=sql-server-2017||>=sql-server-linux-ver15"
@@ -14,13 +14,21 @@ monikerRange: ">=sql-server-2017||>=sql-server-linux-ver15"
1414

1515
**microsoftml** is a Python package from Microsoft that provides high-performance machine learning algorithms. It includes functions for training and transformations, scoring, text and image analysis, and feature extraction for deriving values from existing data. The package is included in [SQL Server Machine Learning Services](../sql-server-machine-learning-services.md) and supports high performance on big data, using multicore processing, and fast data streaming.
1616

17-
## Full reference documentation
17+
| Package details | Information |
18+
|-----------------------|-------------|
19+
| Current version: | 9.4 |
20+
| Built on: | [Anaconda 4.2](https://www.continuum.io/why-anaconda) distribution of [Python 3.7.1](https://www.python.org/doc) |
21+
| Package distribution: | [SQL Server Machine Learning Services](../sql-server-machine-learning-services.md) version 2017 or 2019. |
1822

19-
The **microsoftml** package is distributed in multiple Microsoft products, but usage is the same whether you get the package in SQL Server or another product. Because the functions are the same, [documentation for individual microsoftml functions](/machine-learning-server/python-reference/microsoftml/microsoftml-package) is published to just one location under the [Python reference](/machine-learning-server/python-reference/introducing-python-package-reference). Should any product-specific behaviors exist, discrepancies will be noted in the function help page.
23+
## How to use microsoftml
24+
25+
The **microsoftml** module is installed as part of SQL Server Machine Learning Services when you add Python to your installation. You get the full collection of proprietary packages plus a Python distribution with its modules and interpreters. You can use any Python IDE to write Python script calling functions in **microsoftml**, but the script must run on a computer having SQL Server Machine Learning Services with Python.
26+
27+
**Microsoftml** and **revoscalepy** are tightly coupled; data sources used in **microsoftml** are defined as [**revoscalepy**](/machine-learning-server/python-reference/revoscalepy/revoscalepy-package) objects. Compute context limitations in **revoscalepy** transfer to **microsoftml**. Namely, all functionality is available for local operations, but switching to a remote compute context requires [RxSpark](/machine-learning-server/python-reference/revoscalepy/rxspark) or [RxInSQLServer](/machine-learning-server/python-reference/revoscalepy/rxinsqlserver).
2028

2129
## Versions and platforms
2230

23-
The **microsoftml** module is based on Python 3.5 and available only when you install one of the following Microsoft products or downloads:
31+
The **microsoftml** module is available only when you install one of the following Microsoft products or downloads:
2432

2533
+ [SQL Server Machine Learning Services](../install/sql-machine-learning-services-windows-install.md)
2634
+ [Python client libraries for a data science client](setup-python-client-tools-sql.md)
@@ -32,26 +40,26 @@ The **microsoftml** module is based on Python 3.5 and available only when you in
3240

3341
Algorithms in **microsoftml** depend on [revoscalepy](ref-py-revoscalepy.md) for:
3442

35-
+ Data source objects. Data consumed by **microsoftml** functions are created using **revoscalepy** functions.
36-
+ Remote computing (shifting function execution to a remote SQL Server instance). The **revoscalepy** package provides functions for creating and activating a remote compute context for SQL server.
43+
+ Data source objects - Data consumed by **microsoftml** functions are created using **revoscalepy** functions.
44+
+ Remote computing (shifting function execution to a remote SQL Server instance) - The **revoscalepy** package provides functions for creating and activating a remote compute context for SQL server.
3745

3846
In most cases, you will load the packages together whenever you are using **microsoftml**.
3947

4048
## Functions by category
4149

42-
This section lists the functions by category to give you an idea of how each one is used. You can also use the [table of contents](/machine-learning-server/python-reference/introducing-python-package-reference) to find functions in alphabetical order.
50+
This section lists the functions by category to give you an idea of how each one is used. You can also use the table of contents to find functions in alphabetical order.
4351

4452
## 1-Training functions
4553

4654
| Function | Description |
4755
|----------|-------------|
48-
|[microsoftml.rx_ensemble](/machine-learning-server/python-reference/microsoftml/rx-ensemble) | Train an ensemble of models. |
49-
|[microsoftml.rx_fast_forest](/machine-learning-server/python-reference/microsoftml/rx-fast-forest) | Random Forest. |
50-
|[microsoftml.rx_fast_linear](/machine-learning-server/python-reference/microsoftml/rx-fast-linear) | Linear Model. with Stochastic Dual Coordinate Ascent. |
51-
|[microsoftml.rx_fast_trees](/machine-learning-server/python-reference/microsoftml/rx-fast-trees) | Boosted Trees. |
52-
|[microsoftml.rx_logistic_regression](/machine-learning-server/python-reference/microsoftml/rx-logistic-regression) | Logistic Regression. |
53-
|[microsoftml.rx_neural_network](/machine-learning-server/python-reference/microsoftml/rx-neural-network) | Neural Network. |
54-
|[microsoftml.rx_oneclass_svm](/machine-learning-server/python-reference/microsoftml/rx-oneclass-svm) | Anomaly Detection. |
56+
|[microsoftml.rx_ensemble](reference/microsoftml/rx-ensemble.md) | Train an ensemble of models. |
57+
|[microsoftml.rx_fast_forest](reference/microsoftml/rx-fast-forest.md) | Random Forest. |
58+
|[microsoftml.rx_fast_linear](reference/microsoftml/rx-fast-linear.md) | Linear Model. with Stochastic Dual Coordinate Ascent. |
59+
|[microsoftml.rx_fast_trees](reference/microsoftml/rx-fast-trees.md) | Boosted Trees. |
60+
|[microsoftml.rx_logistic_regression](reference/microsoftml/rx-logistic-regression.md) | Logistic Regression. |
61+
|[microsoftml.rx_neural_network](reference/microsoftml/rx-neural-network.md) | Neural Network. |
62+
|[microsoftml.rx_oneclass_svm](reference/microsoftml/rx-oneclass-svm.md) | Anomaly Detection. |
5563

5664
<a name="ml-transforms"></a>
5765

@@ -61,56 +69,53 @@ This section lists the functions by category to give you an idea of how each one
6169

6270
| Function | Description |
6371
|----------|-------------|
64-
|[microsoftml.categorical](/machine-learning-server/python-reference/microsoftml/categorical) | Converts a text column into categories. |
65-
|[microsoftml.categorical_hash](/machine-learning-server/python-reference/microsoftml/categorical-hash) | Hashes and converts a text column into categories. |
72+
|[microsoftml.categorical](reference/microsoftml/categorical.md) | Converts a text column into categories. |
73+
|[microsoftml.categorical_hash](reference/microsoftml/categorical-hash.md) | Hashes and converts a text column into categories. |
6674

6775
### Schema manipulation
6876

6977
| Function | Description |
7078
|----------|-------------|
71-
|[microsoftml.concat](/machine-learning-server/python-reference/microsoftml/concat) | Concatenates multiple columns into a single vector. |
72-
|[microsoftml.drop_columns](/machine-learning-server/python-reference/microsoftml/drop-columns) | Drops columns from a dataset. |
73-
|[microsoftml.select_columns](/machine-learning-server/python-reference/microsoftml/select-columns) | Retains columns of a dataset. |
74-
79+
|[microsoftml.concat](reference/microsoftml/concat.md) | Concatenates multiple columns into a single vector. |
80+
|[microsoftml.drop_columns](reference/microsoftml/drop-columns.md) | Drops columns from a dataset. |
81+
|[microsoftml.select_columns](reference/microsoftml/select-columns.md) | Retains columns of a dataset. |
7582

7683
### Variable selection
7784

7885
| Function | Description |
7986
|----------|-------------|
80-
|[microsoftml.count_select](/machine-learning-server/python-reference/microsoftml/count-select) |Feature selection based on counts. |
81-
|[microsoftml.mutualinformation_select](/machine-learning-server/python-reference/microsoftml/mutualinformation-select) | Feature selection based on mutual information. |
82-
87+
|[microsoftml.count_select](reference/microsoftml/count-select.md) |Feature selection based on counts. |
88+
|[microsoftml.mutualinformation_select](reference/microsoftml/mutualinformation-select.md) | Feature selection based on mutual information. |
8389

8490
### Text analytics
8591

8692
| Function | Description |
8793
|----------|-------------|
88-
|[microsoftml.featurize_text](/machine-learning-server/python-reference/microsoftml/featurize-text) | Converts text columns into numerical features. |
89-
|[microsoftml.get_sentiment](/machine-learning-server/python-reference/microsoftml/get-sentiment) | Sentiment analysis. |
90-
94+
|[microsoftml.featurize_text](reference/microsoftml/featurize-text.md) | Converts text columns into numerical features. |
95+
|[microsoftml.get_sentiment](reference/microsoftml/get-sentiment.md) | Sentiment analysis. |
9196

92-
### Image analytics
97+
### Image analytics
9398

9499
| Function | Description |
95100
|----------|-------------|
96-
|[microsoftml.load_image](/machine-learning-server/python-reference/microsoftml/load-image) | Loads an image. |
97-
|[microsoftml.resize_image](/machine-learning-server/python-reference/microsoftml/resize-image) | Resizes an Image. |
98-
|[microsoftml.extract_pixels](/machine-learning-server/python-reference/microsoftml/extract-pixels) | Extracts pixels from an image. |
99-
|[microsoftml.featurize_image](/machine-learning-server/python-reference/microsoftml/featurize-image) | Converts an image into features. |
101+
|[microsoftml.load_image](reference/microsoftml/load-image.md) | Loads an image. |
102+
|[microsoftml.resize_image](reference/microsoftml/resize-image.md) | Resizes an Image. |
103+
|[microsoftml.extract_pixels](reference/microsoftml/extract-pixels.md) | Extracts pixels from an image. |
104+
|[microsoftml.featurize_image](reference/microsoftml/featurize-image.md) | Converts an image into features. |
100105

101106
### Featurization functions
102107

103108
| Function | Description |
104109
|----------|-------------|
105-
|[microsoftml.rx_featurize](/machine-learning-server/python-reference/microsoftml/rx-featurize) | Data transformation for data sources |
110+
|[microsoftml.rx_featurize](reference/microsoftml/rx-featurize.md) | Data transformation for data sources |
106111

107112
<a name="ml-scoring"></a>
108113

109-
## 3-Scoring functions
114+
## Scoring functions
110115

111116
| Function | Description |
112117
|----------|-------------|
113-
|[microsoftml.rx_predict](/machine-learning-server/python-reference/microsoftml/rx-predict) | Scores using a Microsoft machine learning model |
118+
|[microsoftml.rx_predict](reference/microsoftml/rx-predict.md) | Scores using a Microsoft machine learning model |
114119

115120
## How to call microsoftml
116121

@@ -129,4 +134,4 @@ from revoscalepy.etl.RxImport import rx_import_datasource
129134
## See also
130135

131136
+ [Python tutorials](../tutorials/python-tutorials.md)
132-
+ [Python Reference](/machine-learning-server/python-reference/introducing-python-package-reference)
137+
+ [Manage Python packages](../package-management/python-package-information.md)

‎docs/machine-learning/python/ref-py-revoscalepy.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ SQL Server and Python use different data types in some cases. For a list of mapp
7777
|[rx_logit](/machine-learning-server/python-reference/revoscalepy/rx-logit) | Create a logistic regression model|
7878
|[rx_summary](/machine-learning-server/python-reference/revoscalepy/rx-summary) | Produce univariate summaries of objects in revoscalepy.|
7979

80-
You should also review the functions in [microsoftml](/machine-learning-server/python-reference/microsoftml/microsoftml-package) for additional approaches.
80+
You should also review the functions in [microsoftml](../python/ref-py-microsoftml.md) for additional approaches.
8181

8282
<a name="ml-scoring"></a>
8383

‎docs/machine-learning/python/reference/microsoftml/adadelta-optimizer.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
3+
# required metadata
4+
title: "adadelta_optimizer: adadelta_optimizer"
5+
description: "Adaptive learning rate method."
6+
keywords: "optimizer, adadelta"
7+
author: "garyericson"
8+
ms.author: "garye"
9+
manager: "cgronlun"
10+
ms.date: 07/15/2019
11+
ms.topic: "reference"
12+
ms.prod: "sql"
13+
ms.technology: "machine-learning-services"
14+
ms.service: ""
15+
ms.assetid: ""
16+
17+
# optional metadata
18+
ROBOTS: ""
19+
audience: ""
20+
ms.devlang: "Python"
21+
ms.reviewer: ""
22+
ms.suite: ""
23+
ms.tgt_pltfrm: ""
24+
ms.custom: ""
25+
monikerRange: ">=sql-server-2017||>=sql-server-linux-ver15"
26+
27+
---
28+
29+
# *microsoftml.adadelta_optimizer*: Adaptive learing rate method
30+
31+
32+
33+
34+
35+
## Usage
36+
37+
38+
39+
```
40+
microsoftml.adadelta_optimizer(decay: numbers.Real = 0.95,
41+
cond: numbers.Real = 1e-06)
42+
```
43+
44+
45+
46+
47+
48+
## Description
49+
50+
Adaptive learning rate method.
51+
52+
53+
## Arguments
54+
55+
56+
### decay
57+
58+
Decay rate (settings).
59+
60+
61+
### cond
62+
63+
Condition constant (settings).
64+
65+
66+
## See also
67+
68+
[`sgd_optimizer`](sgd-optimizer.md)

‎docs/machine-learning/python/reference/microsoftml/avx-math.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
---
2+
3+
# required metadata
4+
title: "avx_math: avx_math"
5+
description: "Implementation accelerated with AVX instructions."
6+
keywords: "neural network, math, avx"
7+
author: "garyericson"
8+
ms.author: "garye"
9+
manager: "cgronlun"
10+
ms.date: 07/15/2019
11+
ms.topic: "reference"
12+
ms.prod: "sql"
13+
ms.technology: "machine-learning-services"
14+
ms.service: ""
15+
ms.assetid: ""
16+
17+
# optional metadata
18+
ROBOTS: ""
19+
audience: ""
20+
ms.devlang: "Python"
21+
ms.reviewer: ""
22+
ms.suite: ""
23+
ms.tgt_pltfrm: ""
24+
ms.custom: ""
25+
monikerRange: ">=sql-server-2017||>=sql-server-linux-ver15"
26+
27+
---
28+
29+
# *microsoftml.avx_math*: Acceleration with AVX instructions
30+
31+
32+
33+
34+
35+
## Usage
36+
37+
38+
39+
```
40+
microsoftml.avx_math()
41+
```
42+
43+
44+
45+
46+
47+
## Description
48+
49+
Implementation accelerated with AVX instructions.
50+
51+
52+
## See also
53+
54+
[`clr_math`](clr-math.md),
55+
[`gpu_math`](gpu-math.md),
56+
[`mkl_math`](mkl-math.md),
57+
[`sse_math`](sse-math.md)

0 commit comments

Comments
 (0)