Skip to content

Validate the hidden size of GRU and LSTM operators #644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fdwr merged 10 commits into from
May 1, 2024
Merged

Validate the hidden size of GRU and LSTM operators #644

Changes from 4 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
7fe0e79
Validate the maximum hidden size of GRU and LSTM operators
inexorabletash Apr 17, 2024
4c2de1a
Merge remote-tracking branch 'origin/main' into validate-hidden-size-…
inexorabletash Apr 18, 2024
dda2735
Merge remote-tracking branch 'origin/main' into validate-hidden-size-…
inexorabletash Apr 19, 2024
70d5a4d
Merge remote-tracking branch 'origin/main' into validate-hidden-size-…
inexorabletash Apr 24, 2024
6ebab8a
Merge remote-tracking branch 'origin/main' into validate-hidden-size-…
inexorabletash Apr 25, 2024
7ace80c
Merge remote-tracking branch 'origin/main' into validate-hidden-size-…
inexorabletash Apr 25, 2024
788009a
Merge branch 'refs/heads/review' into validate-hidden-size-multiples
inexorabletash Apr 27, 2024
248e58c
Merge remote-tracking branch 'origin/main' into validate-hidden-size-…
inexorabletash Apr 28, 2024
98e3892
fix merge residue
inexorabletash Apr 28, 2024
9feeefb
Merge branch 'refs/heads/review' into validate-hidden-size-multiples
inexorabletash Apr 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3073,6 +3073,11 @@ partial interface MLGraphBuilder {
1. If [=MLGraphBuilder/validating operand=] with [=this=] and any of |input|, |weight|, |recurrentWeight|, |options|.{{MLGruOptions/bias}} (if it [=map/exists=]), |options|.{{MLGruOptions/recurrentBias}} (if it [=map/exists=]), and |options|.{{MLGruOptions/initialHiddenState}} (if it [=map/exists=]) returns false, then [=exception/throw=] a {{TypeError}}.
1. If |options|.{{MLGruOptions/activations}} [=map/exists=], and [=MLGraphBuilder/validating activation=] with [=this=] and any [=list/item=] in it returns false, then [=exception/throw=] a {{TypeError}}.
1. If the [=MLOperand/rank=] of any of |input|, |weight| or |recurrentWeight| is not 3, then [=exception/throw=] a {{TypeError}}.
1. If |hiddenSize| * 6 is not a [=valid dimension=], then [=exception/throw=] a {{TypeError}}.
<details class=note>
<summary>Why |hiddenSize| * 6 ?</summary>
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLGruOptions/bias}} and {{MLGruOptions/recurrentBias}}. Therefore, 3 * |hiddenSize| + 3 * |hiddenSize| must also be a [=valid dimension=].
</details>
1. If |options|.{{MLGruOptions/bias}} [=map/exists=]:
1. If its [=MLOperand/shape=][1] is not equal to 3 * |hiddenSize|, then [=exception/throw=] a {{TypeError}}.
1. If |options|.{{MLGruOptions/recurrentBias}} [=map/exists=]:
Expand Down Expand Up @@ -3241,6 +3246,11 @@ partial interface MLGraphBuilder {
1. If [=MLGraphBuilder/validating operand=] with [=this=] and any of |input|, |weight|, |recurrentWeight|, |hiddenState|, |options|.{{MLGruCellOptions/bias}} (if it [=map/exists=]), and |options|.{{MLGruCellOptions/recurrentBias}} (if it [=map/exists=]) returns false, then [=exception/throw=] a {{TypeError}}.
1. If |options|.{{MLGruCellOptions/activations}} [=map/exists=], and [=MLGraphBuilder/validating activation=] with [=this=] and any [=list/item=] in it returns false, then [=exception/throw=] a {{TypeError}}.
1. If the [=MLOperand/rank=] of any of |input|, |weight|, |recurrentWeight| or |hiddenState| is not 2, then [=exception/throw=] a {{TypeError}}.
1. If |hiddenSize| * 6 is not a [=valid dimension=], then [=exception/throw=] a {{TypeError}}.
<details class=note>
<summary>Why |hiddenSize| * 6 ?</summary>
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLGruCellOptions/bias}} and {{MLGruCellOptions/recurrentBias}}. Therefore, 3 * |hiddenSize| + 3 * |hiddenSize| must also be a [=valid dimension=].
</details>
1. If |weight|'s [=MLOperand/shape=][0] is not equal to 3 * |hiddenSize|, then [=exception/throw=] a {{TypeError}}.
1. If |recurrentWeight|'s [=MLOperand/shape=][0] is not equal to 3 * |hiddenSize|, then [=exception/throw=] a {{TypeError}}.
1. If |options|.{{MLGruOptions/bias}} [=map/exists=]:
Expand Down Expand Up @@ -3964,6 +3974,11 @@ partial interface MLGraphBuilder {
1. Let |numDirections| be 1 if |options|.{{MLLstmOptions/direction}} is {{MLRecurrentNetworkDirection/"forward"}}, or otherwise let it be 2.
1. If the [=MLOperand/rank=] of any of |input|, |weight| or |recurrentWeight| is not 3, then [=exception/throw=] a {{TypeError}}.
1. If |input|'s [=MLOperand/shape=][0] is not equal to |steps|, then [=exception/throw=] a {{TypeError}}.
1. If |hiddenSize| * 8 is not a [=valid dimension=], then [=exception/throw=] a {{TypeError}}.
<details class=note>
<summary>Why |hiddenSize| * 8 ?</summary>
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLLstmOptions/bias}} and {{MLLstmOptions/recurrentBias}}. Therefore, 4 * |hiddenSize| + 4 * |hiddenSize| must also be a [=valid dimension=].
</details>
1. Let |batchSize| be |input|'s [=MLOperand/shape=][1].
1. If |options|.{{MLLstmOptions/bias}} [=map/exists=]:
1. If its [=MLOperand/rank=] is not 2, then [=exception/throw=] a {{TypeError}}.
Expand Down Expand Up @@ -4169,6 +4184,11 @@ partial interface MLGraphBuilder {
1. If [=MLGraphBuilder/validating operand=] with [=this=] and any of |input|, |weight|, |recurrentWeight|, |hiddenState|, |cellState|, |options|.{{MLLstmCellOptions/bias}} (if it [=map/exists=]), |options|.{{MLLstmCellOptions/recurrentBias}} (if it [=map/exists=]), and |options|.{{MLLstmCellOptions/peepholeWeight}} (if it [=map/exists=]) returns false, then [=exception/throw=] a {{TypeError}}.
1. If |options|.{{MLLstmCellOptions/activations}} [=map/exists=], and [=MLGraphBuilder/validating activation=] with [=this=] and any [=list/item=] in it returns false, then [=exception/throw=] a {{TypeError}}.
1. If the [=MLOperand/rank=] of any of |input|, |weight|, |recurrentWeight|, |hiddenState| or |cellState| is not 2, then [=exception/throw=] a {{TypeError}}.
1. If |hiddenSize| * 8 is not a [=valid dimension=], then [=exception/throw=] a {{TypeError}}.
<details class=note>
<summary>Why |hiddenSize| * 8 ?</summary>
Some underlying platforms operate on a single bias tensor which is a concatenation of {{MLLstmCellOptions/bias}} and {{MLLstmCellOptions/recurrentBias}}. Therefore, 4 * |hiddenSize| + 4 * |hiddenSize| must also be a [=valid dimension=].
</details>
1. Let |batchSize| be |input|'s [=MLOperand/shape=][0].
1. If |options|.{{MLLstmCellOptions/bias}} [=map/exists=]:
1. If its [=MLOperand/rank=] is not 1, then [=exception/throw=] a {{TypeError}}.
Expand Down