Skip to content

Commit 252f36f

Browse files
author
Deshui Yu
committed
NNI dogfood version 1
1 parent 781cea2 commit 252f36f

214 files changed

Lines changed: 33151 additions & 10 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitattributes‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
*.ts text eol=lf
2+
*.py text eol=lf
3+
*.sh text eol=lf

‎Makefile‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
BIN_PATH ?= /usr/bin
2+
NODE_PATH ?= /usr/share
3+
EXAMPLE_PATH ?= /usr/share/nni/examples
4+
5+
SRC_DIR := ${PWD}
6+
7+
.PHONY: build install uninstall
8+
9+
build:
10+
### Building NNI Manager ###
11+
cd src/nni_manager && yarn && yarn build
12+
13+
### Building Web UI ###
14+
cd src/webui && yarn && yarn build
15+
16+
### Building Python SDK ###
17+
cd src/sdk/pynni && python3 setup.py build
18+
19+
### Building nnictl ###
20+
cd tools && python3 setup.py build
21+
22+
23+
install:
24+
mkdir -p $(NODE_PATH)/nni
25+
mkdir -p $(EXAMPLE_PATH)
26+
27+
### Installing NNI Manager ###
28+
cp -rT src/nni_manager/dist $(NODE_PATH)/nni/nni_manager
29+
cp -rT src/nni_manager/node_modules $(NODE_PATH)/nni/nni_manager/node_modules
30+
31+
### Installing Web UI ###
32+
cp -rT src/webui/build $(NODE_PATH)/nni/webui
33+
ln -sf $(NODE_PATH)/nni/nni_manager/node_modules/serve/bin/serve.js $(BIN_PATH)/serve
34+
35+
### Installing Python SDK dependencies ###
36+
pip3 install -r src/sdk/pynni/requirements.txt
37+
### Installing Python SDK ###
38+
cd src/sdk/pynni && python3 setup.py install
39+
40+
### Installing nnictl ###
41+
cd tools && python3 setup.py install
42+
43+
echo '#!/bin/sh' > $(BIN_PATH)/nnimanager
44+
echo 'cd $(NODE_PATH)/nni/nni_manager && node main.js $$@' >> $(BIN_PATH)/nnimanager
45+
chmod +x $(BIN_PATH)/nnimanager
46+
47+
install -m 755 tools/nnictl $(BIN_PATH)/nnictl
48+
49+
### Installing examples ###
50+
cp -rT examples $(EXAMPLE_PATH)
51+
52+
53+
dev-install:
54+
### Installing Python SDK dependencies ###
55+
pip3 install --user -r src/sdk/pynni/requirements.txt
56+
### Installing Python SDK ###
57+
cd src/sdk/pynni && pip3 install --user -e .
58+
59+
### Installing nnictl ###
60+
cd tools && pip3 install --user -e .
61+
62+
63+
uninstall:
64+
-rm -r $(EXAMPLE_PATH)
65+
-rm -r $(NODE_PATH)/nni
66+
-pip3 uninstall -y nnictl
67+
-pip3 uninstall -y nni
68+
-rm $(BIN_PATH)/nnictl
69+
-rm $(BIN_PATH)/nnimanager
70+
-rm $(BIN_PATH)/serve

‎README.md‎

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,21 @@
1+
# Introduction
2+
Neural Network Intelligence(NNI) is a light package for supporting hyper-parameter tuning or neural architecture search.
3+
It could easily run in different environments, such as: local/remote machine/cloud.
4+
And it offers a new annotation language for user to conveniently design search space.
5+
Also user could write code using any language or any machine learning framework.
16

2-
# Contributing
7+
# Getting Started
8+
TODO: Guide users through getting your code up and running on their own system. In this section you can talk about:
9+
1. Installation process
10+
2. Software dependencies
11+
3. Latest releases
12+
4. API references
313

4-
This project welcomes contributions and suggestions. Most contributions require you to agree to a
5-
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
6-
the rights to use your contribution. For details, visit https://cla.microsoft.com.
14+
# Build and Test
15+
TODO: Describe and show how to build your code and run the tests.
716

8-
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide
9-
a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions
10-
provided by the bot. You will only need to do this once across all repos using our CLA.
17+
# Contribute
18+
TODO: Explain how other users and developers can contribute to make your code better.
1119

12-
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
13-
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
14-
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
20+
# Privacy Statement
21+
The [Microsoft Enterprise and Developer Privacy Statement](https://privacy.microsoft.com/en-us/privacystatement) describes the privacy statement of this software.

‎docs/EnableAssessor.md‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
**Enable Assessor in your expeirment**
2+
===
3+
Assessor module is for assessing running trials. One common use case is early stopping, which terminates unpromising trial jobs based on their intermediate results.
4+
5+
## Using NNI built-in Assessor
6+
Here we use the same example `examples/trials/mnist-annotation`. We use `Medianstop` assessor for this experiment. The yaml configure file is shown below:
7+
```
8+
authorName: your_name
9+
experimentName: auto_mnist
10+
# how many trials could be concurrently running
11+
trialConcurrency: 2
12+
# maximum experiment running duration
13+
maxExecDuration: 3h
14+
# empty means never stop
15+
maxTrialNum: 100
16+
# choice: local, remote
17+
trainingServicePlatform: local
18+
# choice: true, false
19+
useAnnotation: true
20+
tuner:
21+
tunerName: TPE
22+
optimizationMode: Maximize
23+
assessor:
24+
assessorName: Medianstop
25+
optimizationMode: Maximize
26+
trial:
27+
trialCommand: python mnist.py
28+
trialCodeDir: /usr/share/nni/examples/trials/mnist-annotation
29+
trialGpuNum: 0
30+
```
31+
For our built-in assessors, you need to fill two fields: `assessorName` which chooses NNI provided assessors (refer to [here]() for built-in assessors), `optimizationMode` which includes Maximize and Minimize (you want to maximize or minimize your trial result).
32+
33+
## Using user customized Assessor
34+
You can also write your own assessor following the guidance [here](). For example, you wrote an assessor for `examples/trials/mnist-annotation`. You should prepare the yaml configure below:
35+
```
36+
authorName: your_name
37+
experimentName: auto_mnist
38+
# how many trials could be concurrently running
39+
trialConcurrency: 2
40+
# maximum experiment running duration
41+
maxExecDuration: 3h
42+
# empty means never stop
43+
maxTrialNum: 100
44+
# choice: local, remote
45+
trainingServicePlatform: local
46+
# choice: true, false
47+
useAnnotation: true
48+
tuner:
49+
tunerName: TPE
50+
optimizationMode: Maximize
51+
assessor:
52+
assessorCommand: your_command
53+
assessorCodeDir: /path/of/your/asessor
54+
assessorGpuNum: 0
55+
trial:
56+
trialCommand: python mnist.py
57+
trialCodeDir: /usr/share/nni/examples/trials/mnist-annotation
58+
trialGpuNum: 0
59+
```
60+
You only need to fill three field: `assessorCommand`, `assessorCodeDir` and `assessorGpuNum`.

0 commit comments

Comments
 (0)