Skip to content

Commit ad35067

Browse files
yangzhgyangzhg
authored
[chore][docs] add deploy spark/flink connectors to maven release repo docs (#7616)
1 parent 482bf05 commit ad35067

10 files changed

Lines changed: 188 additions & 24 deletions

File tree

‎docs/.vuepress/public/images/close_staging.png‎

191 KB
Loading

‎docs/.vuepress/public/images/release-stage.png‎

219 KB
Loading

‎docs/.vuepress/public/images/staging_repo.png‎

383 KB
Loading

‎docs/en/community/how-to-deploy-to-maven.md‎

Lines changed: 85 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ under the License.
7575
<activeProfile>gpg</activeProfile>
7676
</activeProfiles>
7777
```
78-
### Publish to SNAPSHOT
78+
## Publish to SNAPSHOT
7979
### 1. Deploy flink connector
8080

8181
Switch to the flink connector directory, let’s take flink version 1.11.6 and scalar 2.12 as examples
@@ -105,3 +105,87 @@ Switch to the spark connector directory, let’s take spark version 2.3.4 and sc
105105
export SCALA_VERSION="2.11"
106106
mvn deploy
107107
```
108+
## Publish to Release
109+
110+
### 1. Prepare GitHub authority
111+
Encrypt github token password: `mvn --encrypt-password <token>` This token is a token generated by github users to access the apache warehouse. The output cat is similar to `{bSSA+TC6wzEHNKukcOn...}` in `~/.m2/settings. Add to the `<servers>` tag in xml`
112+
```xml
113+
<server>
114+
<id>github</id>
115+
<username>github user name</username>
116+
<password>{bSSA+TC6wzEHNKukcOn...}</password>
117+
</server>
118+
```
119+
120+
### 2. Publish to maven staging
121+
Take the release of Doris Flink Connector 1.0.0 as an example, the flink version is 1.13.5, the scala version is 2.12, and others are similar
122+
```bash
123+
cd extension/flink-doris-connector/
124+
export DORIS_HOME=$PWD/../../
125+
source ${DORIS_HOME}/env.sh
126+
export FLINK_VERSION=1.13.5
127+
export SCALA_VERSION=2.12
128+
mvn release:clean
129+
mvn release:prepare
130+
```
131+
After that, maven needs to enter three information
132+
1. The version information of Doris Flink Connector, we can do it by default, you can directly press Enter or enter the version you want
133+
2. The release tag of Doris Flink Connector, because the release process will add a tag in apache/incubator-doris, so a tag name is required. If it is default, just press Enter.
134+
3. The version number of the next version of Doris Flink Connector. Here we need to pay attention. Since our version number is {flink_version}_{scala_version}_1.0.0, maven's algorithm will increase the scala version number by one, if we want to use it next time This version of 1.0.1 can be changed to 1.13.5-2.12-1.0.1-SNAPSHOT
135+
```
136+
...
137+
[INFO] [prepare] 3/17 check-dependency-snapshots
138+
[INFO] Checking dependencies and plugins for snapshots ...
139+
[INFO] [prepare] 4/17 create-backup-poms
140+
[INFO] [prepare] 5/17 map-release-versions
141+
What is the release version for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) 1.13.5-2.12-1.0.0: :
142+
[INFO] [prepare] 6/17 input-variables
143+
What is the SCM release tag or label for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) doris-flink-connector-1.13.5-2.12-1.0.0: :
144+
[INFO] [prepare] 7/17 map-development-versions
145+
What is the new development version for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) 1.13.5-3.12-1.0.0-SNAPSHOT: : 1.13.5-2.12-1.0.1-SNAPSHOT
146+
...
147+
[INFO] [prepare] 17/17 end-release
148+
[INFO] Release preparation complete.
149+
[INFO] ------------------------------------------------------------------------
150+
[INFO] BUILD SUCCESS
151+
[INFO] ------------------------------------------------------------------------
152+
[INFO] Total time: 01:00 min
153+
[INFO] Finished at: 2022-01-05T15:01:55+08:00
154+
[INFO] ------------------------------------------------------------------------
155+
```
156+
Perform after prepare, and after perform, find the just released version in stagingRepositories in `https://repository.apache.org/`
157+
![](/images/staging_repo.png)
158+
159+
### 3. vote
160+
You need to close staging before voting
161+
![](/images/close_staging.png)
162+
After that, you can vote in the dev@doris.apache.org, the following is an example of the mail
163+
```
164+
Hi,
165+
166+
We are ready to deploy Doris connectors to Maven Central Repository, as the version 1.0.0, and the next version will be 1.0.1
167+
This Release contains Spark connectors and Flink connectors for spark 2/3 and flink 1.11/1.12/1.13
168+
169+
GitHub tags:
170+
https://github.com/apache/incubator-doris/releases/tag/doris-spark-connector-3.1.2-2.12-1.0.0
171+
https://github.com/apache/incubator-doris/releases/tag/doris-spark-connector-2.3.4-2.11-1.0.0
172+
https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.13.5-2.12-1.0.0
173+
https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.12.7-2.12-1.0.0
174+
https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.11.6-2.12-1.0.0
175+
176+
Staging repo:
177+
https://repository.apache.org/content/repositories/maven-[YOUR REPOSITORY ID]/
178+
https://repository.apache.org/content/repositories/maven-[YOUR REPOSITORY ID]/[PATH-TO]-source-release.zip
179+
180+
Staging site:
181+
https://repository.apache.org/content/repositories/orgapachedoris-1000
182+
183+
Vote open for at least 72 hours.
184+
185+
[ ] +1
186+
[ ] +0
187+
[ ] -1
188+
```
189+
190+
After the vote is passed, it can be published to Maven Central
191+
![](/images/release-stage.png)

‎docs/zh-CN/community/how-to-deploy-to-maven.md‎

Lines changed: 83 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ under the License.
7777
<activeProfile>gpg</activeProfile>
7878
</activeProfiles>
7979
```
80-
### 发布到SNAPSHOT
80+
## 发布到SNAPSHOT
8181
### 1. 部署 flink connector
8282

8383
切换到 flink connector 目录， 我们以 flink 版本 1.11.6， scalar 2.12 为例
@@ -107,3 +107,85 @@ under the License.
107107
export SCALA_VERSION="2.11"
108108
mvn deploy
109109
```
110+
111+
## 发布到 Release
112+
113+
### 1. 准备GitHub权限
114+
加密 github token 密码： `mvn --encrypt-password <token>` 这个token 是github 用户生成的访问apache仓库的token 输出cat类似`{bSSA+TC6wzEHNKukcOn...}` 在`~/.m2/settings.xml` 中的`<servers>` 标签中加入
115+
116+
```xml
117+
<server>
118+
<id>github</id>
119+
<username>github user name</username>
120+
<password>{bSSA+TC6wzEHNKukcOn...}</password>
121+
</server>
122+
```
123+
124+
### 2. 发布到 maven staging
125+
以发布Doris Flink Connector 1.0.0 为例， flink 版本是 1.13.5 scala 版本是2.12, 其他类似
126+
```bash
127+
cd extension/flink-doris-connector/
128+
export DORIS_HOME=$PWD/../../
129+
source ${DORIS_HOME}/env.sh
130+
export FLINK_VERSION=1.13.5
131+
export SCALA_VERSION=2.12
132+
mvn release:clean
133+
mvn release:prepare
134+
```
135+
之后maven 需要输入三个信息
136+
1. Doris Flink Connector 的版本信息， 我们默认就可以，可以直接回车或者输入自己想要的版本
137+
2. Doris Flink Connector 的release tag, 因为release 过程会在apache/incubator-doris中添加一个tag, 因此需要一个tag名称，如果默认，直接回车即可
138+
3. Doris Flink Connector 下一个版本的版本号，这里需要注意，由于我们的版本号是 {flink_version}_{scala_version}_1.0.0 因此 maven 的算法会将scala 版本号加一，如果我们下次想用1.0.1 这个版本 我们可以改成1.13.5-2.12-1.0.1-SNAPSHOT
139+
```
140+
...
141+
[INFO] [prepare] 3/17 check-dependency-snapshots
142+
[INFO] Checking dependencies and plugins for snapshots ...
143+
[INFO] [prepare] 4/17 create-backup-poms
144+
[INFO] [prepare] 5/17 map-release-versions
145+
What is the release version for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) 1.13.5-2.12-1.0.0: :
146+
[INFO] [prepare] 6/17 input-variables
147+
What is the SCM release tag or label for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) doris-flink-connector-1.13.5-2.12-1.0.0: :
148+
[INFO] [prepare] 7/17 map-development-versions
149+
What is the new development version for "Doris Flink Connector"? (org.apache.doris:doris-flink-connector) 1.13.5-3.12-1.0.0-SNAPSHOT: : 1.13.5-2.12-1.0.1-SNAPSHOT
150+
...
151+
[INFO] [prepare] 17/17 end-release
152+
[INFO] Release preparation complete.
153+
[INFO] ------------------------------------------------------------------------
154+
[INFO] BUILD SUCCESS
155+
[INFO] ------------------------------------------------------------------------
156+
[INFO] Total time: 01:00 min
157+
[INFO] Finished at: 2022-01-05T15:01:55+08:00
158+
[INFO] ------------------------------------------------------------------------
159+
```
160+
prepare 之后 perform即可， perform 之后就在`https://repository.apache.org/` 里面的stagingRepositories 找到刚刚发布的版本
161+
![](/images/staging_repo.png)
162+
163+
### 3. 投票
164+
投票前需要先close staging
165+
![](/images/close_staging.png)
166+
之后就可以在dev邮件组发起投票， 下面是一个邮件示例
167+
```
168+
Hi,
169+
170+
We are ready to deploy Doris connectors to Maven Central Repository, as the version 1.0.0, and the next version will be 1.0.1
171+
This Release contains Spark connectors and Flink connectors for spark 2/3 and flink 1.11/1.12/1.13
172+
173+
GitHub tags:
174+
https://github.com/apache/incubator-doris/releases/tag/doris-spark-connector-3.1.2-2.12-1.0.0
175+
https://github.com/apache/incubator-doris/releases/tag/doris-spark-connector-2.3.4-2.11-1.0.0
176+
https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.13.5-2.12-1.0.0
177+
https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.12.7-2.12-1.0.0
178+
https://github.com/apache/incubator-doris/releases/tag/doris-flink-connector-1.11.6-2.12-1.0.0
179+
180+
Staging repo:
181+
https://repository.apache.org/content/repositories/orgapachedoris-1000
182+
183+
Vote open for at least 72 hours.
184+
185+
[ ] +1
186+
[ ] +0
187+
[ ] -1
188+
```
189+
190+
投票通过后就可以发布到 Maven Central 了
191+
![](/images/release-stage.png)

‎extension/flink-doris-connector/pom.xml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,9 @@ under the License.
4141
</license>
4242
</licenses>
4343
<scm>
44-
<connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
45-
<developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
46-
<url>scm:git:git@github.com:apache/incubator-doris.git</url>
44+
<connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
45+
<developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
46+
<url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
4747
<tag>HEAD</tag>
4848
</scm>
4949
<issueManagement>
@@ -76,6 +76,7 @@ under the License.
7676
<maven-source-plugin.version>3.2.1</maven-source-plugin.version>
7777
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
7878
<doris.thirdparty>${env.DORIS_THIRDPARTY}</doris.thirdparty>
79+
<project.scm.id>github</project.scm.id>
7980
</properties>
8081

8182
<profiles>

‎extension/spark-doris-connector/pom.xml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,9 @@
4141
</license>
4242
</licenses>
4343
<scm>
44-
<connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
45-
<developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
46-
<url>scm:git:git@github.com:apache/incubator-doris.git</url>
44+
<connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
45+
<developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
46+
<url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
4747
<tag>HEAD</tag>
4848
</scm>
4949
<issueManagement>
@@ -77,6 +77,7 @@
7777
<maven-source-plugin.version>3.2.1</maven-source-plugin.version>
7878
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
7979
<doris.thirdparty>${env.DORIS_THIRDPARTY}</doris.thirdparty>
80+
<project.scm.id>github</project.scm.id>
8081
</properties>
8182

8283
<profiles>

‎fe/pom.xml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,9 +42,9 @@ under the License.
4242
</license>
4343
</licenses>
4444
<scm>
45-
<connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
46-
<developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
47-
<url>scm:git:git@github.com:apache/incubator-doris.git</url>
45+
<connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
46+
<developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
47+
<url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
4848
<tag>HEAD</tag>
4949
</scm>
5050
<issueManagement>
@@ -87,6 +87,7 @@ under the License.
8787
<jmockit.version>1.49</jmockit.version>
8888
<log4j2.version>2.17.1</log4j2.version>
8989
<revision>0.15-SNAPSHOT</revision>
90+
<project.scm.id>github</project.scm.id>
9091
</properties>
9192

9293
<profiles>

‎fe_plugins/pom.xml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,9 +42,9 @@ under the License.
4242
</license>
4343
</licenses>
4444
<scm>
45-
<connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
46-
<developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
47-
<url>scm:git:git@github.com:apache/incubator-doris.git</url>
45+
<connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
46+
<developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
47+
<url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
4848
<tag>HEAD</tag>
4949
</scm>
5050
<issueManagement>
@@ -75,6 +75,7 @@ under the License.
7575
<properties>
7676
<log4j2.version>2.17.1</log4j2.version>
7777
<doris.version>0.15-SNAPSHOT</doris.version>
78+
<project.scm.id>github</project.scm.id>
7879
</properties>
7980
<profiles>
8081
<!-- for custom internal repository -->

‎fs_brokers/apache_hdfs_broker/pom.xml‎

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,9 @@ under the License.
4343
</license>
4444
</licenses>
4545
<scm>
46-
<connection>scm:git:git@github.com:apache/incubator-doris.git</connection>
47-
<developerConnection>scm:git:git@github.com:apache/incubator-doris.git</developerConnection>
48-
<url>scm:git:git@github.com:apache/incubator-doris.git</url>
46+
<connection>scm:git:https://git@github.com/apache/incubator-doris.git</connection>
47+
<developerConnection>scm:git:https://git@github.com/apache/incubator-doris.git</developerConnection>
48+
<url>scm:git:https://git@github.com/apache/incubator-doris.git</url>
4949
<tag>HEAD</tag>
5050
</scm>
5151
<issueManagement>
@@ -74,6 +74,7 @@ under the License.
7474
<maven.compiler.source>1.8</maven.compiler.source>
7575
<maven.compiler.target>1.8</maven.compiler.target>
7676
<log4j2.version>2.17.1</log4j2.version>
77+
<project.scm.id>github</project.scm.id>
7778
</properties>
7879

7980
<profiles>
@@ -254,13 +255,6 @@ under the License.
254255
<version>0.13.0</version>
255256
</dependency>
256257

257-
<!-- https://mvnrepository.com/artifact/log4j/log4j -->
258-
<dependency>
259-
<groupId>log4j</groupId>
260-
<artifactId>log4j</artifactId>
261-
<version>1.2.17</version>
262-
</dependency>
263-
264258
<!-- https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api -->
265259
<dependency>
266260
<groupId>org.apache.logging.log4j</groupId>

0 commit comments

Comments
 (0)