> [!IMPORTANT]
> All scripts/examples assume you are inside the repository root folder.

# Generate a repository containing GAPIC Client Libraries

Running the docker image built from `hermetic_build/library_generation`
directory, you can generate a repository containing GAPIC client libraries (a
monorepo, for example, google-cloud-java) from a configuration file.

## Environment

- OS: Linux
- Docker

## Prerequisites

### Generation configuration

A generation configuration file is required to generate GAPIC Client Libraries.

Please refer to [Generation configuration](#configuration-to-generate-a-repository)
for more information.

### versions.txt

In order to generate a version for each library, a `versions.txt` has to exist
in `repository-path`.
Please refer to [Repository path](#repository-path-repository-path-optional)
for more information.

## Parameters to generate a repository using the docker image

### Generation configuration yaml (`generation-config-path`), optional

An absolute or relative path to a configuration file containing parameters to
generate the repository.
Please refer to [Configuration to generate a repository](#configuration-to-generate-a-repository)
for more information.

The default value is `$(pwd)/generation_config.yaml`, i.e., `generation_config.yaml`
in the current working directory.

This will raise `FileNotFoundError` if the specified generation config does not
exist or, in case `generation-config-path` is not specified, the default
generation config does not exist.

### Repository path (`repository-path`), optional

The path to where the generated repository goes.

The default value is the current working directory when running the script.
For example, `cd google-cloud-java && python /path/to/entry_point.py ...` without
specifying the `--repository_path` option will modify the `google-cloud-java`
repository the user `cd`'d into.

Note that `versions.txt` has to exist in `repository_path` in order to generate
the right version for each library.
Please refer go/java-client-releasing#versionstxt-manifest for more info
of versions.txt.

### A list of library names (`library-names`), optional

A list of library names that will be generated, separated by comma.
The library name of a library is the value of `library_name` or `api_shortname`,
if `library_name` is not specified, in the generation configuration.

If not specified, all libraries in the generation
configuration will be generated.

### Api definitions path (`api-definitions-path`), optional

The path to where the api definition (proto, service yaml) resides.

The default value is the current working directory when running the script.

**Note that you need not only the protos defined the service, but also the
transitive dependencies of those protos.**
Any missing dependencies will cause `File not found` error.

For example, if your service is defined in `example_service.proto` and it
imports `google/api/annotations.proto`, you need the `annotations.proto` resides
in a folder that has the exact structure of the import statement (`google/api`
in this case), and set `api_definitions_path` to the path contains the root
folder (`google` in this case).

## Output

### GAPIC libraries

For each module (e.g. `google-cloud-java/java-asset`), the following
files/folders will be created/modified:

| Name                                | Notes                                                                    |
|:------------------------------------|:-------------------------------------------------------------------------|
| google-*/                           | Source code generated by gapic-generator-java                            |
| google-*/pom.xml                    | Only be generated if it does not exist                                   |
| grpc-*/                             | Source code generated by grpc generator, one per each version            |
| grpc-*/pom.xml                      | Only be generated if it does not exist                                   |
| proto-*/                            | Source code generated by Protobuf default compiler, one per each version |
| proto-*/pom.xml                     | Only be generated if it does not exist                                   |
| samples/snippets/generated/         | Only be generated if `include_samples` is set to true                    |
| google-*-bom/pom.xml                | Library BOM, only be generated if it does not exist                      |
| pom.xml                             | Library parent BOM, only be generated if it does not exist               |
| .repo-metadata.json                 | Always generated from inputs                                             |
| .OwlBot-hermetic.yaml               | Only be generated from a template if it does not exist                   |
| owlbot.py                           | Only be generated from a template if it does not exist                   |
| README.md                           | Always generated from inputs                                             |
| gapic-libraries-bom/pom.xml         | Always generated from inputs                                             |
| pom.xml (repo root dir)             | Always generated from inputs                                             |
| versions.txt                        | New entries will be added if they don’t exist                            |

## Configuration to generate a repository

There are three levels of parameters in the configuration: repository level,
library level and GAPIC level.

### Repository level parameters

The repository level parameters define the version of API definition (proto)
and tools.
They are shared by library level parameters.

| Name                    | Required | Notes                                     |
|:------------------------|:--------:|:------------------------------------------|
| gapic_generator_version |    No    | set through env variable if not specified |
| googleapis_commitish    |   Yes    |                                           |
| libraries_bom_version   |    No    | empty string if not specified             |

### Library level parameters

The library level parameters define how to generate a (multi-versions) GAPIC
library.
They are shared by all GAPICs of a library.

| Name                  | Required | Notes                                                                                                                                     |
|:----------------------|:--------:|:------------------------------------------------------------------------------------------------------------------------------------------|
| api_shortname         |   Yes    |                                                                                                                                           |
| api_description       |   Yes    |                                                                                                                                           |
| name_pretty           |   Yes    |                                                                                                                                           |
| product_docs          |   Yes    |                                                                                                                                           |
| library_type          |    No    | `GAPIC_AUTO` if not specified                                                                                                             |
| release_level         |    No    | `preview` if not specified                                                                                                                |
| api_id                |    No    | `{api_shortname}.googleapis.com` if not specified                                                                                         |
| api_reference         |    No    |                                                                                                                                           |
| codeowner_team        |    No    |                                                                                                                                           |
| client_documentation  |    No    |                                                                                                                                           |
| distribution_name     |    No    | `{group_id}:google-{cloud_prefix}{library_name}` if not specified                                                                         |
| excluded_poms         |    No    |                                                                                                                                           |
| excluded_dependencies |    No    |                                                                                                                                           |
| googleapis_commitish  |    No    | use repository level `googleapis_commitish` if not specified.                                                                             |
| group_id              |    No    | `com.google.cloud` if not specified                                                                                                       |
| issue_tracker         |    No    |                                                                                                                                           |
| library_name          |    No    | `api_shortname` is not specified. This value should be unique among all libraries.                                                        |
| rest_documentation    |    No    |                                                                                                                                           |
| rpc_documentation     |    No    |                                                                                                                                           |
| cloud_api             |    No    | `true` if not specified                                                                                                                   |
| requires-billing      |    No    | `true` if not specified                                                                                                                   |
| transport             |    No    | must be one of `grpc`, `rest` or `both`. This value would only be used for generating .repo-metadata.json and relevant sections in README |


Note that `cloud_prefix` is `cloud-` if `cloud_api` is `true`; empty otherwise.

### GAPIC level parameters

The GAPIC level parameters define how to generate a GAPIC library.

| Name       | Required | Notes                                     |
|:-----------|:--------:|:------------------------------------------|
| proto_path |   Yes    | versioned proto_path starts with `google` |

### An example of generation configuration

```yaml
gapic_generator_version: 2.34.0
googleapis_commitish: 1a45bf7393b52407188c82e63101db7dc9c72026
libraries_bom_version: 26.37.0
libraries:
  - api_shortname: apigeeconnect
    name_pretty: Apigee Connect
    product_documentation: "https://cloud.google.com/apigee/docs/hybrid/v1.3/apigee-connect/"
    api_description: "allows the Apigee hybrid management plane to connect securely to the MART service in the runtime plane without requiring you to expose the MART endpoint on the internet."
    release_level: "stable"
    library_name: "apigee-connect"
    GAPICs:
      - proto_path: google/cloud/apigeeconnect/v1

  - api_shortname: cloudasset
    name_pretty: Cloud Asset Inventory
    product_documentation: "https://cloud.google.com/resource-manager/docs/cloud-asset-inventory/overview"
    api_description: "provides inventory services based on a time series database. This database keeps a five week history of Google Cloud asset metadata. The Cloud Asset Inventory export service allows you to export all asset metadata at a certain timestamp or export event change history during a timeframe."
    library_name: "asset"
    client_documentation: "https://cloud.google.com/java/docs/reference/google-cloud-asset/latest/overview"
    distribution_name: "com.google.cloud:google-cloud-asset"
    release_level: "stable"
    issue_tracker: "https://issuetracker.google.com/issues/new?component=187210&template=0"
    api_reference: "https://cloud.google.com/resource-manager/docs/cloud-asset-inventory/overview"
    GAPICs:
      - proto_path: google/cloud/asset/v1
      - proto_path: google/cloud/asset/v1p1beta1
      - proto_path: google/cloud/asset/v1p2beta1
      - proto_path: google/cloud/asset/v1p5beta1
      - proto_path: google/cloud/asset/v1p7beta1
```

# Run the library generation image

1. Download the API definitions to a local directory, e.g., from [googleapis](https://github.com/googleapis/googleapis).

2. Run the docker image.
   ```shell
   # Assume you want to generate the library in the current working directory
   # and the generation configuration is in the same directory.
   docker run \
     --rm \
     --quiet \
     -u "$(id -u):$(id -g)" \
     -v "$(pwd):/workspace" \
     -v /path/to/api_definition:/workspace/apis \
     gcr.io/cloud-devrel-public-resources/java-library-generation:image-tag
   ```

   * `-u "$(id -u)":"$(id -g)"` makes docker run the container impersonating
     yourself. 
     This avoids folder ownership changes since it runs as root by default.
   * `-v "$(pwd):/workspace"` maps the host machine's current working directory
     to the /workspace folder. 
     The image is configured to perform changes in this directory.
   * `-v /path/to/api_definition:/workspace/apis` maps the host machine's API 
     definitions folder to `/workspace/apis` folder.
 
3. An advanced example:
   ```shell
   docker run \
     --rm \
     --quiet \
     -u "$(id -u):$(id -g)" \
     -v "$(pwd):/workspace" \
     -v /path/to/api_definition:/workspace/apis \
     gcr.io/cloud-devrel-public-resources/java-library-generation:image-tag \
     --generation-config-path=/workspace/generation_config_file \
     --library-names=apigee-connect,asset \
     --repository-path=/workspace \
     --api-definitions-path=/workspace/apis
   ```
    
   * `--generation-config-path=/workspace/generation_config_file` set the
     generation configuration to `/workspace/generation_config_file`.
   * `--api-definitions-path=/workspace/apis` set the API definition path to
     `/workspace/apis`.

To debug the image, please refer to [development guide](DEVELOPMENT.md#debug-the-library-generation-container)
for more info.

## An example to generate a repository using the docker image

If you run the docker image with the example [configuration](#an-example-of-generation-configuration)
shown above, the repository structure is:
```
$repository_path
|_gapic-libraries-bom
|    |_pom.xml
|_java-apigee-connect
|    |_google-cloud-apigee-connect
|    |    |_src
|    |    |_pom.xml
|    |_google-cloud-apigee-connect-bom
|    |    |_pom.xml
|    |_grpc-google-cloud-apigee-connect-v1
|    |    |_src
|    |    |_pom.xml
|    |_proto-google-cloud-apigee-connect-v1
|    |    |_src
|    |    |_pom.xml
|    |_samples
|    |    |_snippets
|    |    |    |_generated
|    |_.OwlBot-hermetic.yaml
|    |_.repo-metadata.json
|    |_owlbot.py
|    |_pom.xml
|    |_README.md
|_java-asset
|    |_google-cloud-asset
|    |    |_src
|    |    |_pom.xml
|    |_google-cloud-asset-bom
|    |    |_pom.xml
|    |_grpc-google-cloud-asset-v1
|    |    |_src
|    |    |_pom.xml
|    |_grpc-google-cloud-asset-v1p1beta1
|    |    |_src
|    |    |_pom.xml
|    |_grpc-google-cloud-asset-v1p2beta1
|    |    |_src
|    |    |_pom.xml
|    |_grpc-google-cloud-asset-v1p5beta1
|    |    |_src
|    |    |_pom.xml
|    |_grpc-google-cloud-asset-v1p7beta1
|    |    |_src
|    |    |_pom.xml
|    |_proto-google-cloud-asset-v1
|    |    |_src
|    |    |_pom.xml
|    |_proto-google-cloud-asset-v1p1beta1
|    |    |_src
|    |    |_pom.xml
|    |_proto-google-cloud-asset-v1p2beta1
|    |    |_src
|    |    |_pom.xml
|    |_proto-google-cloud-asset-v1p5beta1
|    |    |_src
|    |    |_pom.xml
|    |_proto-google-cloud-asset-v1p7beta1
|    |    |_src
|    |    |_pom.xml
|    |_samples
|    |    |_snippets
|    |    |    |_generated
|    |_.OwlBot-hermetic.yaml
|    |_.repo-metadata.json
|    |_owlbot.py
|    |_pom.xml
|    |_README.md
|_pom.xml
|_versions.txt
```

# Generate release notes from API definition changes

The script, `hermetic_build/release_note_generation/cli/generate_release_note.py`
allows you to generate a file containing release notes from API definition
changes in [googleapis](https://github.com/googleapis/googleapis) GitHub
repository.

## Environment

- OS: Linux
- Python (3.12.0 or above)

## Parameters to generate a release note

### Baseline generation configuration path (`baseline-generation-config-path`)

Absolute or relative path to a generation configuration.
Please refer to [Configuration to generate a repository](#configuration-to-generate-a-repository)
for more information.

Note that the `googleapis_commitish` in this configuration is used to retrieve
the first commit, exclusively, to generate the release notes.

### Current generation configuration path (`current-generation-config-path`)

Absolute or relative path to a generation configuration.
The release notes will be generated from commits that are related to the
libraries specified in this configuration.
Please refer to [Configuration to generate a repository](#configuration-to-generate-a-repository)
for more information.

Note that the `googleapis_commitish` entry in this configuration is used to
retrieve the last commit, inclusively, to generate the release notes.

### Repository path (`repository-path`), optional

The path to which the file, `pr_description.txt` containing the release notes
will be sent.
If not specified, the file will be generated to the current working directory.

## Generate a release notes file in a local environment

1. Install python (>= 3.12.0).
It is recommended to create a python virtual environment through the
[official guide](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments).

2. Run the following commands to install python packages
   ```shell
   cd /path/to/sdk-platform-java 
   pip install --require-hashes -r hermetic_build/common/requirements.txt
   pip install hermetic_build/common
   pip install --require-hashes -r hermetic_build/release_note_generation/requirements.txt
   pip install hermetic_build/release_note_generation
   ```
3. Run the following commands to generate a release note
   ```shell
   cd /path/to/sdk-platform-java 
   python hermetic_build/release_note_generation/cli/generate_release_note.py generate \
     --baseline-generation-config-path=/path/to/baseline_generation_config \
     --current-generation-config-path=/path/to/current_generation_config \
     --repository-path=/path/to/send/release_note
   ```

# OwlBot Java Postprocessor

We have transferred the
[implementation](https://github.com/googleapis/synthtool/tree/59fe44fde9866a26e7ee4e4450fd79f67f8cf599/docker/owlbot/java)
of Java Owlbot Postprocessor into `sdk-platform-java/hermetic_build/library_generation`.

## Reflecting changes in synthtool/docker/owlbot/java into this repository
The transfer was not a verbatim copy, it rather had modifications:
 * `format-source.sh` was replaced by a call to `mvn fmt:format`
 * `entrypoint.sh` was modified to have input arguments and slightly modified
   the way the helper scripts are called
 * Other helper scripts were modified to have input arguments.
 * `fix_poms.py` modified the way the monorepo is detected

All these modifications imply that whenever we want to reflect a change from the
original owlbot in synthtool we may be better off modifying the affected source
files one by one. The mapping is from
[`synthtool/docker/owlbot/java`](https://github.com/googleapis/synthtool/tree/59fe44fde9866a26e7ee4e4450fd79f67f8cf599/docker/owlbot/java)
to
[`sdk-platform-java/library_generation/owlbot`](https://github.com/googleapis/sdk-platform-java/tree/move-java-owlbot/library_generation/owlbot)
