Skip to content

Commit e41b2a7

Browse files
ianthomas23ianthomas23
authored
Add docs on environment variables and wasm-specific behaviour (#166)
1 parent f8ef848 commit e41b2a7

4 files changed

Lines changed: 175 additions & 14 deletions

File tree

‎docs/env_vars.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Environment variables
2+
3+
4+
## In all builds
5+
6+
The following four environment variables should be set for `git2cpp commit` and `git2cpp merge`
7+
subcommands. The use of `git2cpp config` instead is partially supported and will be improved in
8+
time.
9+
10+
`GIT_AUTHOR_EMAIL`
11+
: The email for the "author" field.
12+
13+
`GIT_AUTHOR_NAME`
14+
: The human-readable name in the "author" field.
15+
16+
`GIT_COMMITTER_EMAIL`
17+
: The email for the "committer" field.
18+
19+
`GIT_COMMITTER_NAME`
20+
: The human-readable name for the "committer" field.
21+
22+
23+
## In WebAssembly build only
24+
25+
(git_cors_proxy)=
26+
`GIT_CORS_PROXY`
27+
: In-browser remote `git2cpp` operations such as `clone`, `fetch` and `push` usually require use of
28+
a [CORS proxy server](cors_proxy). Use this environment variable to specify how the target URL is
29+
encoded into the CORS proxy URL, details of which depend on how the CORS proxy server is
30+
implemented.
31+
32+
The `GIT_CORS_PROXY` should contain the URL of the CORS proxy itself, followed by a number of
33+
substitutions which are denoted by curly braces. To illustrate the substitutions, assume that the
34+
`git2cpp` command is for the repository at `https://github.com/organisation/repository`.
35+
36+
Substitutions:
37+
38+
- `{host}` is replaced by `github.com`
39+
- `{path}` is replaced by `/organisation/repository/` followed by extra information that depends
40+
on details of the `git2cpp` operation being performed
41+
- `{protocol}` is replaced by `https:`
42+
- `{url}` is equivalent to `{protocol}//{host}{path}`
43+
- `{api_key}` is replaced by the value of environment variable `GIT_CORS_PROXY_API_KEY` if it is
44+
set.
45+
46+
If no substitutions are specified then `{url}` is appended.
47+
48+
All of the substitutions except `{api_key}` have an `:encode` variant such as `{url:encode}`
49+
that passes the argument through the
50+
[encodeURIComponent](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent)
51+
JavaScript function, which some CORS proxies require.
52+
53+
You can verify the actual URL used for requests in the Network tab of your browser's Developer
54+
Tools for debugging purposes.
55+
56+
See [CORS proxy server](cors_proxy) for usage examples.
57+
58+
`GIT_CORS_PROXY_API_KEY`
59+
: This value is used to replace the `{api_key}` in `GIT_CORS_PROXY` and is intended for use with a
60+
CORS proxy that requires an API key. Alternatively the API key could be put directly in the
61+
`GIT_CORS_PROXY` instead.
62+
63+
(git_http_timeout)=
64+
`GIT_HTTP_TIMEOUT`
65+
: In the WebAssembly build, all http(s) requests are limited by a timeout which has a default of 10
66+
seconds. To use a different timeout set the `GIT_HTTP_TIMEOUT` environment variable. For example,
67+
to set a timeout of 20 seconds use:
68+
69+
```bash
70+
export GIT_HTTP_TIMEOUT=20
71+
```

‎docs/index.md‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,28 @@
11
# Overview
22

33
`git2cpp` is a C++ wrapper of [libgit2](https://libgit2.org/) to provide a command-line interface
4-
(CLI) to `git` functionality. The intended use is in WebAssembly in-browser terminals (the
5-
[cockle](https://github.com/jupyterlite/cockle) and
6-
[JupyterLite terminal](https://github.com/jupyterlite/terminal) projects) but it can be compiled and
4+
(CLI) to `git` functionality. The intended use is in WebAssembly in-browser terminals (see the
5+
[cockle](https://github.com/jupyterlite/cockle),
6+
[JupyterLite terminal](https://github.com/jupyterlite/terminal) and
7+
[Notebook.link](https://notebook.link) projects) but it can be compiled and
78
used on any POSIX-compliant system.
89

910
The Help pages here are generated from the `git2cpp` command and subcommands to show the
1011
functionality that is currently supported. If there are features missing that you would like to use,
1112
please create an issue in the [git2cpp github repository](https://github.com/QuantStack/git2cpp).
1213

14+
The Appendix contains additional information on [Environment variables](env_vars.md) used in
15+
`git2cpp` and about the behaviour of the [WebAssembly build](wasm_build.md).
16+
1317
```{toctree}
1418
:caption: Help pages
1519
:hidden:
1620
created/git2cpp
1721
```
1822

19-
## Environment variables
20-
21-
`GIT_HTTP_TIMEOUT`
22-
: In the WebAssembly build, all http(s) requests are limited by a timeout which has a default of 10
23-
seconds. To use a different timeout set the `GIT_HTTP_TIMEOUT` environment variable. For example,
24-
to set a timeout of 20 seconds use:
25-
26-
```bash
27-
export GIT_HTTP_TIMEOUT=20
28-
```
23+
```{toctree}
24+
:caption: Appendix
25+
:hidden:
26+
env_vars
27+
wasm_build
28+
```

‎docs/wasm_build.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# WebAssembly build
2+
3+
The in-browser WebAssembly build of `git2cpp` is intended to behave as similar as possible to other
4+
builds but there are some differences when remotely accessing remote servers, in particular with
5+
blocking requests and the requirement for a CORS proxy server.
6+
7+
8+
## Blocking requests
9+
10+
Remote `git2cpp` requests in non-WebAssembly builds are progressive and feedback is provided as the
11+
data is streamed back from the remote. But in WebAssembly builds such remote requests are blocking
12+
and no feedback can be provided until the entire response is received back from the remote.
13+
This can be a long time to wait without feedback, and if there is an error such that a response is
14+
not received it could block forever, leaving the in-browser terminal unusable.
15+
16+
Hence the WebAssembly build limits http(s) requests with a timeout that defaults to 10 seconds.
17+
This timeout can be increased using the [`GIT_HTTP_TIMEOUT` environment variable](git_http_timeout).
18+
19+
In addition, when a `git2cpp` response is received that is larger than 10 MB, a prompt is presented
20+
to the user to confirm whether to proceed to unpack the response or not. Note that the size of the
21+
response may be smaller or larger than the size of the directory structure it unpacks to.
22+
23+
24+
(cors_proxy)=
25+
## CORS proxy server
26+
27+
The fetching of resources in a browser from one domain to another is often limited by a browser
28+
security feature called
29+
[Cross-Origin Resource Sharing](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CORS)
30+
(CORS). For this to be allowed the target domain must indicate it is happy to accept cross-origin
31+
requests by adding certain headers to `https` responses. Most git servers such as `github.com` do
32+
not add these headers, so a `git2cpp clone` from `github.com` will fail with a CORS error if run
33+
from within a browser whereas there is no such limitation if run from a terminal of a real computer.
34+
35+
The solution to this problem is to use a separate CORS proxy server. The `git2cpp` remote request is
36+
sent to this intermediate server which send the request to the target server and as this request is
37+
coming from outside a browser it is not subject to CORS restrictions. The proxy receives the
38+
response from the target server and adds the required CORS headers before returning it to `git2cpp`.
39+
40+
Various public CORS proxy servers are available for use or you serve your own. It can be useful to
41+
serve your own on `localhost` when experimenting to confirm that everything works as expected
42+
before moving on to a more complex solution. Be aware that the CORS proxy server is able to read the
43+
content of your request so be careful if you are using authentication tokens.
44+
45+
The [`GIT_CORS_PROXY` environment variable](git_cors_proxy) is used to specify how the target URL is
46+
encoded in the CORS proxy URL.
47+
48+
### Example running a local CORS proxy server
49+
50+
If you are running a local [cockle](https://github.com/jupyterlite/cockle) or
51+
[JupyterLite terminal](https://github.com/jupyterlite/terminal) deployment you can also run a local
52+
CORS proxy such as [CORS Anywhere](https://github.com/Rob--W/cors-anywhere) to test it out.
53+
In a separate terminal on your host machine on which you have `nodejs` available, `cd` to a new
54+
clean directory, and download and run the CORS proxy server using:
55+
56+
```js
57+
npm install cors-anywhere
58+
HOST=localhost PORT=8881 node node_modules/cors-anywhere/server.js
59+
```
60+
61+
This will start the CORS proxy server listening on `http://localhost:8881/`. To use this in your
62+
local `cockle` or `JupyterLite terminal` deployment in your browser set the `CORS_PROXY_URL` to be
63+
```bash
64+
export GIT_CORS_PROXY=http://localhost:8881/
65+
```
66+
and then try a `git2cpp clone` using something like:
67+
```bash
68+
git2cpp clone https://github.com/some-organisation/some-repository
69+
```
70+
71+
### Example using a public CORS proxy server
72+
73+
There is a public instance of [CORS Anywhere](https://github.com/Rob--W/cors-anywhere) available at
74+
`https://cors-anywhere.herokuapp.com/`. This can be used for demonstration purposes but it requires
75+
an explicit opt-in and your access will be time-limited. To request temporary access go to
76+
`https://cors-anywhere.herokuapp.com/` and follow the instructions there.
77+
78+
Once you have access, you can try this out in one of the public deployments such as those at
79+
[https://jupyterlite.github.io/cockle](https://jupyterlite.github.io/cockle) or
80+
[https://jupyterlite.github.io/terminal](https://jupyterlite.github.io/terminal).
81+
82+
Set the `CORS_PROXY_URL` to be
83+
```bash
84+
export GIT_CORS_PROXY=https://cors-anywhere.herokuapp.com/
85+
```
86+
and then try a `git2cpp clone` using something like:
87+
```bash
88+
git2cpp clone https://github.com/some-organisation/some-repository
89+
```

‎src/wasm/stream.cpp‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,8 @@ EM_JS(
116116
);
117117

118118
EM_JS(const char*, js_maybe_convert_url, (const char* url_str), {
119-
// Convert URL to use CORS proxy based on env vars GIT_CORS_PROXY and GIT_CORS_PROXY_TYPE.
119+
// Convert URL to use CORS proxy based on env vars GIT_CORS_PROXY and possible
120+
// GIT_CORS_PROXY_API_KEY.
120121
// If no conversion occurs, return the original unconverted URL as a new string.
121122
const url_js = UTF8ToString(url_str);
122123
const url = new URL(url_js);

0 commit comments

Comments
 (0)