Skip to content

Commit 31be899

Browse files
marwissMoult
marwiss
authored and
Moult
committed
Replace existing code in src/opencdeserver with code from the "Kontroll OpenCDE API:s server" POC project on https://github.com/marwiss/Kontroll 
1 parent 6d7a6ee commit 31be899

File tree

124 files changed

+9066
-3388
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

124 files changed

+9066
-3388
lines changed

‎src/opencdeserver/.gitignore‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
/api/app/logs/*
2+
/api/app/favicon.ico
3+
/data/
4+
/etc/
5+
/neo4j/data/*
6+
/neo4j/import/*
7+
/neo4j/logs/*
8+
/secrets/*
9+
/TODO.txt
10+
/venv/
11+
/.idea/
12+
/secrets/

‎src/opencdeserver/LICENSE‎

Lines changed: 661 additions & 0 deletions
Large diffs are not rendered by default.

‎src/opencdeserver/README.md‎

Lines changed: 268 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,275 @@
1-
# Server-Test
1+
# Kontroll BCF API server - OpenCDE APIs
22

3-
```
4-
$ pip install -r requirements.txt
5-
$ cd bcfserver/
6-
$ python
7-
>>> from run import db
8-
>>> db.create_all()
9-
$ export FLASK_APP=run.py
10-
$ flask run
11-
```
3+
## What is this?
124

13-
Go to [http://localhost:5000](http://localhost:5000) to see the server
5+
OpenCDE APIs:
6+
Foundation API,
7+
BCF API,
8+
Documents API
9+
implemented in python,
10+
the FastAPI framework
11+
and the Neo4j graph database.
1412

15-
# Register the user
13+
This is server software. To run or test this code you need to set up your server to work with a client.
14+
This server was developed and tested using Solibri as a client. To use a client with this server you need to configure
15+
a client id and a client key.
1616

17-
1. Go to http://localhost:5000/register to register the user
18-
2. Create the client
19-
3. For grant type enter authorization_code
20-
4. For response_type enter code secret
21-
5. Enter the scope and create the client
17+
You also need to configure encrypted communication and authentication using HTTP and OAuth2.
18+
And then you need an SSL-certificate. To get an SSL-certificate you need your own domain name.
19+
Solibri can only connect to preconfigured domains.
2220

23-
### You will be redirected to the page with the details of your client id and secret
21+
This means that it is not so easy to test this software.
22+
In the future I hope an open source BCF/CDE-client can be developed to access this server.
23+
Maybe Blender BIM in the future could be extended with such BCF/CDE-client functionality.
2424

25-
# Foundation API
25+
## Background
2626

27-
- Set the Base URL will be `http://127.0.0.1:5000/`
27+
You can read some background information about the project here:
28+
29+
https://martin.wiss.se/data/bcf_ifc_graph.pdf
30+
31+
The name of this project is Kontroll BCF API server because it was originally
32+
developed as a POC prototype to test if BCF API could be used for digital inspection plans
33+
for building permits (translated as "kontroll-plan" in Swedish language).
34+
35+
## Code structure
36+
37+
/docker-compose.yml
38+
You bring up the project using this file.
39+
40+
/api/Dockerfile
41+
Contains instructions for the FastAPI container.
42+
Referenced by /docker-compose.yml
43+
44+
/api/app
45+
Contains the server application.
46+
47+
/api/app/main.py
48+
This is where the application starts.
49+
The FastAPI-container will run this script on uvicorn.
50+
main.py will load routes from scripts in the /api/app/api folder.
51+
52+
/api/app/api
53+
This is where the FastAPI routes are defined.
54+
55+
/api/app/repository
56+
Routes often runs methods that will read or write data to or from a database.
57+
This is where methods that read or write data from database are declared.
58+
59+
/api/app/database
60+
Some general methods to interact with database.
61+
62+
/api/app/db_config
63+
When the Neo4j container is launched it will initialize a database
64+
with some test data from init.cypher. This initial state does not represent
65+
all data. Model.cypher contains a more complete model.
66+
67+
/api/app/ifcgraph
68+
Script to import data from IFC STEP file to Neo4j.
69+
70+
/api/app/models
71+
Pydantic models used to define the JSON structures of requests and responses.
72+
Models are arranged in files depending on if it is request or response
73+
or both (common) or not at all (other).
74+
75+
/api/app/security
76+
Contains methods for OAuth2 and a method to get the secrets used.
77+
You need to create your own secrets and put them in the /secrets folder.
78+
Secrets are more secure than environment values.
79+
80+
/api/app/templets
81+
Jinja2 templates used to generate some webpages necessary for OAuth2 flow and Documents API flow.
82+
83+
/api/app/requirements.txt
84+
Generated using pipreqs not pip freeze.
85+
86+
## Deployment
87+
88+
To get this code running properly you need:
89+
90+
- your own linux server, on premise or in the cloud
91+
- a domain name and a TLS-certificate for your domain name
92+
- a proxy server, nginx was used
93+
- to set up your proxy server to use HTTPS
94+
- to use proxy as bridge between docker network and remote
95+
- proxy (outside container) and uvicorn (inside container) should access same TLS certificate
96+
- to edit your .env file with values specific for your server and API
97+
- to replace hard coded values in code with specific values for your server and API
98+
- to create three secrets and put them in the /secrets folder
99+
- to put apoc-5.7.0-core and apoc-5.7.0-extended jars in /neo4j/plugins folder
100+
- to put an ifcopenshell.zip in /api/app/ifcopenshell folder
101+
- Docker and Neo4j software
102+
- a favicon at /favicon.ico
103+
- to run "bash restart.sh" to start and restart containers
104+
- a BCF client, registered with client ID and secret
105+
106+
This code was developed and tested using Solibri Office as BCF client, however:
107+
Solibri Office can only connect to pre-registered servers. This means that you cannot
108+
connect to your server using a regular version of Solibri Office. If you want to use Solibri,
109+
then you will have to first kindly ask Solibri if they want to add your server
110+
to the list of registered servers. An alternative is to develop your own BCF client
111+
or to use this existing BCF API client module for python to build your own client script:
112+
113+
https://pypi.org/project/bcf-client/
114+
115+
I do not know of any working BCF API client plugins for Revit or Archicad that allows
116+
connecting to arbitrary BCF API servers. It seems like most plugins instead were developed to
117+
only connect to a specific BCF API server. I think that is contradictory because BCF is an open standard
118+
intended to facilitate communication between multiple clients and servers.
119+
A BCF API client that can connect to multiple arbitrary BCF API servers would be a good thing to develop.
120+
121+
## TODO
122+
123+
The original intention of the code was just to create a prototype for private testing.
124+
Improvements are needed to collaborate on this code or to deploy this system in production.
125+
126+
Suggested improvements:
127+
128+
- General code structure refactoring
129+
- Better commenting
130+
- Testing scripts, unit testing
131+
- Better error handling
132+
- More consistent logging
133+
- Easier deployment
134+
- User registration and admin system
135+
- BCF client registration functionality
136+
- OData to Cypher converter
137+
- Documents API and BCF API integration
138+
- More complete OpenAPI documentation of routes
139+
- More complete documentation of pydantic models
140+
- Improvements on IFC-graph
141+
- BCF event logging
142+
- Implement also the routes that Solibri Office does not support
143+
- Replace hard coded values with .env values
144+
- Improve security before any kind of deployment in production
145+
- Making full use of native IFC (for example allowing BCF API to edit IFC directly using BIM snippets)
146+
- Integrate BCF API BIM snippets editing with GIT commits
147+
148+
## Prototype
149+
150+
This code was developed and tested on a system using:
151+
152+
- Ubuntu 20
153+
- Nginx 1.18
154+
- Lets encrypt, certbot
155+
- Docker 20
156+
157+
## Suggested reading
158+
159+
### API
160+
161+
- FastAPI documentation
162+
https://fastapi.tiangolo.com/
163+
164+
- Building Python Microservices with FastAPI
165+
https://www.packtpub.com/product/building-python-microservices-with-fastapi/9781803245966
166+
167+
- Building Python Web APIs with FastAPI
168+
https://www.packtpub.com/product/building-python-web-apis-with-fastapi/9781801076630
169+
170+
- Building Data Science Applications with FastAPI - Second Edition
171+
https://www.packtpub.com/product/building-data-science-applications-with-fastapi-second-edition/9781837632749
172+
173+
- Pydantic docs
174+
https://docs.pydantic.dev/latest/
175+
176+
- OData v4
177+
https://www.odata.org/documentation/
178+
179+
### OAuth2
180+
181+
- Mastering OAuth 2.0
182+
https://www.packtpub.com/product/mastering-oauth-20/9781784395407
183+
184+
- OAuth 2.0 Cookbook
185+
https://www.packtpub.com/product/oauth-20-cookbook/9781788295963
186+
187+
- OAuth2 with Password (and hashing), Bearer with JWT tokens
188+
https://fastapi.tiangolo.com/ur/tutorial/security/oauth2-jwt/
189+
190+
- Authorization Code Flow
191+
https://auth0.com/docs/get-started/authentication-and-authorization-flow/authorization-code-flow
192+
193+
- What is the OAuth 2.0 Authorization Code Grant Type?
194+
https://developer.okta.com/blog/2018/04/10/oauth-authorization-code-grant-type
195+
196+
### Neo4j and Cypher
197+
198+
- Neo4j Cypher Manual
199+
https://neo4j.com/docs/cypher-manual/current/introduction/
200+
201+
- Using Neo4j from python
202+
https://neo4j.com/docs/getting-started/languages-guides/neo4j-python/
203+
204+
- Building Neo4j Applications with Python
205+
https://graphacademy.neo4j.com/courses/app-python/
206+
207+
- The Property Graph Model
208+
https://neo4j.com/developer/graph-database/#property-graph
209+
210+
### OpenCDE APIs
211+
212+
- BCF API
213+
https://github.com/buildingSMART/BCF-API
214+
215+
- Documents API
216+
https://github.com/buildingSMART/documents-API
217+
218+
- Foundation API
219+
https://github.com/buildingSMART/foundation-API
220+
221+
- Open CDE APIs OAuth2 Example
222+
https://github.com/buildingSMART/foundation-API/blob/v1.0/OAuth2Examples.md
223+
224+
### Server
225+
226+
- Docker documentation
227+
https://docs.docker.com/
228+
229+
- Docker Compose overview
230+
https://docs.docker.com/compose/
231+
232+
- How to use secrets in Docker Compose
233+
https://docs.docker.com/compose/use-secrets/
234+
235+
- NGINX Reverse Proxy documentation
236+
https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/
237+
238+
- Certbot documentation
239+
https://eff-certbot.readthedocs.io/en/stable/
240+
241+
### IFC
242+
243+
- Native IFC
244+
https://github.com/brunopostle/ifcmerge/blob/main/docs/whitepaper.rst
245+
246+
- IFC Specifications Database
247+
https://technical.buildingsmart.org/standards/ifc/ifc-schema-specifications/
248+
249+
- IFC-graph for facilitating building information access and query
250+
https://www.sciencedirect.com/science/article/pii/S0926580523000389
251+
252+
- IFCOpenShell documentation
253+
https://blenderbim.org/docs-python/
254+
255+
### Frontend
256+
257+
- Jinja2 Documentation
258+
https://svn.python.org/projects/external/Jinja-2.1.1/docs/_build/html/index.html
259+
260+
- Bootstrap
261+
https://getbootstrap.com/docs/4.1/getting-started/introduction/
262+
263+
- jQuery
264+
https://api.jquery.com/
265+
266+
## License
267+
268+
The GNU Affero General Public License is a free,
269+
copyleft license for software and other kinds of works,
270+
specifically designed to ensure cooperation with the community
271+
in the case of network server software.
272+
273+
https://www.gnu.org/licenses/agpl-3.0.en.html
274+
275+
See the /LICENSE file.

‎src/opencdeserver/api/Dockerfile‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
FROM python:3.10-bullseye AS base
2+
RUN apt-get update && apt-get install -y \
3+
unzip \
4+
&& rm -rf /var/lib/apt/lists/* \
5+
COPY ./app/ifcopenshell/ifcopenshell.zip /ifcopenshell.zip
6+
CMD ["unzip", "/ifcopenshell.zip", "-d", "/usr/local/lib/python3.10/site-packages/"]
7+
WORKDIR /code/app
8+
COPY ./app/requirements.txt /code/requirements.txt
9+
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
10+
RUN pip install neo4j==5.9.0 uvicorn ifcopenshell Jinja2 python-multipart bcrypt --upgrade
11+
# pip install: Install packages.
12+
# --no-cache-dir: You can shrink the image size by disabling the cache.
13+
# --upgrade: Upgrade packages.
14+
# -r: use requirements file.
15+
COPY ./app /code/app
16+
17+
FROM base AS server
18+
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--proxy-headers", \
19+
"--ssl-keyfile", "/etc/letsencrypt/live/kontroll.digital/privkey.pem", \
20+
"--ssl-certfile", "/etc/letsencrypt/live/kontroll.digital/fullchain.pem"]
21+
# If running behind a proxy like Nginx or Traefik add --proxy-headers.
22+
# --ssl-keyfile: Path to keyfile.
23+
# --ssl-certificat: Path to certificate.

‎src/opencdeserver/api/__init__.py‎

Whitespace-only changes.

‎src/opencdeserver/api/app/api/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)