This monorepo contains two projects: CortexJS and CortexCPP.
- All of the stateful endpoints:
- /threads
- /messages
- /models
- /runs
- /vector_store
- /settings
- /?auth
- …
- Database & Filesystem
- API Gateway
- Authentication & Authorization
- Observability
- All of the high performance, stateless endpoints:
- /chat/completion
- /audio
- /fine_tuning
- /embeddings
- /load_model
- /unload_model
- Kernel - Hardware Recognition
.
├── cortex-js/
│ ├── package.json
│ ├── README.md
│ ├── Dockerfile
│ ├── docker-compose.yml
│ ├── src/
│ │ ├── controllers/
│ │ ├── modules/
│ │ ├── services/
│ │ └── ...
│ └── ...
├── cortex-cpp/
│ ├── app/
│ │ ├── controllers/
│ │ ├── models/
│ │ ├── services/
│ │ ├── ?engines/
│ │ │ ├── llama.cpp
│ │ │ ├── tensorrt-llm
│ │ │ └── ...
│ │ └── ...
│ ├── CMakeLists.txt
│ ├── config.json
│ ├── Dockerfile
│ ├── docker-compose.yml
│ ├── README.md
│ └── ...
├── scripts/
│ └── ...
├── README.md
├── package.json
├── Dockerfile
├── docker-compose.yml
└── docs/
└── ...
- Pre-install script:
npm pre-install script; platform specific (MacOS / Windows / Linux)- Tag based:
npm install @janhq/cortex
npm install @janhq/cortex#cuda
npm install @janhq/cortex#cuda-avx512
npm install @janhq/cortex#cuda-avxcortex init (AVX2 + Cuda)
Enable GPU Acceleration?
1. Nvidia (default) - detected
2. AMD
3. Mac Metal
Enter your choice:
CPU Instructions
1. AVX2 (default) - Recommend based on what the user has
2. AVX (old CPU)
3. AVX512
Enter your choice:
Downloading cortex-cuda-avx.so........................25%
Cortex is ready!
It seems like you have installed models from other applications. Do you want to import them?
1. Import from /Users/HOME/jan/models
2. Import from /Users/HOME/lmstudio/models
3. Import everything
Importing from /Users/HOME/jan/models..................17%POST /settings
{
"gpu_enabled": true,
"gpu_family": "Nvidia",
"cpu_instructions": "AVX2"
}TBD