Skip to content

llm-api-catalog.md

Latest commit

 

History

History
89 lines (62 loc) · 2.7 KB

llm-api-catalog.md

File metadata and controls

89 lines (62 loc) · 2.7 KB

Creating an LLM Chain

---
depth: 2
local: true
backlinks: none
---

Implementing the Method

  1. Edit the RetrievalAugmentedGeneration/examples/simple_rag_api_catalog/chains.py file and add the following import statements:

    :language: python
:start-after: start-llm-imports
:end-before: end-llm-imports

  2. Update the llm_chain method with the following statements:

    :language: python
:start-after: start-llm-chain-method
:end-before: end-llm-chain-method

Building and Running with Docker Compose

Using the containers has one additional step this time: exporting your NVIDIA API key as an environment variable.

  1. Build the container for the Chain Server:

    $ docker compose --env-file deploy/compose/compose.env -f deploy/compose/simple-rag-api-catalog.yaml build chain-server

  2. Export your NVIDIA API key in an environment variable:

    $ export NVIDIA_API_KEY=nvapi-...

  3. Run the containers:

    $ docker compose --env-file deploy/compose/compose.env -f deploy/compose/simple-rag-api-catalog.yaml up -d

Verify the LLM Chain Method Using Curl

You can access the Chain Server with a URL like http://localhost:8081.

  • Confirm the llm_chain method runs by submitting a query:

    $ curl -H "Content-Type: application/json" http://localhost:8081/generate \
    -d '{"messages":[{"role":"user", "content":"What should I see in Paris?"}], "use_knowledge_base": false}'

    Example Output

Next Steps

  • You can stop the containers by running the docker compose -f deploy/compose/simple-rag-api-catalog.yaml down command.