API
To use the API, you wil first need to find your API key, accessible from the UI. Click on your user avatar and navigate to Settings, and then the Account page in settings. Here you will be able to see the API Keys. Expand this section and copy either the JWT token or create an API key.
To connect to the API, there are a few endpoints you can use depending on your use case. The primary way these endpoints differ is in the format of the response.
The Ollama endpoint is https://genai.rcac.purdue.edu/ollama/api/chat.
For a streaming response, this will return results formated like:
{"model":"llama3.1:latest","created_at":"2024-11-15T18:19:55.588019874Z","message":{"role":"assistant","content":"I"},"done":false}
For an OpenAI API-compatible response, use https://genai.rcac.purdue.edu/api/chat/completions
For a streaming response, this will return results formated like:
data: {"id":"chatcmpl-808","object":"chat.completion.chunk","created":1731694445,"model":"llama3.1:latest","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"I"},"finish_reason":null}]}
To use either of the endpoints, insert your chosen endpoint and API key in the example Python code below:
url = {insert chosen endpoint here}
headers = {
"Authorization": f"Bearer {jwt_token_or_api_key}",
"Content-Type": "application/json"
}
body = {
"model": "llama3.1:latest",
"messages": [
{
"role": "user",
"content": "What is your name?"
}
],
"stream": True
}
response = requests.post(url, headers=headers, json=body)
if response.status_code == 200:
print(response.text)
else:
raise Exception(f"Error: {response.status_code}, {response.text}")
This will return output in a JSON format along with metadata.