ZK-AI
/
ollama_proxy_server
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

upgraded

This commit is contained in:
Saifeddine ALOUI 2024-01-16 01:49:15 +01:00
parent 9d6c4b40f0
commit 57aad06779
3 changed files with 17 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@ -14,17 +14,17 @@ Make sure you have Python (>=3.8) and Apache installed on your system before pro
### Servers configuration (config.ini) ### Servers configuration (config.ini)
Create a file named `config.ini` in the same directory as your script, containing server configurations: Create a file named `config.ini` in the same directory as your script, containing server configurations:
```makefile ```makefile
[Server1] [DefaultServer]
url = http://localhost:8080/ url = http://localhost:11434
queue_size = 5 queue_size = 5
[Server2] [SecondaryServer]
url = http://localhost:8081/ url = http://localhost:3002
queue_size = 3 queue_size = 3
# Add as many servers as needed, in the same format as [Server1] and [Server2]. # Add as many servers as needed, in the same format as [DefaultServer] and [SecondaryServer].
``` ```
Replace `http://localhost:8080/` with the URL and port of the first server. The `queue_size` value indicates the maximum number of requests that can be queued at a given time for this server. Replace `http://localhost:11434/` with the URL and port of the first server. The `queue_size` value indicates the maximum number of requests that can be queued at a given time for this server.
### Authorized users (authorized_users.txt) ### Authorized users (authorized_users.txt)
Create a file named `authorized_users.txt` in the same directory as your script, containing a list of user:key pairs, separated by commas and each on a new line: Create a file named `authorized_users.txt` in the same directory as your script, containing a list of user:key pairs, separated by commas and each on a new line:

6
config.ini
View File

@ -1,8 +1,10 @@
[SERVER1] [DefaultServer]
url = http://localhost:11434 url = http://localhost:11434
queue_size = 5
[SERVER2] [SecondaryServer]
url = http://localhost:3002 url = http://localhost:3002
queue_size = 3
# Add more servers as you need. # Add more servers as you need.

7
ollama_proxy_server/main.py
View File

@ -1,3 +1,10 @@
"""
project: ollama_proxy_server
file: main.py
author: ParisNeo
description: This is a proxy server that adds a security layet to one or multiple ollama servers and routes the requests to the right server in order to minimize the charge of the server.
"""
import configparser import configparser
from http.server import BaseHTTPRequestHandler, HTTPServer from http.server import BaseHTTPRequestHandler, HTTPServer
from socketserver import ThreadingMixIn from socketserver import ThreadingMixIn