Repository navigation
Enhancement: Support for LocalAI endpoint #403
Description
Contact Details
What features would you like to see added?
I would like the possibility to use my custom LocalAI instance (check https://github.com/go-skynet/LocalAI).
It is a drop-in replacement for the OpenAI API but without the need of OpenAI API Key since you run this in your own server.
It extends functionalities to use other custom models.
More details
Not as far as I am aware.
Which components are impacted by your request?
UI, Endpoints
Pictures
No response
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
danny-avila commentedon May 29, 2023
Have you tried setting the OPENAI_REVERSE_PROXY env variable to your localAI endpoint?
You can also set OPENAI_MODELS in the env file to what you need sent to the endpoint for "model". If it doesn't matter, you can ignore it. Default example:
OPENAI_MODELS=gpt-3.5-turbo,gpt-3.5-turbo-0301,text-davinci-003,gpt-4Let me know if you then run into any errors so I can optimize it if necessary
EchedelleLR commentedon May 29, 2023
I reported because some people also force OpenAI API key as mandatory too.
danny-avila commentedon May 29, 2023
It’s not forced as far as I know, it would just throw an error using official OpenAI endpoint
let me know! I had trouble running localai personally (generation was too slow)
EchedelleLR commentedon May 29, 2023
Is the repo cloning and building frontend needed or is everything inside the official Docker image?
I ask because I see some things tied in the docker-compose file and I am unsure if it is because was also used for building or because the Node API needs to be built and dotEnv file inside its folder despite using it already from the docker-compose file.
danny-avila commentedon May 29, 2023
when you run
docker-compose upit will install both the frontend and backend, building the frontend as well. Currently, env files are used in both /api/ and /client/ (they will be consolidated in the future)With docker, there are extra env variables set in the compose (as they need to be set specific to docker. the default options already in place will work for most use cases)