Enhancement: Support for LocalAI endpoint

[EchedelleLR]

Contact Details

elr@disroot.org

What features would you like to see added?

I would like the possibility to use my custom LocalAI instance (check https://github.com/go-skynet/LocalAI).

It is a drop-in replacement for the OpenAI API but without the need of OpenAI API Key since you run this in your own server.

It extends functionalities to use other custom models.

More details

Not as far as I am aware.

Which components are impacted by your request?

UI, Endpoints

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Have you tried setting the OPENAI_REVERSE_PROXY env variable to your localAI endpoint?

You can also set OPENAI_MODELS in the env file to what you need sent to the endpoint for "model". If it doesn't matter, you can ignore it. Default example:
OPENAI_MODELS=gpt-3.5-turbo,gpt-3.5-turbo-0301,text-davinci-003,gpt-4

Let me know if you then run into any errors so I can optimize it if necessary

[EchedelleLR]

I reported because some people also force OpenAI API key as mandatory too.

[danny-avila]

I reported because some people also force OpenAI API key as mandatory too.

It’s not forced as far as I know, it would just throw an error using official OpenAI endpoint

let me know! I had trouble running localai personally (generation was too slow)

[EchedelleLR]

Is the repo cloning and building frontend needed or is everything inside the official Docker image?

I ask because I see some things tied in the docker-compose file and I am unsure if it is because was also used for building or because the Node API needs to be built and dotEnv file inside its folder despite using it already from the docker-compose file.

[danny-avila]

when you run docker-compose up it will install both the frontend and backend, building the frontend as well. Currently, env files are used in both /api/ and /client/ (they will be consolidated in the future)

With docker, there are extra env variables set in the compose (as they need to be set specific to docker. the default options already in place will work for most use cases)