Feature Request: Self Hosted AI Options
Brad Thompson
Hello,
I want to applaud the JoomShaper team for being bold and integrating AI into SPPB right away.
It would be nice if you could also support locally hosted AI options like LM Studio. https://lmstudio.ai/
This would allow the use of different LLMs locally, which would cut costs of having to subscribe to OpenAI.
Thanks, Brad
Mehtaz Afsana Borsha
Accepted AnswerHi
Thanks for contacting us and thanks for your suggestions. I will add it in our feature request list so that our DEV team will take care of it.
-Regards.
Paul Frankowski
Accepted AnswerIf I may...
right now SPPB package is about 4MB, with addicional (built-in) AI code it may have 50-100MB - so I am not sure if all users hostings will handle installing such a big file. Idea is nice... but the actual demand may be small, and the problem may be too large. Yes, I know it can be done as extra installed package.
Brad Thompson
Accepted AnswerHi Paul,
I'm not asking to put an LLM into SPPB, but rather have other API options supported.
Proper LLMs can be multiple GBs in size.
LMStudio allows you to download and run your preferred LLM locally, which has its own API you can referrence similar ro OpenAi's chat-gpt. The APIs should be similar.
Make more sense?
Paul Frankowski
Accepted AnswerYes, but as I know SPPB will need more extra code anyway, maybe not +50M ;p
Anyway, as @Mehtaz said we added that to wish-list already and our developers on one of meetings will consider that idea. Certainly need to consider the pros and cons of this type of integration. But we both cannot promise anything. We are just messengers.