My very own AI server

Projects
81

I want the best conversational program I can get, but without the nanny minding. AI requires morals and ethics for safety. People are assholes. I understand why a publicly available system has to have the guardrails built in, people are assholes and will try to do really dumb (to the point of dangerous) stuff. But on my own personal system, that will only be used by me, I choose to behave and act according to my morals and ethics, and don’t appreciate having an overbearing, one size fits all, approach forced on me.

The example I give people is, If I want to write a book and one of the characters is cooking meth, and I need a reasonably accurate description of the method. I ask ChatGPT, and I get

I’m not going to make meth, I have no interest in making meth, I just want to know how it’s done.

I picked the Model and size off of a few recommendations I found (reddit, google, so forth). So I’ve got an R720 Dell, with a 2080ti modded to 22gb of Vram. (that’s why I picked 13B) and I want an excellent conversationalist with the mandated guard rails stripped off.

What model would you reccomend for that?

Also, cause I’m a total noob here, I don’t thing openwebui is in the plan (unless it’s baked in somewhere.) I’m basically setting up voice conversational control of Home Assistant.

82

If you want a conversation model, and not a role-play model, you should give gemma-3-12b-abliterated a try. Abliteration surgically removes the ability for the model to refuse, so it will hallucinate more but it won’t say no (or anything that would be equivalent to saying no).

The problem with using resources such as reddit and searching online for model recommendations is that it is a moving target – there are new model releases frequently and using one that is 2 years old at this point is not recommended unless you have a specific reason for it. I can’t emphasize highly enough how much more intelligent and conversant newer models are compared to the old ones.

1 Like
83

The problem with not using such resources is that I’m an idiot with no place to start learning. You learn from failing, and If I’m gonna fail, I’m going at it full speed Wiley E. Coyote style.

Then I hopefully get up, learn from it and, if truly lucky, get some expert guidance.

I guess I just wanna say thanks for helping. Whether it’s correcting me, or whatever you’re doing with Amal’s thing, just know it’s appreciated. Truly.

2 Likes
84

I wouldn’t consider myself an expert – I just happened to do the Wyle E Coyte thing a bit earlier, and I don’t mind sparing others the bruises I endured if I am able to. It has been a wild ride these past 2 years, and it doesn’t seem to be slowing down.

It really feels to me like the internet in the late 90s – capabilities growing seemingly without limit, a kind of lawlessness with a performative helicopter parent type attitude by the big players which everyone amusedly ignored. The sense of disdainful admiration by the general public while those in the know try to get across the sheer potential, not yet corrupted and completely immeasurable. But, like its predecessor, there is a bubble, and no one really knows how to make money with it. The bubble will pop but the tech won’t go anywhere, and people will figure out how to monetize it, and thus corrupt it, and eventually the world will be worse off. But the potential was there.

2 Likes
85

I’m on the verge of giving up on this. Probably will for at least a week, may be permanent.

I did install debian12, not because I wanted linux, but because running it in windows, is just running linux inside windows. So all the issues I have with command line, plus a 200 dollar windows 11 license.

I have Debian12, docker, home assistant, ollama, and several different ai models. I set up my zigbee adapter, and sengled lights in home assistant. I can log in, and change colour / brightness and / or toggle them on / off.

I tried the gemma3 models, but they all had an unacceptably long pause before starting a reply, on the order of 3-4 seconds. While I did go back to Wizard-Vicuna-uncensored, this is really a minor issue and not what’s killing me.

Currently trying to integrate the AI LLM into Home Assistant.

What’s brought me to a stop is that the only way I can work on the system is to tell chat gpt what I want to do, and then follow it’s instructions step by step. I’m almost always copy pasting into the command line. When something goes wrong, I can only copy paste from the command line window into Chat gpt, or if I’m working on the Home assistant GUI, try to verbally describe what I’m seeing to Chat gpt. It goes down paths of outdated information, and then works back up problem by problem.

For ex, it had me trying to use a command line involving Remove and after repeatedly failing, told me that was an outdated command and I had to use -rm instead.

In another example, it had me work to integrate HACS by editing the configuration.yaml file. But it gave me bad instructions. Then changed them though multiple steps until it finally quit throwing errors. When I tried to use it inside Home Assistant, I was greeted with an error that stated HACS wasn’t configurable via configuration.yaml.

So basically, I’m doing something I don’t understand, in a foreign language, with a mentally deficient guide.

I’m tired, frustrated, cranky, and I’m no longer having fun. Gonna let it be for awhile.

2 Likes
86

Man thats frustrating. Unfortunately Chatgpt isn’t going to known anything about software projects that are either small or change rapidly.

This stuff isn’t easy. I just had a bad update on my NAS that caused me to spend 8 hours trying to get my media server back running again.

I’m going to be honest with you, this is a hobby that requires time and effort. You can decide how much you want to put in, but keep in mind that whatever systems you put together, you will need to maintain. Software will break, there will be bad updates. Weird, stupid issues will pop up. Chatgpt won’t be able to help with most of it as you’ve seen.

My 2 cents is: take a step back, cool off for a week, and decide on your goals.

There are plenty of resources out there. Whenever I deploy new software, I look at the project, read the install and configuration documents, and then google what I’m interested in doing with “tutorial”. For example: “home assistant HACS tutorial”. 90% of the time their are a few different blog posts that can describe an easy way to get things set up.

If you do this and run into an issue, just keep googling. “Docker compose tutorial”, “Linux ssh tutorial” “Linux command line tutorial”. Then use.chatgpt to fill in the blanks.

Things will start to fall into place, but I think chatgpt is making this harder for you, since its giving you bad directions and preventing you from learning a solid foundation.

Also also, ive personally found home assistant docker to be a bit of a pain. Their OS has much easier integration with everything and its very easy to setup and leave alone.

I’m happy to answer any questions you have

1 Like
87

Unsurprising friend. How many things you listed are new to you? All of them? Every single one of them is going to require a lot of curiosity and poking around and learning. Just to give you an idea, I had home assistant OS up and running on a virtual machine guest image for 14 months before I even configured my first device that wasn’t auto-detected.

This will have to be a labor of love that you make incremental progress on overtime… don’t approach this like a whittling hobby where you’re trying to create Notre Dame the first time you pick up the knife. When you get some time here and there, you will pick one small aspect of the thing you’re trying to do here and decide you’re going to fix or figure out that one little thing. You’ll do this many times until things start coming together.

ChatGPT is probably the most contemporary and powerful double-edged sword I can think of. If you take a much more leisurely and fun pace with this, you can try to tango with chatGPT but we are also here to help give guidance through experience.

4 Likes