My very own AI server

Projects
81

I want the best conversational program I can get, but without the nanny minding. AI requires morals and ethics for safety. People are assholes. I understand why a publicly available system has to have the guardrails built in, people are assholes and will try to do really dumb (to the point of dangerous) stuff. But on my own personal system, that will only be used by me, I choose to behave and act according to my morals and ethics, and don’t appreciate having an overbearing, one size fits all, approach forced on me.

The example I give people is, If I want to write a book and one of the characters is cooking meth, and I need a reasonably accurate description of the method. I ask ChatGPT, and I get

I’m not going to make meth, I have no interest in making meth, I just want to know how it’s done.

I picked the Model and size off of a few recommendations I found (reddit, google, so forth). So I’ve got an R720 Dell, with a 2080ti modded to 22gb of Vram. (that’s why I picked 13B) and I want an excellent conversationalist with the mandated guard rails stripped off.

What model would you reccomend for that?

Also, cause I’m a total noob here, I don’t thing openwebui is in the plan (unless it’s baked in somewhere.) I’m basically setting up voice conversational control of Home Assistant.

82

If you want a conversation model, and not a role-play model, you should give gemma-3-12b-abliterated a try. Abliteration surgically removes the ability for the model to refuse, so it will hallucinate more but it won’t say no (or anything that would be equivalent to saying no).

The problem with using resources such as reddit and searching online for model recommendations is that it is a moving target – there are new model releases frequently and using one that is 2 years old at this point is not recommended unless you have a specific reason for it. I can’t emphasize highly enough how much more intelligent and conversant newer models are compared to the old ones.

1 Like
83

The problem with not using such resources is that I’m an idiot with no place to start learning. You learn from failing, and If I’m gonna fail, I’m going at it full speed Wiley E. Coyote style.

Then I hopefully get up, learn from it and, if truly lucky, get some expert guidance.

I guess I just wanna say thanks for helping. Whether it’s correcting me, or whatever you’re doing with Amal’s thing, just know it’s appreciated. Truly.

2 Likes
84

I wouldn’t consider myself an expert – I just happened to do the Wyle E Coyte thing a bit earlier, and I don’t mind sparing others the bruises I endured if I am able to. It has been a wild ride these past 2 years, and it doesn’t seem to be slowing down.

It really feels to me like the internet in the late 90s – capabilities growing seemingly without limit, a kind of lawlessness with a performative helicopter parent type attitude by the big players which everyone amusedly ignored. The sense of disdainful admiration by the general public while those in the know try to get across the sheer potential, not yet corrupted and completely immeasurable. But, like its predecessor, there is a bubble, and no one really knows how to make money with it. The bubble will pop but the tech won’t go anywhere, and people will figure out how to monetize it, and thus corrupt it, and eventually the world will be worse off. But the potential was there.

2 Likes
85

I’m on the verge of giving up on this. Probably will for at least a week, may be permanent.

I did install debian12, not because I wanted linux, but because running it in windows, is just running linux inside windows. So all the issues I have with command line, plus a 200 dollar windows 11 license.

I have Debian12, docker, home assistant, ollama, and several different ai models. I set up my zigbee adapter, and sengled lights in home assistant. I can log in, and change colour / brightness and / or toggle them on / off.

I tried the gemma3 models, but they all had an unacceptably long pause before starting a reply, on the order of 3-4 seconds. While I did go back to Wizard-Vicuna-uncensored, this is really a minor issue and not what’s killing me.

Currently trying to integrate the AI LLM into Home Assistant.

What’s brought me to a stop is that the only way I can work on the system is to tell chat gpt what I want to do, and then follow it’s instructions step by step. I’m almost always copy pasting into the command line. When something goes wrong, I can only copy paste from the command line window into Chat gpt, or if I’m working on the Home assistant GUI, try to verbally describe what I’m seeing to Chat gpt. It goes down paths of outdated information, and then works back up problem by problem.

For ex, it had me trying to use a command line involving Remove and after repeatedly failing, told me that was an outdated command and I had to use -rm instead.

In another example, it had me work to integrate HACS by editing the configuration.yaml file. But it gave me bad instructions. Then changed them though multiple steps until it finally quit throwing errors. When I tried to use it inside Home Assistant, I was greeted with an error that stated HACS wasn’t configurable via configuration.yaml.

So basically, I’m doing something I don’t understand, in a foreign language, with a mentally deficient guide.

I’m tired, frustrated, cranky, and I’m no longer having fun. Gonna let it be for awhile.

2 Likes
86

Man thats frustrating. Unfortunately Chatgpt isn’t going to known anything about software projects that are either small or change rapidly.

This stuff isn’t easy. I just had a bad update on my NAS that caused me to spend 8 hours trying to get my media server back running again.

I’m going to be honest with you, this is a hobby that requires time and effort. You can decide how much you want to put in, but keep in mind that whatever systems you put together, you will need to maintain. Software will break, there will be bad updates. Weird, stupid issues will pop up. Chatgpt won’t be able to help with most of it as you’ve seen.

My 2 cents is: take a step back, cool off for a week, and decide on your goals.

There are plenty of resources out there. Whenever I deploy new software, I look at the project, read the install and configuration documents, and then google what I’m interested in doing with “tutorial”. For example: “home assistant HACS tutorial”. 90% of the time their are a few different blog posts that can describe an easy way to get things set up.

If you do this and run into an issue, just keep googling. “Docker compose tutorial”, “Linux ssh tutorial” “Linux command line tutorial”. Then use.chatgpt to fill in the blanks.

Things will start to fall into place, but I think chatgpt is making this harder for you, since its giving you bad directions and preventing you from learning a solid foundation.

Also also, ive personally found home assistant docker to be a bit of a pain. Their OS has much easier integration with everything and its very easy to setup and leave alone.

I’m happy to answer any questions you have

1 Like
87

Unsurprising friend. How many things you listed are new to you? All of them? Every single one of them is going to require a lot of curiosity and poking around and learning. Just to give you an idea, I had home assistant OS up and running on a virtual machine guest image for 14 months before I even configured my first device that wasn’t auto-detected.

This will have to be a labor of love that you make incremental progress on overtime… don’t approach this like a whittling hobby where you’re trying to create Notre Dame the first time you pick up the knife. When you get some time here and there, you will pick one small aspect of the thing you’re trying to do here and decide you’re going to fix or figure out that one little thing. You’ll do this many times until things start coming together.

ChatGPT is probably the most contemporary and powerful double-edged sword I can think of. If you take a much more leisurely and fun pace with this, you can try to tango with chatGPT but we are also here to help give guidance through experience.

4 Likes
88

So I haven’t entirely given up. I’ve torn down and reset up multiple times, and I’m fairly comfortable with Ollama.

I tried a bunch of models and finally settled right back on wizard-vicuna-uncensored 13b. I wanted the 30b, but it’s just a little too pokey for my taste. The 13b is set to a high quantization, but I’m in a weird spot where the 30b is just a touch too big for my GPU and the 13b has lots of leftover room. So plenty of room for a little extra precision. And crazy fast. I do kinda wish there was a 24b-ish model.

Now I’m kinda picking and poking at it. Basically just fooling with the parts I want to play with as I learn each chunk.

Today I packed up the whole server, desk-chair-monitor and all, and schlepped it over to a friends house so I could record her voice. Took about 3 hours to capture one of the twangiest Ark-La-Tex accents I know of. That’s all 1150 spoken lines in Piper Recording Studio. I’m planning to record a few more similar voices and then take a stab at synthesising one. Apparently, Coqui TTS is capable of this. Maybe. Probably. :man_shrugging:

Any who, not dead yet.

4 Likes
89

I tried to jump into this at one point but didn’t get very far.

So there is a specific script you read out and it creates the voice from that? Can you free-train it? Like can I feed in my TEDx talk instead?

1 Like
90

I’ve gotten a little farther, but it’s not very clear how you proceed. The data exports in one format, but needs to be in a different format (WTF Piper?) plus it’s all at the wrong hz, so now I’m converting a second time (seriously Piper?) Note, it records in .webm at 48000 but needs to be .wav at 16000hz. Oh and it structured the files one way, but recorded them in the configuration document a different way. Easy fix, but PITA to figure out.

Yes. You start “Piper Recording Studio” in a terminal, and then open a specific address in your browser. You press record, speak the written line on the screen, then hit submit. Afterwards you have a .txt file, and a .wav file in matched pairs, that need to be converted to proper format and specs so you can load them into “Piper” which is a totally separate thing.

Yes. Note the above about differences between “Piper Recording Studio” and “Piper”. If you generate audio files and a matching transcript from videos / recordings / audio files, then you’ve basically done what Piper Recording Studio does, and you’re ready to go to Piper. Assuming you’re in the proper format and expected parameters. The process breaks a long audio file into smaller bits, and you’ll have to generate a transcript of each one with the same name, then put together a configuration file for it. It’s something that can be scripted, but just be aware what it entails.

Network Chuck did a section on this in HIS YOUTUBE VIDEOS. He scraped the voice for his “Terry” A.I. from videos of Terry Crews (NFL Player).

You may want to be careful about the quality of what you import. The following is a file from my training data, I didn’t monitor the recording, and she clearly. read. each. word. separately. I don’t know if that’s going to be a problem yet. You can also hear the mouse clicks. I haven’t done it, but there is supposed to be a way to get the mouse clicks out when training. Also, FWIW, the recording was done on a Rode NT USB Mini microphone. Basically as much :moneybag:quality as I could swing.

91

I’m training a voice now, and Oh Buddy, can you hear the cooling fans hit turbo mode!

Anywho.
I was able to work through everything that I didn’t understand, which was mostly everything, using chatgpt again. But what’s interesting was the process that I used.

I started by telling chatgpt what I wanted to do, was given a few options, then I was instructed to open a terminal and run the code I was presented.

Copy-Paste.

The terminal gave me output.

Copy-Paste back to chatgpt.

chatgpt gave me a brief explanation of the results and/or errors, told me the next step, and presented more code to run.

Copy-Paste.

I ran this basic process on a loop most of last evening, and some again this morning. During the whole thing I was nothing more than meat-in-the-middle executing the Copy-Paste command. Sure I made a few minor decisions, but mostly in an overarching clarification of goals kind of way.

I can see a day, not too distant, where chat gpt is given the ability to open a terminal, run code, and read the results. Given how everything runs in terminal in Linux, the only thing I would need to do is make sure it’s plugged in, and give some type of vision.

Imagine you told it, “I’ve got a IOT toaster oven, and I want to run DOOM on it, port that over.” Then it says, " I found a Toastmaster 3000 on your network, is that where you want to run Doom?" I confirm, and it gets to work, just working it out until it finishes and says “I’ve installed Doom on your Toastmaster 3000, and it’s ready to play. Would you like me to install it on any other appliances?”

The interesting part is that it will fail, repeatedly, then adjust/adapt/overcome just like it did with me last night, but without the Copy/Paste.

The concept is probably held back by how twitchy and or hallucinogenic AI can be. In it’s current state, it could misfire and cause as much harm as good, but eventually… It could be good enough to do it.