My very own AI server

[ODaily]

O.K. Repairs happening, as we speak.

New motherboard. It came with some hardware. I had had the H310 raid controller, but the new board came with a H710P, which I decided to keep. I also went from a 4 x 1gb RJ45 to a 2x 10gb SFP and 2x1gb RJ45 NIC card. All good stuff. Upgrades.

I noticed 2 slightly bent pins on the CPU sockets, but was able to get them back pretty darn close. Everything else just popped together.

And… Because I’m an uncontrollable idiot, I replaced riser 3 with the 16x option (might wanna upgrade later) and bought another 12 sticks of ram (96gb) to finish filling up the empty slots. Don’t need, just obsessively want. I did get the exact same stuff, by size, speed, capacity, manufacturer, and P/N.

Took me a bit to figure out how to get the new raid controller to accept the old disks / configuration, but that’s sorted now.

It seems to be struggling with the memory / ram. It had a few error messages about specific ram slots, all in the B section. Also, I put all the new ram in the B section, and all the old stuff in the A section. It’s currently chewing on “configuring memory”. Google says this may take awhile. If it persists long enough, I’ll probably try moving or removing the specific ram it doesn’t like. Maybe I can eliminate the ram sticks or the sockets.

I checked, and IDRAC is accesible, doesn’t need to have the firefox settings tweaked to access it, and has the enterprise license so I can use the dedicated port and a few other features.

Also, I looked at the memory error codes.

I’m not getting a whole lot of AI work done (soon), but it’s been both frustrating and fun learning about my server.

[ODaily]

IT’S ALIVE!!!

3 dead sticks of ram, one partially dead, out of the 12 I got from ebay. Sent a polite message to the seller requesting he swap the dead ones out. We’ll see.

I gotta tell you, she didn’t like having her brains scrambled and parts swapped. It started, stopped, reconfigured, and adjusted over and over till it got sorted out. Gonna try to update bios and stuff next. If it survives that, I’ll be plugging the GPU in sometime this weekend.

[Hamspiced]

Nice man.

I hope idrac isn’t horribly out of date. I know it took me some 15 hours to incrementally update idrac completely.

[ODaily]

@Hamspiced

I hope idrac isn’t horribly out of date.

From 2017. Only 8 years old.

Need help. Be advised, I’m an idiot, so I probably did something bass-ackwards.

Lifecyle Controller will not mass update. I think this is cause IDRAC is so hopelessly old.

Solution- Update IDRAC.

My IDRAC version is 2.50.50.50
Looking at Dell Downloads for the R720 Poweredge, I see a current version of 2.65.65.65

Looking through previous versions, I find ( and download) everything AFTER 2.50.50.50, including the latest version.

Note, this is all downloaded onto the desktop of my regular PC. I then access IDRAC via it’s IP address, navigate to IDRAC SETTING / UPDATE ROLLBACK. From there I select “local” and browse to the next idrac.exe file. In this case, 2.52.52.52 and hit the upload button. It does it’s thing, hits 100% and then displays the following error message.

Any guesses where I went wrong?

[Hamspiced]

So it’s always worth restarting idrac. I can’t remember if I did idrac first or life cycle first. But I had to install each incremental update until I was current for both idrac and the lifecycle controller

[amal]

I think those exe files are meant to be executed in a Windows environment running on the r72/ machine with the idrac installed. There should all be other files that I think have a .bin extension which are loaded into the web interface of the idrac

[Hamspiced]

I think they are actually exe files. Standby

[ODaily]

There were win 32/64 files available separately. FWIW.

If all else fails, I’ll try inside Windows.

[Hamspiced]

Can’t seem to find my old files. But I’m fairly certain it was exe.

Here was my old post of having to update every single version release.

Show off your Homelab!

idrac was out of date by like 4 years. It was so out of date i couldnt access from the web interface because its encryption was too poor. i had to use an old Windows 7 computer to log into it. [Screenshot 2023-11-17 at 1.52.58 AM] So i started doing incremental updates version by version to get it current. At version 2.20.20.20 fans all got pegged at 100%. Bios settings, idrac settings changing things around i could only ever get them as low as 50% ALL THE TIME. googling said that there …

[amal]

Interesting… I swear I did some other file type but possibly no.

[ODaily]

I’m pretty sure linux gets a .bin file.

There’s 3 options.
install from Windows
Install from IDRAC (.exe)
Install from Redhat / linux (.bin)

But now I have a bigger problem. A friend took me out for dinner, and when I came back I decided to pull down those windows files and try them. According to dell’s support website, they can’t find a R720. Even when it suggests the specific R720 Poweredge model in a drop down menu, and I select it.

I can’t use the service tag number either.

I tried going back to the previous link in my browser’s history, and it will re-download the files I’ve previously downloaded. But I’ve no way to access the files I haven’t yet gotten.

If I try to return to the page that gave the description of the driver (and linked to previous versions) I get the following error.

What the fuck, Dell?

Also note, I tried from both the server and my regular PC. I also dropped the VPN on my regular PC just in case.

[amal]

[ODaily]

exactly.

Huh, I tried looking for random drivers for whatever models it might suggest, nothing available. I wonder if they’re down for maintenance?

NOTE.
Nevermind that. I had the thought, if it’s a .exe file, maybe it will execute from windows. Nope, it won’t. But… It will unzip. I got a releasenotes.txt and a firmimg.d7 file.

So I tried IDRAC installing the .d7 file, and it took.

Colour me confused.

[Hamspiced]

So depending on how old it is you may run into the same issue I did. I was accessing idrac from the Web interface. There was one update that messed with the html cert and made it http and no browser would allow me to connect to it at all. Even browsers in compatibility mode and mimicking ie7.

I had to find an old laptop running Windows 7 and it let me connect to idrac to install a few more updates that then allowed me to connect via https

[ODaily]

BEHOLD!!!

So Dell un-borked their cranial-rectum inversion. I got back in and gathered what I needed. Getting all the firmware installed was a PITA, but it’s done.

Installed the GPU and really really had the butterflies when it came time to power it up. But it booted, reconfigured, rebooted and downloaded drivers.

@ODaily

So I figure I need a game plan.
Stage 1, Hardware. Physically assembling the basic system.
Stage 2a, Software. I need to get a conversational AI up and running on text input.
Stage 2b, Software continued. Time to bring together Home Assistant, Voice control, and generally smooth it out.
Stage 3, Voice. I want to give it a unique voice / sound. I have ideas here. Will need to learn about voice training, but that’s part of stage 3 anyways.
Stage 4, Hardware. Time to take voice to multiple rooms, and start controlling lights. I’d also like to work in some sensors. I’m thinking an outdoor weather station, and indoor temp / humidity.

Stage 1 complete.
Time to start working out Home Assistant / Ollama / Piper. Not tonight or tomorrow though, I’m wiped and have plans early in the a.m.

[ODaily]

It’s been awhile since I’ve had occasion to say so, but I absolutely hate linux.

I’ve been trying to install docker / home asisstant / Ollama / wizard-vicuna-uncensored 13b

I’m pretty sure I’ve got them all installed. Kinda sure. Docker is sitting on my windows desktop, but everything under that is just linux with a nice docker wrap on it.

Just because I got them there doesn’t mean they do anything. Right now my biggest accomplishment is docker -ps

I wouldn’t have gotten anywhere if I hadn’t completely relied on chatgpt, and with that, I’m not really understanding anything I’m doing. Just copy and paste.

Anywho. Progress, aggravation and irritation.
Gonna go watch tv and cool off a bit.

[ODaily]

Long day today, very tired.

To keep it short, I got Docker, Home Assistant, Ollama, and Wizard Vicuna Uncensored 13B running. ChatGPT provided some test code, and I was getting either 52 or 56 tokens/sec (so tired).

Tommorrow’s task is to set up Home Assistant, with my USB Zigbee adapter and 4 RGB lights I bought for that purpose.

Then integrate Ollama / Wizard Vicuna Uncensored 13B.

Then I’ve got to start looking at connecting the Home Assistant Voice Preview module, followed by digging into Piper for voice.

Anywho, that’s the roadmap for now.

[Plato]

Some advice: document everything you are doing. When I do stuff like this the first time I never remember what I did and have to relearn it all when I come back to it eventually. It may be years from now but you will have to modify or reinstall some or all of these things and you will probably appreciate some detailed write-ups of what you did.

[Plato]

Also, FYI wizard-vicuna-uncensored-13b is extremely old and I wouldn’t recommend it. If you have control over the context and systems prompts (you do with ollama and openwebui) you don’t need an ‘uncensored’ model because you can just make it do whatever you want (edit it’s last reply to start complying and let it go from there) but even if you want an uncensored/smut model there are much better ones available now. As of this posting, gemma-3-12b is probably the best model in that size and it has vision support (you will need to load the mmproj file along with the main gguf), but if you want something story-driven/smutty check out models tuned by ‘TheDrummer’. Though fair warning that smut models are basically retuned on a dataset that includes a lot of adult and or just offensive stuff to override the safety training and can be notably dumber and/or unpredictable and/or require strange sampler settings to be coherent.