• 21 Posts
  • 439 Comments
Joined 2 years ago
cake
Cake day: July 4th, 2023

help-circle



  • Buying new: Basically all of the integrated memory units like macs and amd’s new AI chips, after that any modern (last 5 years) gpu while focusing only on vram (currently nvidia is more properly supported in SOME tools)

    Buying second hand: not likely to find any of the integrated memory stuff, so any GPU from the last decade that is still officially supported and focusing on vram.

    8gb is enough to run basic small models, 20+ for pretty capable 20-30b models, 50+ for the 70b ones and 100-200+ for full sized models.

    These are rough estimates, do your own research as well.

    For the most part with LLMs for a single user you really only care about VRAM and storage speed(ssd) Any GPU will perform faster than you can read for anything that fully fits on it’s VRAM, so the GPU only matters if you intend on running large models at extreme speeds (for automation tasks, etc) And the storage is a bottleneck at model load, so depending on your needs it might not be that big of an issue for you, but for example with a 30gb model you can expect to wait 2-10 minutes for it to load into the vram from an HDD, about 1 minute with a sata SSD, and about 4-30 seconds with an NVMe.



  • It’s very very unlikely that your TV and your device connected to it both support and enable ethernet over HDMI by default. But if you are unsure you can test it by connecting and seeing if the TV is getting a connection.

    Personally I also opened my TV and disconnected the wifi card since in theory the TV could also just try to connect to any open wifi in the area without me knowing, but to each their own threat model.













  • I once wanted to see how long I can go, at 32 hours I felt like I was about to die so I stopped. Literally one bite made all of the terrible feeling go away and to this day it both annoys me and freaks me out that it was all psychological.

    Like seriously brain? Couldn’t handle not eating so badly that you decided to make me feel like shit? But also damn, if that’s how I felt about not eating, addictions sound much scarier now.