I’ve been trying but it quickly breaks for me. I’ve got 64GB of system memory and a 6800xt but after a few responses it starts to fail to generate a response. Idk if the context window is becoming too large or what but it’s unfortunate. The responses are decent but not being able to maintain a thread sucks. This is using LM studio. Maybe i need to tweak the load settings or something
Have you tried turning on “developer mode” in lmstudio and looking through it’s logs? No idea about your particular case, but any time generation in LM Studio has failed for me I’ve been able to figure out why and work around it by looking at the logs.
I’ve been trying but it quickly breaks for me. I’ve got 64GB of system memory and a 6800xt but after a few responses it starts to fail to generate a response. Idk if the context window is becoming too large or what but it’s unfortunate. The responses are decent but not being able to maintain a thread sucks. This is using LM studio. Maybe i need to tweak the load settings or something
FWIW I have this same issue with lm studio
Have you tried turning on “developer mode” in lmstudio and looking through it’s logs? No idea about your particular case, but any time generation in LM Studio has failed for me I’ve been able to figure out why and work around it by looking at the logs.