I hate Python and I’ve barely used it. It’s a dependency nightmare. I wrote something that calls the OAI API in Java and managed to install Automatic and SD, but I’ll go the easy route. I have an older 6GB like GTX1660 I think. Is that enough for kobold?
That should work in koboldcpp. I’m running mine on an older gpu than that but with more vram. Use the gpulayers parameter to control exactly how much of the work gets offloaded to the gpu to control how much vram gets used up.
I hate Python and I’ve barely used it. It’s a dependency nightmare. I wrote something that calls the OAI API in Java and managed to install Automatic and SD, but I’ll go the easy route. I have an older 6GB like GTX1660 I think. Is that enough for kobold?
That should work in koboldcpp. I’m running mine on an older gpu than that but with more vram. Use the gpulayers parameter to control exactly how much of the work gets offloaded to the gpu to control how much vram gets used up.