AI Knowledge HubHardware & Performance
VRAM
Memory attached to the graphics processor.
The space where a model resides while being computed. Too little capacity forces splitting a large model across devices.
ExampleA 13B model will not fit in 8GB of VRAM in one piece.