More params and lower quant or higher quant and less params? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		throwaway2027 21 days ago \| parent \| context \| favorite \| on: Right-sizes LLM models to your system's RAM, CPU, ... More params and lower quant or higher quant and less params?

bobmcnamara 21 days ago [–]

ISA dependent quant?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact