Toot

Written by atzanteol@sh.itjust.works on 2024-12-13 at 14:05

I’m also curious experimenting with a no-GPU setup.

In my limited experimentation on a 12th Gen Intel® Core™ i7-12700H with 64GB of RAM this is probably not worth it for anything beyond simple usage. You can do it but I get something like 4-5 toks/sec with LLama 3B Instruct and that’s one of the faster models.

=> More informations about this toot | View the thread | More toots from atzanteol@sh.itjust.works

Mentions

=> View sith@lemmy.zip profile

Tags