Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Jetson is basically like Apple. Theoretically its good, but the models with enough RAM are too expensive.

Smartphones aside, little Ryzen 6000 boxes would be OK.

Used DDR5 laptops with a little discrete GPU would be even better. I have one with a broken screen that may be dedicated to this very task.



You can get an Orin with 8GB. Or the 32/64GB AGX module for 1.3/2.3k (3k for the dev kit). Not cheap but plenty of ram and 60W power target maximum.

You could maybe run something on the 4GB Jetson Nano?

But very slow: https://www.reddit.com/r/LocalLLaMA/comments/12c7w15/the_poi...


IIRC 32GB of shared RAM is not enough for llama 70B, and 8GB is just barely enough for llama 7B... So yeah, that value proposition is not good at all.

A 32GB+ ddr5 laptop with a dGPU and some RAM will (IIRC, just barely) do llama 70B for far less money and a similar TDP.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: