TFA takes us on this journey and then at the end, "Image maps ended up not working for us" without telling us what they did. My money is on JavaScript for that "expressive" aspect of the hover.
I played with gemma-3-4b-it-qat recently using a mid-tier graphics card and a few things stood out to me:
1. It was very fast, between 35 and 70 tokens per second, with initial response in under 200ms. That kind of speed is a feature.
2. It was very useful. I had a brainstorming session with it that was both fluid and fruitful
3. I can't wrap my head around so much knowledge being contained in about 3GB of data. It seems to know something about everything. Imperfect, but very useful.
My understanding is the opposite, see papers for "synthetic" data training. They use a small bit if real data to generate lots of synthetic data and get usable results.
The bias leans towards overfitting the data, which in some use cases - such as missile or drone design which doesn't need broad comparisons like 747s or artillery to complete it's training.
Kind of like neural net back propogation but in terms of model /weights
Nirsoft saved my ass so many times on different things. I remember when I lived somewhere without (reliable or consistent) internet access, I scraped all the tools to take with me. They still are in my tools folder to this day!