Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Most definitely, what I meant though is how do we know mental attention is not essentially a big softmax (over some context, e.g. short or long term memory)?

The eye physically focusing on one thing at time seems like a special case (and this fact isn't even true of all animals, e.g. many prey species), and not a part of the brain's attention mechanism.



Oh, I don't know, it very well might! However, the brain's physical structure doesn't look conducive to 4096x4096 matmuls!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: