On Windows I'm using emacs from http://gnu.mirror.vexxhost.com/emacs/windows/

On Ubuntu I'm using this ppa : https://launchpad.net/~cassou/+archive/emacs

To have the latest orgmode version I use the `org-plus-contrib` package from : http://orgmode.org/elpa.html I'm not sure if you need to delete `lisp\org` first (orgmode is included with emacs).

I think you can also use git to keep orgmode up-to-date.