I've done it, and it's not that hard. Anything that you would normally write or draw on the whiteboard to explain the problem, you just explain verbally instead. I found it helpful to write everything out on paper anyway to keep my explanation organized.

The candidate brought along their own laptop with a screen reader and a nifty Braille display, so when it was time to actually write some code, they just fired up Emacs and we were good to go.