Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Next browser – web browser in Common LISP (atlas.engineer)
182 points by lelf on June 1, 2019 | hide | past | favorite | 49 comments


You know what'd be extra-awesome? If it had a text editor and an Emacs Lisp emulation layer …

Seriously, a web browser which is capable of running the decades of elisp out there would be basically the only tool I need on my desktop: text editor, web browser, mail reader, git client, console, shell, calculator, diary, organiser. Boot directly into that and life would be perfect.


Next is the next best thing. Instead of making your browser the environment, think of it as a fantastic browser extension for Emacs.

You can use the SLIME Emacs extension to connect to a running instance of the Next browser. Then, you have complete programmatic access to the full browser data structure through a REPL in Emacs (eg: switching tabs by using shortcut/command in Emacs). And Lisp REPLs are awesomer than something like ipython!

Keyboard driven browsing is now the least imaginative thing possible.

Take the high-quality tools you have for processing text, and apply that to your browsing workflow! Imagine being able to integrate your browsing workflow with your information management system, eg: org-mode. Switch to a project, and with a couple of keystrokes, open the relevant links on a page in a set of browser tabs/buffers. Filter all the HN pages in your browser and bookmark them to a text file (to be resumed later). Save all arXiv pages to your reference management system (eg: bibtex), etc. When browsing webpages, evaluate code blocks (via Emacs). Download/cache videos for all all Youtube tabs you have open. Emacs becomes the IFTTT-like glue for all your browsing activity!

For some chatter/ideas along these lines, take a look at this Github discussion: https://github.com/atlas-engineer/next/issues/195


+1 nice ideas!

The last time I played with the Next Browser, I loaded it from source and in principle all your ideas are doable. Please post a link if you implement something and writer it up.


Definitely! I became excited by Next when I stumbled on it about a month ago. I have several ideas/itches, but a high activation barrier to implementing and experimenting as I'm a total lisp & emacs newbie.

I'm also given to understand that the Next codebase is currently undergoing significant refactoring, so I hope to delve into this more once the internals have stabilized.

If somebody has a complementary outlook/background, I'm happy to be baited into discussion and activity. As they say, two people together may be thrice as effective.


Honest questions below.

> Imagine being able to integrate your browsing workflow with your information management system, eg: org-mode.

Is this better than alt-tabbing between your browser and your note taking app?

> Save all arXiv pages to your reference management system (eg: bibtex)

All? Which? What would this workflow look like?

> When browsing webpages, evaluate code blocks (via Emacs) > Filter all the HN pages in your browser and bookmark them to a text file

How many times is this useful, really?

> Download/cache videos for all all Youtube tabs you have open

youtube-dl?

> Emacs becomes the IFTTT-like glue for all your browsing activity!

Why is this good?


Note that this appears to be yet another Webkit shell, and not an implementation of parsing/rendering pages in Lisp.


Hi, I am one of the developers. You seem to be mistaken, it is not a WebKit shell (we are currently working on a web engine variant). It is actually a standalone Lisp program that defines a protocol for client browser implementations. Please see the following file for more information: https://github.com/atlas-engineer/next/blob/master/ports/REA...

Eventually we would like to make our own web-renderer, but we recognize that it is a lot of work, and a bit away


> Hi, I am one of the developers. You seem to be mistaken, it is not a WebKit shell (we are currently working on a web engine variant). It is actually a standalone Lisp program that defines a protocol for client browser implementations.

OK, looking through the README in the ports/ directory, I see this:

> Next is made of two programs:

> The Lisp core which offers the user full control over the browsing. It’s 100% Common Lisp.

> The “platform port”, which only purpose is to display a window with two components: the minibuffer and the web view.

> The two parts communicate via D-Bus, an RPC system. The Lisp core sends a D-Bus request to the platform port (for example, “open this URL”); the platform port acts upon it and sends a D-Bus response back to the Lisp core.

So you've put the WebKit stuff into another process, but your browser is WebKit based, unless I completely misunderstood what I just read. You couldn't use Next to drive Firefox, for example, without some interesting re-writes on the Firefox end.

(Just to make it abundantly clear: I think this is a really neat hack and a good way to separate concerns, but the way you phrased things confused me until I dug into the README you linked to and read some source code.)


I wonder if you could do what remote browser does and drive Firefox by the web extensions api, https://github.com/intoli/remote-browser.


I wonder that as well, that is a very fascinating link, thank you!


The latter is a hell of of a task, given the complexity of the current web standards. I'm not sure if it's possible to do this decently without some kind of serious financial backing.


Targeting a subset, i.e. the "document-centric web" (no JS, only basic HTML and CSS), is not that difficult in comparison and has been done by a few others; the most prominent of these "second tier" browsers being https://en.wikipedia.org/wiki/Dillo and https://en.wikipedia.org/wiki/NetSurf .

Of course, writing a text-based browser in Lisp or any other language is far simpler but also less practical.


> Of course, writing a text-based browser in Lisp or any other language is far simpler but also less practical.

Emacs has a few; the included one is called “ewww”, and is pretty nice if you’re into emacs already.


I've been looking for a lightweight browser I can use on older machines- Dillo and NetSurf both seem worth checking out!


This is certainly possible with our current architecture. Maybe we will do it at some point down the road.


Then there shouldn't be claims of a web browser written in lisp when that isn't the case.


Rendeting is done in WebKit [on macOS], that’s correct.

But it’s already a lot more than a “Webkit shell”, e. g. see https://github.com/atlas-engineer/next/blob/master/source/re...


Which is good, otherwise web pages might look like lisp.


> Which is good, otherwise web pages might look like lisp.

(what-is-p wrong (with 'that))

More seriously, there's also the security angle with browser, that using an independently developed engine has certain advantages.


Good to see the recent github activity - I will give it another try when I get to a laptop.

The keyboard driven navigation using search to flip between tabs and the tree structured history is interesting. Also, as always, I am happy to see people using Common Lisp for implementing creative ideas! Off topic, but I have been using Common Lisp professionally since 1983 and the first book I ever wrote was a Common Lisp book for Springer Verlag. It has been a fun ride.

Thanks for this project!


Thank you for your kind words! It's what keeps us going :)


At least in regard to easy navigation, I don’t see much that trumps Chrome’s Vimium extension.


Chromium could treat extensions better. “First class” keybindings provide a much better experience (think of opening a new tab that contains solely an image — you can’t close this tab with Vimium key bindings while the image is loading).

Edit: one example but I’ve found Vimium — while great — falls short due to “the platform.”


>Security warning for macOS users: The Cocoa webkit platform port has been dropped in 1.2.2 and it's currently not possible to build it without exposing Next to a remote-execution vulnerability. We recommend macOS users to wait for the next release which will feature a new platform port.

Anyone out there with a good idea of how serious a risk this is?

Edit - I mean, it sounds very serious, so I don't really want to run it on OSX. That said, I woudn't mind knowing if I am being overly careful here.


Basically the issue is, someone can craft a URL that when you visit it it will allow the execution of arbitrary Lisp code on your machine- a huge problem. The likelihood of someone setting up such a website/link is low, but full disclosure is the policy we've decided to adopt.


When the malicious hackers take up lisp... THEN we'll really have something serious to worry about. ;)


Ahh, that is a problem. Though I sort of like the idea that you are accidentally making macs into remotely operated lisp machines.


I wonder if there's any effort to make this work on Windows? I tried Next on my Linux box and loved it. Best thing emacs-y browser since Conkeror stopped development. I would love to switch to it, but I'm forced to use Windows most of the time.


There is not a current effort targeting Windows, but the development of the PyQt port will add Windows support, so keep watching :D


We'll see, but from my experience when the tagline is "<product> in <language>", we no distinctive feature for end-users, the project never goes very far.


Yes, only a mere two years of continuous development with a lot of features. Definitely not far at all :)


I don’t like the tone of your parent post, but he does have a point: during communication/marketing, highlight and focus on unique features, not on a unique programming process, or only those interested with browser programming / architecture will be interested, not the majority of end users.

Nonetheless, congratulations on your project! It is already a big accomplishment!


I see what you mean, but on certain projects the language used can be a differentiator (not in marketing, but at least in concept).

See for example Redox. The whole point is to build an operating system in Rust because of the language safety, which allegedly trickles down to more safety in the user space.

Or PyPy, whose whole point is that it's written in Python.


Yes. Because of the language safety. This is a feature, and everyone that has written “malloc” and read about rust understands where it comes from. “Lisp --> better browser” is not so immediate.

Re. Pypy, the advantage of “the language you like written in the language you like” is also pretty immediate.

My point being, dont leave the target of the message do the work of researching and understanding. Just selling the language is normally not enough.


Lisp sould ring a bell for high hackability ! (good points though)


I am genuinly curious; I promise I am not just trolling here...

Why is it important to call out the fact that is in [lang]?


There is something quite special about Common LISP. I've written about it a few times, you can find a short article here:

https://atlas.engineer/technical-article/why-lisp.org

it also contains some nice links at the bottom to further articles. I'd be curious to know your opinion, thanks!


Looks interesting. I presume it'd be easy to add a Vim key mode instead of the current Emacs-y one?


Hi, one of the developers here, yes, we spent a lot of time to make sure that writing a VI style keybinding system (and properly) without all of the EVIL compromises would be possible


Can't wait for the next evil-mode happening


meta key isn't working for me on my 2017 macbook pro... am i doing something wrong?


Probably not, the latest Mac version is a bit flaky, new release coming soon ~1-2 weeks


I used to use Conkeror, but it hasn't seen any development in a longtime.

Hopefully this is or something else will be able to take it's place and eventually implement basic features that for some reason or another no current web browsers have: tiling frames, emacs keybindings, and a powerful extension language that allows for the modification of core structures (analogous to elisp for emacs).


We have emacs bindings, and everything can be extended in CL, live. You can even connect to emacs while it is running in both directions. We wrote an interesting article about that here:

https://next.atlas.engineer/article/emacs-hacks.org


I didn't spend time trying this but when I played with the Next Browser about a month ago it occurred to me that one could mix in application code with a local web server component and a web interface to combine browsing and a custom application in one app. One possible app idea might be a research tool that allowed annotating material on the web - really, anything app that used web content.


That’s awesome. Does it support any ad blockers? Or would I have to go with a dns solution?


DNS solution for the time being, but all provisions are in place for ad-blocking support, just have to tweak this here: https://github.com/atlas-engineer/next/blob/master/source/re...


Just tried it out! Pretty great. Reminds me a conkeror a lot.

One nitpick though: I feel like you should be able to open a new buffer with C-x C-f, and maybe open in the same buffer with C-x f? C-l in emacs centers the text on where your cursor is, so using that and M-l to open links/buffers feels a little odd.

Edit: Also, I don't know if you maintain the AUR package. But I had to install libfixposix manually as it wasn't listed in the dependencies and had to restart the compilation from the sbcl debugger.


We do not maintain the AUR package. None of us use Arch. Libfixposix is a new dependency released a few days ago, we just updated the documentation.

Feel free to add new keybindings in your init.lisp file :) Its just as hackable as Emacs :)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: