Not Op. I have production / scale experience in PyTorch and toy/hobby experience in JAX. I wish I could have time time or liberty to use JAX more. It consists of small, orthogonal set of ideas that combine like lego blocks. I can attempt to reason from first principals about performance. The documentation is super readable and strives to make you understand things.
JAX seems well engineered. One would argue so was TensorFlow. But ideas behind JAX were built outside Google (autograd) so it has struck right balance with being close to idiomatic Python / Numpy.
PyTorch is where the tailwinds are, though. It is a wildly successful project which has acquired ton of code over the years. So it is little harder to figure out how something works (say torch-compile) from first principles.
Bifrost is the fastest LLM gateway on the market. Built in Go with careful garbage collection, it adds just about 11 microseconds of overhead at 5,000 requests per second (with 4,100 RPS throughput) on a t3.xlarge instance.
Some features:
• Built-in governance and routing rules
• Supports over 1,000 models from different providers
• MCP gateway included (HTTP, SSE, and console transport)
• Out-of-the-box observability and OTel-compatible metrics
A fellow Godot enthusiast here. Love to see Godot being used in commercially successful indie game like this. In 2021-22 time, I tried (unsuccessfully!) building educational video games for maths using Godot and I have fond memories of being in the flow state while working with Godot. IMO Godot fits well with programmer's brain much better than Unity etc.
I'm honestly in two minds on this one. On one hand, I do agree that valuations have run a bit too far in AI and some shedding is warranted. A skeptical position coming from a company like MSFT should help.
On the other hand, I think MSFT was trying to pull a classic MSFT on AI. They thought they can piggyback on top of OpenAI's hard-work and profit massively from it and are now having second thoughts, thats better too. MSFT has mostly launched meh products on top of AI.
Paras, as a an Indian founder, I've watched your journey for few years now. You are an inspiration and a thoughtful leader. Your "Mental Models for Startup Founders", is a very well written mirror for every founder to look into.
Hope you get some nice time off and go back with vigour to Turing's Dream now...
Andy's collaborator Michael Nielsen has a nice blog post, "using space repetition system to see through a piece of maths"[0]. He makes a point that the idea is to commit more and more higher order concepts to memory. But he does emphasise that Anki is one way to achieve his and a more simpler pen-paper method that you wrote might work.
I was going to say the same thing. For some real world estimation tasks where I don't want 100% accuracy (example: analysing working capital of a business based on balance sheet, analysing some images and estimating inventory etc.) the job done by GPT-4o is better than fresh MBA graduates from tier 2/tier 3 cities in my part of world.
Job seekers currently in college have no idea what is about to hit them in 3-5 years.
I agree. HN's and the tech bubble's bias many people are not noticing is that it's full of engineers comparing GPT-4 to software engineering tasks. In programming, the margin of error is incredibly slim in the way that a compiler either accepts entirely correct code (in its syntax of course) or rejects it. There is no in between, and verifying software to be correct is hard.
In any other industry where just need an average margin of error close to a human's work and verification is much easier than generating possible outputs, the market will change drastically.
On the other hand, programming and software engineering data is almost certainly over-represented on the internet compared to information from most professional disciplines. It also seems to be getting dramatically more focus than other disciplines from model developers. For those models that disclose their training data, I've been seeing decent sized double-digit percentages of the training corpus being just code. Finally, tools like copilot seem ideally positioned to get real-world data about model performance.
Disagreeing here! I think we often overlook the value of excellent educational materials. Karpathy has truly revitalized the AI field, which is often cluttered with overly complex and dense mathematical descriptions.
Take CS 231, for example, which stands as one of Stanford's most popular AI/ML courses. Think about the number of students who have taken this class from around 2015 to 2017 and have since advanced in AI. It's fair to say a good chunk of credit goes back to that course.
Instructors who break it down, showing you how straightforward it can be, guiding you through each step, are invaluable. They play a crucial role in lowering the entry barriers into the field. In the long haul, it's these newcomers, brought into AI by resources like those created by Karpathy, who will drive some of the most significant breakthroughs. For instance, his "Hacker's Guide to Neural Networks," now almost a decade old, provided me with one of the clearest 'aha' moments in understanding back-propagation.
I don’t think we disagree. Education is crucial and the value is enormous, but this hasn’t been what he was paid for in the past. I am hopeful that he finds a way to make this his job more directly than at Tesla or OpenAI as the whole world will benefit.
Most countries in South, South East Asia have made exams as a make and break deal for every student. In India, there are so many kids staying away from home in cities which are just exam preparation centres, with routine news of suicides.
Looking back on my life I think we asians have definitely stretched this way too far. Unfortunately, in high & young population countries like ours these exams are perceived as the only non corrupt way of moving out of low income trap. So this will go on :-(
Spot on! Lot of voice assistants have been following "if we could" line instead of "if we should line". For many straightforward applications, clicking through well defined interface can be the least error prone way to get the job done.
JAX seems well engineered. One would argue so was TensorFlow. But ideas behind JAX were built outside Google (autograd) so it has struck right balance with being close to idiomatic Python / Numpy.
PyTorch is where the tailwinds are, though. It is a wildly successful project which has acquired ton of code over the years. So it is little harder to figure out how something works (say torch-compile) from first principles.