I've gotten accustomed lately to spending a lot of time in the Github Copilot / agent management page. In particular I've been having a lot of fun using agents to browse some of my decade-old throwaway projects; telling it to "setup playwright, write some tests, record screenshots/videos and commit them to the repo" works every time and it's a great way to browse memory lane without spending my own time getting some of these projects building and running again.
However this means I'm now using the Github website and services 1000x more than I was previously, and they're trending towards having coin-flip uptime stats.
If Github sold a $5000 box I could plug into a corner in my house and just use that entire experience locally I'd seriously consider it. I'm guessing maybe I could get partway there by spending twice that on a Mac Pro but I have no idea what the software stack would look like today.
Is there a fully local equivalent out-of-the-box experience that anyone can vouch for? I've used local agents primarily through VSCode, but AFAIK that's limited to running a single active agent over your repo, and obviously limited by the constraints of running on a single M1 laptop I currently use. I know at least some people are managing local fleets of agents in some manner, but I really like how immensely easy Github has made it.
None of the open weights models you can run locally will perform at the same level as the hosted frontier models. Some of them are becoming better, but the step-down in output quality is very noticeable for me.
> If Github sold a $5000 box I could plug into a corner in my house and just use that entire experience locally I'd seriously consider it. I'm guessing maybe I could get partway there by spending twice that on a Mac Pro but I have no idea what the software stack would look like today.
Right now, the only reasons to host LLMs locally are if you want to do it as a hobby or you are sensitive about data leaving your local network. If you only want a substitute for Copilot when GitHub is down, any of the hosted LLMs will work right away with no up front investment and lower overall cost. Most IDEs and text editors have built-in support for connecting to other hosted models or installing plugins for it.
> I know at least some people are managing local fleets of agents in some manner,
If your goal is to run fleets of agents in parallel, local LLM hosting is going to be a bottleneck. Familiarize yourself with some of the different tool options out their (Claude Code, Cline, even the new Mistral Vibe) and sign up for their cloud API. You can also check OpenRouter for some more options. The cloud hosted LLMs will absorb parallel requests without problem.
Thank you, a bit sad to hear that local inference isn't really at this level of performance yet. I was previously using the VSCode agent chat and playing with both OpenAI and Github hosted models but I switched to using the Github web UI directly a lot since my workflow became a lot more issue/PR-focused. Sounds like I should probably tighten up the more generic IDE-centric workflow and make it a keyboard shortcut to switch around when a given provider is down. I haven't actually used Claude directly yet but I think Github agents often use it under the hood anyway.
An NVIDIA DGX Spark is $4000, pair that with a relatively cheap second box to run GitLab in the corner and you would have pretty good local AI inference setup. (you'd probably have to write a nontrivial amount of software to get your setup where you want)
The local models are just right on the edge of being really useful, there's a tipping point to where accuracy is high enough so that getting things done is easy vs models getting continuously stuck. We're in the neighborhood.
Alternatively, just have local GitLab and use one of the many APIs, those are much more stable than github. Honestly just get yourself a Claude subscription.
The DGX Spark is not good for inference though it's very bandwidth limited - around the same as a lower end MacBook Pro. You're much better off with a Apple silicon for performance and memory size at the moment but I'd recommend holding off until the M5 Max comes out early in the early as the M5 has vastly superior performance to any other Apple silicon chip thanks to its matmul instruction set.
Oof, I was already considering an upgrade from the M1 but was hoping I couldn't be convinced to go for the top of the line. Is the performance jump from the M# -> M# Max chips that substantial?
The main jump is from anything to M5; not because it's simply the latest but because it has matmul instructions similar to a CUDA GPU which fixes the slow prompt processing on all previous generation Apple Silicon chips.
I can't say I'm not tempted looking at the Spark, I could probably save some cash on heating my house with that thing. Though yeah unless there's some good software already built around a similar LLM workflow I could use it'd probably be wasted on me, or spend its time desperately trying to pay for itself with crypto mining.
Adding Claude to my rotation is starting to look like the option with the least amount of building the universe from scratch. I have to imagine it can be used in a similar or identical workflow to the Copilot one where it can create PRs and make adjustments in response to feedback etc.
>Though yeah unless there's some good software already built around a similar LLM workflow I could use it'd probably be wasted on me, or spend its time desperately trying to pay for itself with crypto mining.
A big part of my success using LLMs to build software is building the tools to use LLMs and the LLMs making that tool building easy (and possible).
I tried this for a little while and couldn't really get passionate about it; I have too many other backlogged projects that I was eager to tear into with LLMs and I got impatient. That was a while ago though and the ROI for building my own tools has probably gotten a lot more attractive.
I started building my own tool set because I was doing too many projects with LLMs and getting frustrated by a very real need for organization and tooling to get repetitive meaningless tasks out of the way and to get all of my projects organized so I could see what was going on.
Incorrect, you can use GitHub connect to sync licenses, this allows you to license users under GHEC and GHES at the cost of a single seat. You will need an entitlement for Copilot, but the fact is you can absolutely get access while storing none of your code on .com.
I don't work at Github but I'd read here recently that they've been undergoing a herculean migration from whichever cloud provider they were on to Azure since their Microsoft acquisition, and that it coincides with an increase in outages. I'm guessing that the solution here was probably just to not do that and it's too late.
They weren't on any cloud provider previously. They famously had their own "metal cloud" of managed servers with everything being containerized and managed by Kubernetes. It seemed like it's worked pretty well, especially for their complex git operation tasks which had specific hardware requirements, but the official word is that apparently they're running into scaling limits with finding new datacenter capacity.
Yikes, that's worse, I thought the migration was at least a little politically motivated to reduce a dependency on a competitor like AWS or something. It's not exactly a great advertisement in any case to know that bare metal was more reliable for them than their own infrastructure when they now own it all the way through.
I won't blame Azure directly without a direct reason to, but as a developer often in the market for cloud providers it's definitely not the most reassuring that they're seemingly having so many migration pains.
A bit of an aside, I've only personally used Azure on one project at one company but their console UI had some bizarre footguns that caused us problems more than once. They have a habit of hiding any controls and options that your current logged-in user doesn't have permissions to use. In some cases that manifested as important warnings or tools that I wasn't even aware of (and were important to me!), but the owner of the company and other global admins could see. AWS, at least for a lot of the services last time I used it, was comfortable greying most things out with a tooltip telling you your user is missing X permission, which was way more actionable and the Azure version gave me whiplash by comparison.
On what setup? All YouTube videos load and start playing instantly for me. Every time I've experienced otherwise in the last couple years, it's been my first indication that e.g. AWS is exploding that day
I wonder if it depends what country you are in. I only notice it occasionally when the video won't play in FreeTube or PipePipe (which always has the pause at the start since the last few months) and I'm forced to open an incognito browser tab to watch, and then I realize just how many ads other people are being subjected to before they can even watch the video.
I bet you're using Chrome. Open a video in Chrome and the video is immediately playable, load the same video on the same machine in Firefox and you can expect to wait 5+ seconds for the video to be playable.
I suspect that non-Chrome browsers are being intentionally hobbled
I did a pretty low rigor test and just pulled up one of those 4K videos with the swirling ink in Firefox on my M1 Mac and it seemed to load just as fast. The only difference was that it didn't autoplay the video (because I'm logged out I think) but I clicked it as the page loaded and it played instantly.
I don't doubt at all that Google hobbles their sites on Firefox but at least on my machine they aren't doing a great job of it
I do pay for premium but my impression of the parent was that this was independent of ads. The test I did in the other comment didn't trigger an ad for some reason even though I was logged out, which may be why it loaded so fast.
Ah. The parent mentioned several frustrations that I am not familiar with (presumably since I also pay for premium and don’t block the ads), but my impression was that the delay was caused by the code refusing to play the video until the time slot for the ad had completed even if the ad failed to load (as would happen when blocking the ad http request)
FreeBSD + Waterfox, or Firefox for that matter. YouTube really likes to strangle those who are not in their domain.
If I set my user agent to something like Linux/Ubuntu, it loads just fine. If I set my user agent to some unheard Linux distro, it lags as the same with FreeBSD.
I'm not the other poster but he's probably referring to how your comment seems to only be talking about "pure" LLMs and seems pretty out of date, whereas most tools people are using in 2025 use LLMs as glue to stitch together other powerful systems.
reply