What I find problematic, especially for teams that rely on GitOps and GitHub being (relatively) stable, is the fact that GitHub's status page can't be relied on. You arrive at the conclusion, "this must be GitHub," but the status page rejects the notion, and so you go digging. That can lead to a lot of toil and wasted time, in addition to sideways looks paired with comments like, "but status shows green."
A sick person with symptoms is clearly sick, but sometimes they're asymptomatic. That's what the GitHub status page has become. When it's red and showing problems, you know you have problems. When it's all green? You know you MIGHT have problems.
That can make outages like this harder to triage, since reality is a black box.
Github is fast in updating their status compared to other companies. Amazon has to be bombed, on fire, and in the news before their status page shows anything about any ongoing issues.
I assume these companies have some kind of uptime guarantee in their contracts, because they seems to care more about covering their asses than about informing customers.
Gotcha, that's what I figured but I was still curious. I've got a side-project that was originally designed for traditional GitOps but got OCI support as a feature submission from a user. I've been seeing more usage of the OCI backend these days and originally figured it was a side-effect of the project domain (Podman Quadlets) but maybe it is an over all field shift.
In the last couple of weeks, I've been moving a deployment stack off ArgoCD pointing at Git, and ended up settling on Flux fetching OCI artifacts from ECR. The artifacts are built and pushed by GitHub Actions. It seems like a nice way to "have GitOps" without needing GitHub to be up all the time.
> November 4, 2026: We’ll have a brownout of the removal of the ssh-rsa signature type (i.e., RSA keys using SHA-1) and the diffie-hellman-group-exchange-sha256 key exchange algorithm.
nothing :) hope they'll release a statement soon, this is was an assumption on my side, it might be totally wrong. I had intermittent issues with basic git pull/push ops locally so I went to dig a bit. ps: if you do have money to spare, consider donating to the EFF (https://www.eff.org) though B)
Our deploys are tied to GitHub actions and it is becoming a liability at this point. What alternatives do people use? I mean for FOSS there are some decent stuff but we need a reliable provider for private repos.
Like with all things, it really depends on having someone competent stand it up and run it. I think we've already reached the point that this is totally worth it for mid-sized companies as opposed to giving MS money for GHA.
We use GitLab where I work. The only problem is it calls pull requests "merge requests," and I don't think I'll ever get used to saying "MR" instead of "PR."
Because in the good old days when git was still distributed, a merge often started with sending someone an email to obtain your newly-coded commits via `git pull`ing your "gpu-drivers-for-linus" branch from some URL pointing to your personal GitWeb install. In other words: it was a request to pull your code.
Similarly: Why are the Gitlab people so happy to merge? Where is my "Rebase Request", "Squash Request", or "Cherry-pick Request"?
Same for me, ssh access was intermittent, but at least I have reliable local builds when GitHub actions is out but access works. With AI running Jenkins is not that hard. I build large cuda containers so cost alone ruled out hosted builds for my usecase. There's a bit of update churn but nothing broke so far. I ran around 2000 builds last year. I run it on epyc 7v12 128gb ddr4 host.
My deployments are tied to their container registry, so I'm equally screwed... Been thinking of going back to Dokku, I started working on a third wrapper around Dokku (yeah, I've literally done this twice already), and then self-host and self-provision... I honestly thinking seriously about this.
Forgejo backed by Woodpecker CI running on Kubernetes (Talos) has been amazing! Ditch Github Actions syntax for the simplicity of Woodpecker and containers. You'll thank me later.
+1 for Blacksmith. You need to have a Github org to use it as it won't work on personal accounts, but it feels like a magic want that just makes runners 2x as fast.
I have it for all my stuff (dozens of repos and hundreds of PRs/builds per week) and it's only once in the last year that I've exceeded the free resource levels. I don't know if it's just extra capacity or they're burning VC dollars in the hopes of inspiring posts like this, but either way I feel like I'm taking advantage of them at this point.
Your proprietary personal hobby project doesn't need an entire publicly-accessible Forge, it can just live on your local drive or home server. It is also not critical enough that GitHub occasionally being down is a big deal.
If you're looking for a publicly-accessible Forge SaaS for your private repos and are willing to pay for it, you are almost certainly a company developing proprietary software.
self hosted forgejo has been my saviour. im thinking in acquiring a new NUC server to increase compute and storage to my homelab to add some new runners
Seconded. Also very easy to set up. I’ve been running it for years, along with webhooks to kick off testing and website deployment jobs. I especially like the swagger browser that generate API calls for you to try in curl. I use the API pretty extensively.
You don't even have to migrate off GitHub if you don't want to.
Begin by refactoring the deployments so they're driven by checked in shellscripts, that accepts a git reference to deploy (prefer immutable commit SHAs over mutable branches/tags). Make it use the stuff it already has checked in, and make the actual GitHub Actions stuff as small and concise as possible, ideally each "run" should just call a shellscript with the git reference.
Then move so the deploys happen not on GitHub Actions itself, but on a "intermediate deploy host" that has the source code and access to the credentials you need, and make this host have one secure entrypoint, where someone (or something) can enter and run shellscripts, after pushing their code as a normal git push. Then switch so the deploys from GitHub Actions instead use this new host to do the deploys. Lock down this host hard.
When GitHub and/or GitHub Actions go down, the developer responsible for deploying stuff, should be able to get temporary git and ssh access to this host, can push any git data straight to the host, and their user on this host can run the deploy scripts. Keep in mind that nothing in your deployment process should depend on stuff that isn't also available on this "deploy host", if you have multiple synced repositories and so on.
Now things mostly work the same as people are used to, with the added benefit that when things go down again (like next week), you'll still be able to continue to deploy stuff.
I've done this refactor/migration for tens of companies in the last year, more or less following this exact template for all of them, with no issues reported from either of them, but lots of praise :)
Git itself usually works, but we gave up on GHA and use https://www.rwx.com/ for tests, previews, and deployments. It’s 2x to 10x faster than GitHub Actions and testing uncommitted changes on real CI infrastructure is a total game changer.
they marked incident with Pull requests, Git operations and issues - but i don't see any outage in the bars for today other than pull requests? what's the point of showing uptime there then? https://mrshu.github.io/github-statuses is more reliable than the official github status page.
I think blockchain ledger wouldn't be enough, there needs to be AI agents on top of that, that combined with some kind of harness layer, plus, ideally agents gossiping with each other and sometimes fighting with each other to get best results.
For most people here GitHub is already a paid platform. Our companies rely on it as a core part of our development workflows, and we pay a decent bunch of money for it. If it was only the free part of GH having issues, it wouldn't get even remotely near this kind of exposure.
Developers are expensive. If I can't ship a feature on time because GitHub issues keeps forcing me to switch to different tasks to work around git/PR/ticket/Actions downtime, then that costs my company real money - despite us not being the source of the AI DDoS that GitHub has happily been welcoming.
To me, the situation is very simple: either 1) rein in the AI deluge, or 2) isolate paying customers from it, or we'll move to an alternative.
For businesses the current state of unreliability would be unacceptable if it was completely free, the fact that it is actually paid just adds insult to injury.
I managed to push a commit over ssh but then the client threw an error before the push fully completed. I can see the commit on the web, but my client still thinks it isn't pushed. And now when I try to fetch I get "permission denied" after a long wait, which is scary. I first thought something got hacked.
Maybe they need a better AI model for their vibe coding. That'll fix everything, right? At least that's what my friends keep telling me when I say that the free LLMs I played around with a bit are incredibly dumb.
GitHub outages can be a drain on productivity.
If your stack is pulling artifacts from GitHub, and this consistently fails during GitHub outage, you can consider having a file-cache in between.
Some of our customers unexpectedly started using the service we're running as a file-cache around GitHub to mitigate GitHub outages.
A sick person with symptoms is clearly sick, but sometimes they're asymptomatic. That's what the GitHub status page has become. When it's red and showing problems, you know you have problems. When it's all green? You know you MIGHT have problems.
That can make outages like this harder to triage, since reality is a black box.
Don't know about you but at this point I've learned to recognize and ignore GitHub issues even when they haven't admitted to it yet
I assume these companies have some kind of uptime guarantee in their contracts, because they seems to care more about covering their asses than about informing customers.
We have outages on our books that they never reported even after we ticketed them.
They cook the stats like an inner city police department.
> November 4, 2026: We’ll have a brownout of the removal of the ssh-rsa signature type (i.e., RSA keys using SHA-1) and the diffie-hellman-group-exchange-sha256 key exchange algorithm.
I guess something went sideways.
Disclaimer: I am building a code forge at https://juju.bi
You're literally replacing a well designed resilient system with a single point of failure run by a different company...
Wait, Jenkins?
I am requesting to merge my changes into the main branch.
Similarly: Why are the Gitlab people so happy to merge? Where is my "Rebase Request", "Squash Request", or "Cherry-pick Request"?
related tidbit: in git, "pull" is basically shorthand for "fetch + merge".
https://woodpecker-ci.org/docs/usage/services
Gitea Enterprise would be worth looking at.
https://about.gitea.com/products/gitea-enterprise/
I have it for all my stuff (dozens of repos and hundreds of PRs/builds per week) and it's only once in the last year that I've exceeded the free resource levels. I don't know if it's just extra capacity or they're burning VC dollars in the hopes of inspiring posts like this, but either way I feel like I'm taking advantage of them at this point.
If you're looking for a publicly-accessible Forge SaaS for your private repos and are willing to pay for it, you are almost certainly a company developing proprietary software.
It's a great service though
https://gitsocial.org/
Begin by refactoring the deployments so they're driven by checked in shellscripts, that accepts a git reference to deploy (prefer immutable commit SHAs over mutable branches/tags). Make it use the stuff it already has checked in, and make the actual GitHub Actions stuff as small and concise as possible, ideally each "run" should just call a shellscript with the git reference.
Then move so the deploys happen not on GitHub Actions itself, but on a "intermediate deploy host" that has the source code and access to the credentials you need, and make this host have one secure entrypoint, where someone (or something) can enter and run shellscripts, after pushing their code as a normal git push. Then switch so the deploys from GitHub Actions instead use this new host to do the deploys. Lock down this host hard.
When GitHub and/or GitHub Actions go down, the developer responsible for deploying stuff, should be able to get temporary git and ssh access to this host, can push any git data straight to the host, and their user on this host can run the deploy scripts. Keep in mind that nothing in your deployment process should depend on stuff that isn't also available on this "deploy host", if you have multiple synced repositories and so on.
Now things mostly work the same as people are used to, with the added benefit that when things go down again (like next week), you'll still be able to continue to deploy stuff.
I've done this refactor/migration for tens of companies in the last year, more or less following this exact template for all of them, with no issues reported from either of them, but lots of praise :)
I joke, the QoL is really nice
> git@github.com: Permission denied (publickey). fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
git -c credential.helper= -c 'credential.helper=!gh auth git-credential' pull https://github.com/myorg/my-repo.git
git config --global url."https://github.com/".insteadOf git@github.com:
so it pushes / pulls using HTTPS
when things are back to normal run:
git config --global --unset url."https://github.com/".insteadOf
https://rubyonrails.org/2026/11/4/Rails-8.0.5.2-and-8.1.4.1-...
new incident with actions now - https://www.githubstatus.com/incidents/h02n9572mx9y.
Perhaps these types of posts should just be pinned at this point.
Developers are expensive. If I can't ship a feature on time because GitHub issues keeps forcing me to switch to different tasks to work around git/PR/ticket/Actions downtime, then that costs my company real money - despite us not being the source of the AI DDoS that GitHub has happily been welcoming.
To me, the situation is very simple: either 1) rein in the AI deluge, or 2) isolate paying customers from it, or we'll move to an alternative.
For businesses the current state of unreliability would be unacceptable if it was completely free, the fact that it is actually paid just adds insult to injury.
What were the 60'000 people exiting Microsoft doing if not improving Github?
I can live with the occasional outage but this is a constant problem.
https://www.githubstatus.com/incidents/xr3myzj7ms51
https://www.githubstatus.com/incidents/h02n9572mx9y
Get your act together, clowns.
Maybe they need a better AI model for their vibe coding. That'll fix everything, right? At least that's what my friends keep telling me when I say that the free LLMs I played around with a bit are incredibly dumb.
It's up to you if you want more of this instability. Perhaps you should consider a self-hosted alternative.
[0] https://news.ycombinator.com/item?id=50307843
Some of our customers unexpectedly started using the service we're running as a file-cache around GitHub to mitigate GitHub outages.
We wrote about this use case in our blog: https://www.stablebuild.com/blog/protect-your-builds-from-gi...