GitHub, Salesforce and SharePoint Went Down in Four Days. Nobody Attacked Them.

Between Sunday, September 13 and Wednesday, September 16, three of the largest software-as-a-service platforms in the world lost core functions for hours. In each case the company’s own explanation points inward.

GitHub’s postmortem names the cause as an internal data-cleanup job, one that saturated the database cluster behind nearly every authenticated request. Salesforce says requests stalled while waiting on an internal legacy login service, during its own Dreamforce conference. Microsoft points to a configuration change, made to how its servers deliver page-rendering code, which left SharePoint Online sites blank until it was reverted. No attacker, no upstream provider and no hardware failure appears in any of the three accounts.

The three incidents have a similar operational shape, even though the published root causes are not equally complete. In each, trouble inside a shared service path spread well beyond the component where it began. It is a familiar failure shape in hosting too. Control panels, authentication layers, billing systems and shared databases can sit across large parts of a customer-facing platform, so a cleanup script, a stalled authentication path or a configuration push can produce provider-wide impact.

Key facts

• GitHub, September 13, 08:43 to 10:44 UTC. About 28 services degraded. The safeguard pacing a cleanup job watched replica lag while the primary ran out of connections.
• Salesforce, September 16, from about 07:50 UTC. Hundreds of instances across six countries. Requests stalled on an internal login service and consumed the resources healthy requests needed.
• SharePoint Online, September 16, 16:04 to 17:30 GMT, incident SP1472983. A configuration change to page rendering left sites blank until it was reverted.
• The common thread: none of the three blamed an attack or a third-party provider. GitHub and Microsoft name an internal change. Salesforce has traced the failure path but not yet published a root cause.

GitHub: The Safeguard Watched Replica Lag While the Primary Ran Dry

GitHub’s account is the most detailed of the three, and it reads like a checklist of what a shared database can do to everything built on it. An internal data-cleanup job began writing to a shared database cluster that stores permission data at 07:33 UTC on September 13. That cluster, in GitHub’s words, is read on nearly every authenticated request.

The job had a safeguard. It paced itself according to one health signal, how far the database replicas were lagging behind the primary, and that signal stayed low. The load was accumulating on the primary itself, and when the primary ran out of available connections, requests that needed it could not complete.

The failure then spread the way failures spread through a request path with long timeouts. Blocked requests piled up in web servers. Retry loops on token creation kept re-sending writes that were already failing, which held the database saturated.

Monitoring declared the incident at 08:50 UTC and the first status update went out at 09:16, describing degraded availability for API requests, Issues, Pages and Pull Requests. By 09:36 GitHub attributed the errors to increased database replication delays on the cluster it calls collab, causing “increased error rates in authorization endpoints and follow-on increased error rates across the system.” Recovery came at 10:44 UTC, after GitHub shed internal load and paused the job.

The impact figures GitHub disclosed are specific. By its count, new account signup failed more than 90 percent of the time, and creating an issue through the web interface failed for about 96 percent of attempts. At peak, 8.8 percent of requests to create GitHub App installation access tokens failed, and token issuance for Actions affected roughly 4 percent of workflows.

GitHub’s remediation list is worth reading as a template. It commits to:

• rate-limiting background jobs that touch shared, customer-serving databases
• automatic pausing and paging on primary load, not replication lag alone
• surfacing running background work alongside database health dashboards
• bounding the retry loops in the token-issuing path
• adding request-level timeouts, so one unhealthy database cannot consume shared web server capacity
• breaking apart the shared cluster within two weeks, moving service-specific and authorization data out of it

Lorin Hochstein, whose September 19 post on Surfing Complexity walks through the incident, calls it “a great example of a gray failure”, where internal monitoring registers the system as healthy while users are not. In his reading the safeguard misjudged where the safety boundary was, and no signal appeared until it had been crossed.

Salesforce: A Legacy Login Service on the Second Day of Dreamforce

Salesforce’s outage began at about 12:50 AM Pacific on September 16, which is 07:50 UTC, and shortly before 09:00 UTC the company confirmed that multiple instances across all regions were affected. The Register reported hundreds of instances affected in the USA, Japan, India, the UK, France and Germany. Customers, in Salesforce’s own status wording, could “experience severe delays, intermittent errors, or an inability to access some Salesforce services.” The status wording adds that the problem can stop customers opening a new support case.

The root cause statements evolved through the morning. At about 09:10 UTC Salesforce said its investigation indicated “requests are stalling while waiting on a response from an internal login service, which is using up available server resources.” It then described the trigger as an external dependency failure affecting its legacy login server, and later as increased load on a core system component that limited its capacity to process requests.

Computerworld reports the company confirmed there were no issues with third-party infrastructure, so “external” here means external to the affected component, not external to Salesforce. A fix was validated on a test instance around 11:00 UTC and a fleetwide rollout was under way by 11:27 UTC.

Rollout did not mean recovery. At 14:00 UTC Salesforce said the fix had not fully restored every instance and that it had received reports of scheduled jobs not running for customers who could once again log in.

Recovery and closure are not the same timestamp here. By 15:50 UTC, Salesforce said mitigation was in place across nearly all affected instances, with the remaining disruption narrowed to a subset of Hyperforce environments and targeted restarts and manual recovery still running. Computerworld describes the disruption as roughly seven and a half hours long. Salesforce validated for several more hours and marked the incident resolved at 18:59 UTC, on the second day of Dreamforce.

The phrase that matters for anyone who runs a platform is “legacy login server.” Abbas Jaffery of Info-Tech Research Group told Computerworld that a component’s age matters less than “its position in the dependency graph, its blast radius, and the quality of its isolation and failure handling.” A login service sits at the top of the dependency graph by definition. In a hosting company the equivalent is the single sign-on layer between the customer portal, the control panel and the billing system, often the oldest code in the building and the one path every customer session must pass through.

SharePoint: A Configuration Change That Reached Production

Microsoft’s incident was the shortest and the plainest. Between 16:04 and 17:30 GMT on September 16, the same afternoon Salesforce was still recovering, users trying to load SharePoint Online sites or pages received “Sorry, something went wrong: Thread was being aborted.” Microsoft tracked the event as SP1472983, and attributed it to a configuration change affecting how its servers deliver the code used to render SharePoint pages. It reverted the change, confirmed through telemetry that the problem was resolved, and promised a preliminary post-incident report within two business days and a final one within five.

The sentence Microsoft added afterward is the one that carries the lesson: “We’re further assessing the process by which we validate and deploy configuration changes to ensure future rollouts don’t result in similar impact.” The change had, by implication, passed whatever validation existed.

It is also not the first. The Register lists an Azure incident earlier this year, a Microsoft 365 configuration change that brought down a substantial part of the productivity cloud, and a 2025 change that broke Exchange Online access through Outlook on the web. A configuration push is the most common change a hosting operator makes in a week, to a web server, a DNS zone, a WAF ruleset or a panel setting. It is also the change least likely to go through the review a code release gets.

Three Different Companies, One Failure Shape

Put the three explanations side by side and the differences are in the nouns, not the structure. A background job, a legacy dependency and a configuration change each touched something shared. The blast radius was broad each time, because the affected component sat in a heavily shared request path: a permissions database at GitHub, a login service at Salesforce, the page-rendering pipeline at Microsoft.

What differs is how much each company has said about the protection that failed. GitHub documented it exactly: the pacing safeguard watched replicas while the primary saturated. Salesforce’s public updates describe login requests stalling and consuming the resources healthy requests needed, without yet naming the resilience control that should have stopped it. Microsoft says only that it is reassessing how it validates and deploys configuration changes, after a faulty one reached production.

Two of the three recoveries were a matter of undoing. GitHub paused the job and shed load. Microsoft reverted the change. Neither needed to rebuild anything, which is the argument for giving every routine operational change an immediate stop or rollback path.

Salesforce shows the other side of that. Once a shared dependency is already saturated, recovery becomes its own fleetwide operation: restarts abandoned, a fix validated on one instance, a rollout that did not complete everywhere, manual restarts where it had not, and hours of monitoring before the incident could be closed.

For a hosting provider the useful exercise is to name the equivalent components:

• Which database does every panel login and every API call read?
• Which script runs against it at night, and what does that script watch before it slows down?
• Which authentication service has been in place since the first billing system, and what happens to the portal, the panel and ticketing if it stalls for ten minutes?
• Which configuration changes go out without a staged rollout?

GitHub, Salesforce and Microsoft have now published, in the space of four days, what the answers look like when they are wrong.

About the Data

GitHub’s timeline, impact percentages, root cause and remediation commitments come from the incident entry on its status page, with additional context from Lorin Hochstein’s analysis. Salesforce’s timeline and status wording come from its Trust status updates as reported by Salesforce Ben and The Register, and the confirmation that third-party infrastructure was not involved, along with the analyst quotes, comes from Computerworld. Times for Salesforce are given in UTC, converted from Pacific, Eastern and British Summer Time as reported. Microsoft’s incident ID, times, error text and statements come from The Register. The comparison between the three incidents, and what it implies for hosting providers, is our own analysis of those published accounts.