2022 Rogers Communications outage
File:Cloudflare IP prefix withdrawals during 2022 Rogers Communications outage.svg
Chart of withdrawals from IP prefixes from Rogers Communications on July 8, 2022, between approximately 05:00 and 16:30 UTC, as measured by Cloudflare[1]
|
|
Time | 4:30 a.m. (EST)[2] |
---|---|
Duration | 1 day – present (currently ongoing) |
Date | July 8, 2022 | – present
Type | Network outage |
Outcome | <templatestyles src="Plainlist/styles.css"/> |
On July 8, 2022, Canadian telecom provider Rogers Communications experienced a major service outage affecting more than 12 million users of Rogers' cable internet and cellular networks,[3] including those of subsidiary brands Rogers Wireless, Fido, Cityfone, and Chatr. It also impacted internet service providers with wholesale access to the Rogers network, such as TekSavvy,[4] as well as various other information systems nationwide that rely on the Rogers network, including Interac and some federal government services. Multiple international web monitoring companies observed the outage. To date, the outage is not believed to have been fully resolved.
The event spurred new government policies requiring all telecommunications providers to provide mutual assistance to each other in the event of an outage, offer emergency roaming service for rivals' affected customers, and establish plans for how to communicate to the public about said incidents.[3]
Rebates for customers are estimated at cost to Rogers a range of $28 million[5] to $70 million,[6][7] while the economic toll to the Canadian economy was projected at $142 million.[8]
Impact
The outage had a significant impact on various information systems nationwide, much of which extended past residential and commercial Rogers customers, due to their dependencies on Rogers infrastructure:
- Around 25% of Canada lost Internet connectivity, according to the UK cybersecurity organization Netblocks.[9][10] Over 12 million subscribers were affected.[3]
- Interac was taken offline by the outage, most prominently preventing all businesses nationwide (regardless of their internet service provider) from being able to accept debit card transactions. Some stores temporarily closed.[4]
- The outage inhibited the ability to use 9-1-1 services from mobile phones on the Rogers network,[4] resulting in at least one reported case of a death that could have been avoided if not for the inability to contact emergency services during the incident.[11]
- Medical appointments through the Niagara Health System for oncology patients with radiation therapy were rescheduled, with a plan to transfer patients requiring emergency care to Hamilton Health Sciences.[12]
- Along with the inability mentioned above to accept debit, some public transport agencies reported other computer system issues tied to the outage.[4]
- The outage affected some Canada government services, such as Service Canada, Canada Revenue Agency, and passport offices.[13]
- The Canadian Federation of Independent Business (CFIB) reported small businesses lost anywhere from a few hundred to several thousands of dollars. [14]
- In Toronto there was some dependency on Rogers. One quarter of all traffic signals relied on their cellular network for signal timing changes. The Rogers GSM network was also used to remotely monitor fire alarms and sprinklers in municipal buildings. Public parking payments and public bike services were also unavailable. It was also the sole provider of cable TV for the city. [15]
- Canadian musician the Weeknd was forced to postpone a concert that night at Toronto's Rogers Centre—which was to be the first stop on the After Hours til Dawn Tour—due to "service outages impacting venue operation".[16]
Recovery
Rogers had begun to slowly restore service that evening, but CEO Tony Staffieri stated there was no estimated time for when services would become fully operational again.[17] The next day, Rogers stated that it had restored service to the "vast majority" of its customers; however not all service has been restored across the country.[18] As of July 11, a spike of service issues was reported to the web traffic monitoring company Ookla's Downdetector, with some users reporting that service had not recovered for them.[19][20]
Causes
A report by Cloudflare suggested that the outage was due to internal, rather than external, causes. It identified spikes in BGP updates, as well as withdrawals of IP prefixes, noting that Rogers was not advertising its presence, causing other networks to not find the Rogers network.[1] As of the day after the outage, the cause remained unknown. Public Safety Canada stated that it was not a cyberattack.[21] The outage was later said to be caused by a maintenance upgrade that caused routers to malfunction.[22] A similar outage occurred in April 2021 which was attributed to a software update.[5]
Reactions
Critics have cited the outage as an example of the impact of Canada's telecom oligopoly, and an argument against Rogers' proposed acquisition of Shaw Communications (which would further consolidate the industry).[23][17] NDP leader Jagmeet Singh accused the Liberal Party government of "protecting the profits of telecoms giants",[24] while Conservative MP Michelle Rempel Garner called for an investigation, arguing that "given the critical infrastructure that's affected, and that the CRTC itself is affected, the cause of the Rogers outage should be immediately explained."[24] A spokesperson for the Canadian Radio-television and Telecommunications Commission (CRTC) promised that the organization would examine the outage and "put in place the necessary measures to prevent it from happening again."[9]
Critics also highlighted the importance of consumers and businesses diversifying their telecommunications services, such as businesses having backup service contracts with multiple providers and consumers reducing their dependence on bundled "all-in-one" cable, internet and phone service packages.[24]
Response
Rogers plans to credit customers for five days of lost service.[25] On July 10, 2022, Tony Staffieri, president and CEO of Rogers Communications, sent an email to all customers to officially apologize for the incident and inform them that Rogers would credit all customers' accounts for Friday.[26][27] The last time they credited consumers on such a scale was soon after their April 19, 2021 outage, when "wireless calls, SMS and data services were down across Canada for almost an entire day because of an issue with a software update".[28]
On July 11, the CEOs of many of Canada's major telecommunications companies (Rogers, Bell, Telus, Shaw, Vidéotron, SaskTel and EastLink) were all invited to take part in a conference call with federal cabinet minister François-Philippe Champagne to discuss a new "mutual assistance" framework of cooperation between the companies in emergencies to prevent the recurrence of a similar incident in the future.[29] On the same day, Champagne announced new policies requiring all telecommunications providers to provide mutual assistance to each other in the event of an outage, offer emergency roaming service for rivals' affected customers, and establish plans for how to communicate to the public about said incidents. Champagne said that the companies would have 60 days to reach agreements with each other to that effect.[3]
The House of Commons of Canada are discussing requests to conduct a study of the outage involving members of parliament from the Conservative, Liberal, New Democratic Party, and Bloc Québécois. [30]
A class action lawsuit was filed in the Superior Court of Quebec which requested all customers receive a $400 rebate, reasons cited included negligence of the CRTC regulations to make 9-1-1 calls available at all times.[31] The CFIB requested Rogers to give a free month of service to compensate for losses to others.[6]
As a result of the outage the city government of Toronto reviewed its legacy contract for sourcing with a variety of providers. Council meetings were planned to review full cost to the economy and the ability to use local ConnectTO infrastructure instead in future.[15]
See also
References
- ↑ 1.0 1.1 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ 3.0 3.1 3.2 3.3 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ 4.0 4.1 4.2 4.3 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ 5.0 5.1 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ 6.0 6.1 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ 9.0 9.1 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ 15.0 15.1 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ 17.0 17.1 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ 24.0 24.1 24.2 Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Alexandra Posadzki, "Rogers outage prompts meeting with telecom execs, Industry Minister to discuss ‘mutual assistance’ framework, sources say". The Globe and Mail, July 11, 2022.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.
- ↑ Lua error in package.lua at line 80: module 'strict' not found.