Man watered dish to cool it down but overheating knocked it offline for 7 hours.
A Starlink beta user in Arizona said he lost Internet service for over seven hours yesterday when the satellite dish overheated, demonstrating one of the drawbacks of SpaceX’s broadband service. When the user’s Internet service was disrupted, the Starlink app provided an error message saying, “Offline: Thermal shutdown.” The dish “overheated” and “Starlink will reconnect after cooling down,” the error message displayed.
The user, Martin, posted a screenshot of the error message on Reddit. He contacted Starlink support, telling him, “Dishy will go into thermal shutdown at 122F and restart when it reaches 104F.” Martin gave the dish a little water so it could cool down. He pointed a sprinkler at Dishy, and once it cooled enough to turn back on, “I immediately heard YouTube resume playback,” he wrote yesterday.
But the Internet restoration was short-lived, Martin told Ars in a chat today.
“The fix was temporary,” he told us. “When I stopped the sprinkler, [the dish] heated back up and would cycle back on for a few minutes and go back down for thermal shutdown. The overheating started that day at about 11:30 am and came back for good at about 7 pm… I’m currently headed to a hardware store to get materials to build a solar shade/sail around the dish to see if it doesn’t impact connection and speed.”
Martin uses the ground behind his house to set up his dish because it is the only spot with no obstructions. But there’s “no shade to speak of,” he wrote in the Reddit comment thread.
Thermal shutdowns affect other users.
Officially, SpaceX has said that “Dishy McFlatface” is certified to operate from 22° below zero up to 104° Fahrenheit. He said temperatures reached about 120° yesterday in Martin’s town of Topock, near Arizona’s border with California. Though Dishy doesn’t go into thermal shutdown until it hits 122°, the dish can get hotter than the air temperature.
“I’m thinking the radiating heat from the ground is effectively cooking the bottom of the dish, [while] the top of the dish is cooked by the sun,” Martin told Ars. In addition to the shade he’s building, Martin said he is “waiting for permitting for a HAM radio tower” to lift the dish off the ground to help keep it cool enough to operate.
Martin said he also had very short outages on several days since last week, but service came back before he had time to confirm whether they were caused by heat. SpaceX told users to expect periodic outages during beta, so Martin’s previous outages could have been due to heat or satellite availability.
Another user in Virginia experienced a half-hour outage due to overheating on a day with temperatures in the low 80s, according to a Reddit post two months ago.
Martin’s post spurred a reaction from a beta user who reported thermal shutdowns. “You’re not the only one. My Starlink is located 50 miles south of Grand Canyon in a remote area,” one person wrote yesterday. “It’s been off and on also. It stopped today one hour after the cooldown period but quit again as [of] ~12:30. The last reported temp at my weather station was 103 degrees.”
The 122° F shutdown temperature was mentioned three weeks ago in a Reddit post by a user who had also been given the figure by Starlink support. “‘That’s it??’ was my thought. On a 90-degree day, the rooftop of my house can be around 125 degrees,” that user wrote.
“Are you sure that wasn’t Celsius?” another asked. (122° C converts to 251.6° F.)
Like Martin, other Starlink users may have to find creative ways to keep their dishes cool as the summer months arrive.
Dishy’s heat management
As written in December, a teardown of Dishy McFlatface showed some heat-management components, including a metal shield peppered with blue dots made of thermally conductive material that conducts heat away from the PCB and into the shield.
Ken Keiter, the engineer who performed the teardown, was interviewed by Vice’s Motherboard section for a story about the Arizona resident today:
Keiter told Motherboard that while reasonable consideration was given to heat dissipation in Dishy’s design, he could see the potential for problems.
“The phased array assembly comprises a PCBA (printed circuit board assembly) adhered to an aluminum backplate which serves several purposes—acting as RF shielding, providing structural rigidity, and, most relevantly, acting as a radiative thermal mass (heat sink) for the components on the PCBA,” Keiter said.
Heat is funneled from the circuit board to the aluminum backplate using a foam-like thermal interface material (TIM). The backplate resides in a weather-sealed cavity containing a small amount of air. Keiter said that as this backplate heats up, the air surrounding it also heats, transferring thermal energy via the plastic enclosure to the outside environment.
“Here’s the problem: at some point, the combined thermal energy being absorbed by Dishy’s face and dumped by the components into the backplate, the air surrounding it, and the enclosure exceed the amount that is being dissipated to the outside environment,” he noted.
Keiter said that software changes could “make the system more thermally efficient” but that SpaceX may need to make “a significant hardware revision for the commercial launch.” He called it “a tricky engineering problem with insanely tight constraints.”
SpaceX seeks stability before exiting beta.
The Starlink public beta began in October 2020, and there’s still no word on when it will hit commercial availability. But the service could happen within months, as SpaceX CEO Elon Musk has said that Starlink will be available to “most of Earth” by the end of 2021 and the whole planet by next year. Still, SpaceX expects limited slots in each geographic region because of capacity constraints.
SpaceX is seeking Federal Communications Commission permission to deploy up to 5 million user terminals in the US. Over 500,000 people have ordered Starlink, and Musk has said he expects all of those users to get service. But he also said that SpaceX will face “more of a challenge when we get into the several million user ranges.” The most significant limitation would be in densely populated urban areas; rural users would have better odds of getting service.
As noted earlier, Starlink warns beta users to expect “brief periods of no connectivity at all”
Even if they don’t run into thermal shutdowns.
“We still have a lot of work to do to make the network reliable,” SpaceX president and COO Gwynne Shotwell said in April. “We still have dropped, not necessarily just because of where the satellites are in the sky.” SpaceX will keep the service in beta “until the network is reliable and great and something we’d be proud of,” Shotwell said.
The Verge reviewed Starlink last month and found frustrating reliability problems. “Like the similarly over-hyped mmWave 5G, Starlink is remarkably delicate. Even a single tree blocking the dish’s line of sight to the horizon will degrade and interrupt your Starlink signal,” The Verge wrote.
Starlink is only part of the solution.
As Shotwell promised, the service will become more stable when SpaceX moves it from beta to general availability. Even in beta, Starlink provides much-needed connectivity to people without other options. If SpaceX brings reliable broadband to a few million users, that would be a success, but there may be tens of millions of Americans without access to high-speed broadband. Tens of millions of others have to pay whatever the cable company demands because there’s no competition where they live.
Widespread fiber-to-the-home deployment would make a more considerable difference for more Internet users than Starlink. President Joe Biden pledged to lower prices and deploy “future-proof” broadband to all Americans. Still, he’s already scaled back his plan in the face of opposition from Republicans and incumbent ISPs. AT&T has been lobbying against nationwide fiber and funding for municipal networks, and AT&T CEO John Stankey expressed confidence last week that Congress will steer legislation in the direction that AT&T favors.