Rory Nolan remembers the first time he was banned from Facebook. It was October 8, 2020, the day that news broke that a militia group, in concert with an FBI informant, had planned to kidnap Michigan Governor Gretchen Whitmer. He was talking to a friend on Facebook Messenger when, a few minutes later, he got a text message from the same friend asking if Nolan had blocked him. Puzzled, he checked his Facebook account—he had been booted. He tried to appeal the ban, only to be issued an automated message that his account was permanently revoked “because it did not follow our Community Standards. This decision can’t be reversed.” Soon his Instagram account received the same treatment.
Nolan belongs to historical reenactment groups that sometimes dramatize Revolutionary War-era militias (you can begin to see the problem), and he manages the Facebook and Instagram pages for several of them. He tried to establish new accounts under new email addresses, but they didn’t last long before getting swept up in the same moderation process. Again, they were banned with no possibility of appeal. And like that, Nolan’s social media presence—and much of his social life—quietly winked out of existence.
Losing one’s Facebook account rates low on the scale of tragedy, but Nolan’s experience is revealing of the strange, often clumsy, and imprecise ways that Facebook tries to manage content on its platform. Many people don’t know they’ve violated a Facebook policy until they run into the buzzsaw of the company’s automated system. Sometimes they seem no more sophisticated than tracking certain forbidden keywords and flagging pages associated with them. Crudely designed, with little ability to appeal decisions to actual human beings, these systems seem to be more equipped to disable pages like Nolan’s than to respond to right-wing extremism and threats of violence.