“AI-Driven” bots falsely removing comments

It’s been about a month since our page first experienced the automated comment-deletion events that surrounded  Facebook’s “Community Standards”. Although the flagging and violations continue to happen, it seems they are happening a little less frequently. However, they aren’t officially gone. so we decided to address the topic here.

We have researched the topic on the official Facebook Developer forum, various Facebook help pages, and other community-based boards and resources only to come up empty handed. Early on, we had a few ideas on what might be triggering the violation, only to find that it isn’t as “black and white” as we initially thought. We know that as a community of fans, you’ve had a lot of difficulties with our Facebook comments lately. We want you to let know we are actively working on this disturbance, and are weighing the options to reduce our dependence on the Facebook model as a whole.

In addition, this strange bug, or sequence of user events, or content-specific flags that seem to trigger the AI to issue violations… whatever it is, they are impeding our user’s experience with our Facebook contests. We know that it’s difficult to participate and engage with our posts, only to have those comments removed for no reason. It erodes the trust in the system and we hate the idea of our fans feeling duped or left out. Because of this we wanted to touch base with you all, break down our thoughts on what’s happening, review what you can do about it, and discuss our ideas on where we as a community can go from here.

What is happening exactly?

As you know, when we run our Facebook giveaways, we ask our community of fans and followers to “answer a simple question” and this response is their “entry” in our contest. Past questions from different posts were along the lines of “What’s your favorite part of a Thanksgiving meal?“, “What’s your favorite Pasta Dish?“, or “Sweet or Savory?“. We ask very simple questions that are easy to answer, and help create engagement with our posts. However, we have found that some users when posting answers to these posts, like Lori did in our example to the right, they have been placed in “Facebook Jail” for violating the “Community Standards” on Facebook.

At first, with our “Sweet or Savory” post from Grandma’s Kitchen, we thought perhaps the standards violation was centered around the “Spam” clause in Integrity & Authenticy section of their Community Standards. This idea was based on the fact that most of the answers followed the question… they were single words of either “Sweet or Savory”. So, armed with this idea, we encouraged users to use longer responses that also used the context of the posts such as “I like savory foods better’. Unfortunately, this wasn’t the solution we thought it would be. Fan comments continued to be pulled from our posts with no indication of why.

We offered support and guidance to dozens of fans that connected with us asking how to fix the problem. Often times we shared our insight, and directed them to the Facebook help sections. We also encouraged them to officially request “review” of their comment and offered up support as best as we could. In the end, although dozens connected with us, we know that there were hundreds that had their posts wiped from our contests.

(No) Help from Facebook

After scouring the many pages of Facebook’s help section, the countless articles and posts discussing “Community Standards” and even reaching out to the development community via Reddit, LinkedIn, and Facebook’s own developer community, there still is no clear indication as to what is causing the violations to happen. A few of the pages we have sent our fans to while working with them on the problem are:

• Facebook’s Community Standard’s Policy
• Facebook’s CS policy on Integrity & Authenticity: Spam
• Reporting a Problem with Facebook
• Stops to take when disagreeing with CS violations and comments being flagged

As large as Facebook is, it can feel like efforts to connect through these channels often go unnoticed. Much like every organization, COVID-19 has affected Facebook’s ability to review content as swiftly as it has in the past.

However, as frustrating as it is to feel “ignored” during the process, it is the best route to take if you feel you have been unfairly flagged and have been issued a violation. We continue to encourage our fans to take the time to fill out these reports and issuing a formal “Request for review”. This will help you clear your name of any wrongdoing (hopefully) and it will also draw attention to the fact that the algorithm being used to detect these types of violations is flawed.

The more we as a community draw attention to the fact that these benign posts are being wrongfully flagged, the greater the chances are that something will be done to fix it on a larger scale.

Actions on Spam – an example in process

Although we can’t specifically state that the flagged posts were done so under the guise of “spam” we will take a quick second to look at how Facebook handled spam based on their reports. Facebook’s Community Standards report on “Actioned Content” and the data on actions taken, shows the company took action on 777 million incidents of spam from July to September of 2021. We should help clarify that as stated in an information panel on their page

“Content actioned is the total number of pieces of content that Facebook took action on for spam. it includes both content [Facebook] actioned after someone reported it, and content that [Facebook] found proactively.”

Scoll a bit further and you will see there were 18.6k incidents in Q3 that users appealed for review. The same action we indicated following above. 18.6k incidents from the 777 million incidents found. That is a very low appeal rate. Not many users followed through with the action to fight Facebook.

Then, in the third graph on that page, is the number of actioned spam incidents that were later restored. This is a combination of restored content through the appeal process, or through Facebook’s own restoration process.

In July – September of 2021 there were 777 million incidents of spam that was flagged on the platform. Of those, 1,200 were restored after appeal. Another 20.9 million were overturned by Facebook through it’s own automation.

In review

With just the “Spam” metric from their reports, there is a TON of content that gets pulled from the system. Whereas only a fraction of that content is restored; either through an appeal, or it’s own process of restoration.

On this page itself, you can see Facebook has indicated a decline in manual reviews:

NOTE: Due to a temporary reduction in our review capacity as a result of COVID-19, we could not always offer people the option to appeal. We still gave people the option to tell us they disagreed with our decision, which helped us review many of these instances and restore content when appropriate.

Future NCL Contests on/off Facebook?

As for us, and our awesome fans, we’re not ones to simply stand back and let difficulties stand in our way. This past summer, we had fake accounts trying to send our fans to spoofing and phishing pages. We fought against them and their bogus names like “North-Country Locall” or “NorthhCountry Local”. Our team continued to report them along with your help. They were all removed from Facebook entirely. In the end, we won, thanks to YOU!

Now we face a new dilema: The backbone of our contests run on Facebook. Unfortunately, in many ways, that platform is working against us. This unfairness is why we know it is time to do something. If all our fans can’t participate fairly, then we need to review how we can level the playing field. Especially if the Facebook platform itself is issuing flags for no reason. We would be foolish if we didn’t look into our own resources while we review our processes. It will help us avoid any further unfair Facebook interventions if the contests were self-hosted.

One of the ideas we’ve had was to simply run the contests here on our blog. We can still engage all of our fans and followers through the facebook platform. Social posts and updates will still happen. However, we can host contests on our page and link from social to them. Then we will be 100% in control of the comments, content, and filtering that occurs.

Next Steps for NCL…

It is difficult to weigh a decission like this. The page, and site, had been consistently this way for 6 years. On one hand, Facebook is where we share and engage our fans every day. If we loose our contests from the page, it could be a step backwards. On the other hand, we should look at the success of our web-driven stuff. Contests like our Holiday Hunt, Egg Hunt, Treasure Hunt and more. That’s a platform that has been developed by us for our fans. And it’s a platform that is wholly ours to use as we see fit.

Lets try this out; leave us a comment below. Tell us if you feel a transition away from facebook for our contests is good, or bad.