Co-Founder of Integrity Institute
Media executive, CEO & business journalist
Listen now on your favourite podcast app
At the front lines of the battle for truth in the information ecosystem are the social media platforms’ trust and safety teams. Trust and safety teams are the data-science professionals who make sure that social media content conforms to the platforms’ standards. It’s a finger-in-the-dike kind of task, because of both the volume of content—34 million videos uploaded on TikTok every day, for one example–and the judgment needed to distinguish merely obnoxious content from the truly harmful.
And lately, the whole idea has run into significant headwinds, some political, from Republicans who say that trust and safety is just a code word for censorship; And some economic, from platforms leaders, who have been cutting back their trust and safety teams as cost centers and generally more trouble than they’re worth.
Today’s guest, Jeff Allen, is very much part of this world. Jeff’s a former trust and safety executive at Meta, now the founder of the Integrity Institute, which is both a community for trust and safety professionals and an advocacy group for a kinder gentler social internet. Jeff and I discuss what trust and safety professionals really think about free speech; why Instagram search tends to harm young people and Google’s does not; why Mark Zuckerberg doesn’t like trust and safety, in Zuck’s own words; and where those hoping for an internet that does better at fostering human well-being, might find reason for optimism.
Transcript
Eric
Jeff Allen, welcome to In Reality.
Jeff Allen (Integ Inst) (00:15.012)
Thank you for having me.
Eric (00:16.75)
All right, so let me ask you to ground us for a little bit. Tell us about what the Integrity Institute is, when you started it, how you’re trying to affect change. Just sort of get us rolling here.
Jeff Allen (Integ Inst) (00:31.204)
Absolutely. So the Integrity Institute is part professional community, part think tank. On the professional community side, we’re made up of what we call integrity professionals. These are tech workers who have worked at social media companies or social internet companies on platforms.
And they have worked specifically on integrity teams or trust and safety teams or related problems, right? So these are the people who have fought hate speech and fought foreign influence and fought bad actors and scams and frauds from inside of the social media and social internet companies. Then on the think tank side, we’re trying to collect and harness all of that expertise and experience and connect it to the outside world.
We’re seeing now that there’s just a real need for that people who have that specific kind of experience to engage with external stakeholders of the social internet You know policymakers are passing bills and we need to make sure that they’re actually effective and actually creating change the right kind of change within the companies Academics are doing research, you know journalists are writing about it. So, you know, we’re really just trying to harness all of that experience and expertise and create a place for, you know, the people who have been doing the work to help build out that profession, right?
What does it mean to be an integrity worker? What does it mean to be a trust and safety worker? So we want to make sure that the people who have actually done the work and are doing the work right now are in that conversation and have a place to sort of define that, as well as ensuring that all the external stakeholders are very well supported. I do give you a sense, for example, in my experience, I worked as a data scientist for almost four years at Facebook and Instagram. And I worked on a couple of different teams there. So one was the Pages integrity team.
Eric (02:07.757)
Okay.
Jeff Allen (Integ Inst) (02:19.954)
This was a team that was formed specifically to sort of root out all of the bad actors that had infiltrated the pages product on Facebook. Examples there include like the Internet Research Agency operation from Russia. And then when I was on Instagram, I worked on a team that was worried about harmful and violating content in the recommendation surfaces like the export tab and hashtag pages and all of that.
You know, the founding moment of the Integrity Institute was actually in February of 2021. This was when a handful of us ex-Facebookers got together and said this would be an interesting thing to pursue. And so it’s been a full-time job for me coming up on four years. We launched publicly in October of 2021, so there was something like a nine-month incubation period. And so we just had our three-year anniversary from our public launch.
Eric (03:11.818)
All right. Well, congratulations on that. February 21 was an interesting time in American history, a month after the January 6th insurrection. was also kind of that that was the founding spark for the Alliance for Trust and Media, which is my organization. think it was when we look back on that, that will certainly be an inflection point in history. Who funds the Integrity Institute?
Jeff Allen (Integ Inst) (03:21.553)
Yep.
Jeff Allen (Integ Inst) (03:38.994)
So we are foundation funded primarily, like almost entirely. So some of our big donors include like the Hewlett Foundation, Packard Foundation, MacArthur Foundation, Knight Foundation, Omidyar is up there. I might be forgetting one or two, but some of our principles when it comes to funding is we do want to avoid taking money from the companies themselves. We’re a 501c3 for the public good nonprofit. so…
We you know we do it’s not that we think the companies are inherently evil We actually think that there is a really positive role for the social internet to play in our lives And we think that there are you components of that happening today, and so we’re not against the companies But we do try to try and maintain our independence from them
Eric (04:19.561)
Okay, all right, good. It’s hard to not observe that the trust and safety movement or profession or action goal is kind of on its back foot at the moment. And you can just go through the things that have happened. The teams at the major platforms have been trimmed. Twitter seems to have abandoned all efforts altogether on that. Kids Online Safety Act stalled in the house. The Stanford Internet Observatory shut down.
Donald Trump, who was kicked off the platforms for inciting violence on January 6th, is now the president. So what is the case for trust and safety now? And how do you handle the uphill climb that seems to be ahead?
Jeff Allen (Integ Inst) (05:02.534)
Yeah, I mean, I mean, you’re definitely right. I think something that our community has felt really acutely is the round of layoffs that the companies have been doing. You know, this really started in 2022 and you brought up Twitter, you know, which was becoming X at the time, but, you know, huge layoffs at Twitter and an enormous amount of people laid off at other companies too. you know, it kind of, it was definitely a moment for the Integrity Institute where it was like, there’s actually a real need…
…for people in this profession to have a place where they can gather and meet with each other and discuss about things outside other companies because they’ve all just been laid off. And so one of the real use cases that we saw people coming to us with was, I no longer have people who I can talk about the news of the day and talk about this profession with. And I miss that. And they came to us to find that community again, which was really…
…you know, really a service that I’m proud that we were able to offer. You know, yeah, there’s definitely a lot of challenging headwinds. I think the case for trust and safety work and integrity work still is very compelling today. Even today, right? So I think for the companies themselves, you know, it is really important for the platforms to be safe. Like people want to be on platforms that are safe. People want to be on platforms where they can trust the information that they’re reading.
You know, people want to be on platforms where they’re not constantly exposed to sensational or graphic content, you know, that they don’t want to see. And so, you know, if you are just trying to operate a business and you just want people to come to your platform, you have to consider that, right? We always, one of our kind of talking points, right, is that, you know, if all you want to do is maximize engagement tomorrow…
…you know, maximize the number of people that are using your tomorrow, platform tomorrow, then yeah, there are a lot of negative incentives for the companies, right? Like show people hate speech, show people misinformation, show people tons of graphic content, like that will, that will get them to pay a whole lot of attention to you tomorrow, you know, and maybe next week.
Jeff Allen (Integ Inst) (07:11.622)
But the longer and longer horizon you want to think about the growth of your platform, the success of your company, the more you’re going to need to integrate integrity and trust and safety type thinking into your growth model, right? Because yes, misinformation and sensational content will get people to pay attention tomorrow, but it’s not going to get people to stick around for a year or five years or more, right? And so the more the companies can think about their long-term benefit and their long-term growth, the more they’re going to need trust and safety work and integrity work.
And then there’s also advertisers, right? So it’s you know, something we’ve seen that is like as companies pull back their trust and safety efforts, their integrity efforts, advertisers don’t necessarily want their ads to be running next to sensational content, graphic content, which is entirely fair and entirely reasonable. And so there definitely are like a few like hard checks on how much can they truly pull back from trust and safety and integrity work.
Eric (07:41.415)
Right.
Eric (08:08.229)
Right. It’s quite obvious that some of the platforms with the lowest levels of trust and safety and the most meager efforts at covering that parlor gab rumble are really not very large and seem to have stalled in their growth. Let’s turn to politics. The impression that one gets is that Jim Jordan and his supporters have won in the sense that they’ve at least succeeded in making trust and safety suspect in the eyes of many Americans. Jordan, just for the sake of ideas, the Republican congressman from Ohio, the chairman of the select committee on the weaponization of the federal government. And it focuses a lot on supposed censorship of pro-Trump or anti-vaccine views. He’s popularized this phrase, the censorship industrial complex, which presumably includes organizations like the Integrity Institute.
Does, just to raise the question of free speech, does that actually demand disabling or cutting back on trust and safety or modifying the goals of trust and safety?
Jeff Allen (Integ Inst) (09:20.274)
Yeah, not at all, actually. So to give you a sense of where our members are, we did a survey last year around, we called it one of our policy surveys, because we saw that regulators around the world were implementing new policies. And so we asked our members, the idea was to get at what kind of policies do they want to see. But we also asked the question of what problems do you think it’s really important to address on the social internet right now?
And like, yes, some of the top problems were like child safety, right? It’s like, you know, and basically what came out of that survey was, you know, people doing the work on the inside or have a lot of the same concerns that the general public does, right? Like the outside world does have. And so like child safety was up pretty high, know, threats to democracy was up pretty high, but also free speech was very high, right? I think that was like the third or fourth most common answer for like what is a concern that you would like to see policy.
Eric (10:15.486)
Hmm.
Jeff Allen (Integ Inst) (10:19.61)
address. You know, it’s you don’t get into the social media industry if you don’t care very deeply about free speech, if you don’t care very deeply about empowering people to share their lives, share their stories, you know, communicate with the rest of the world. It’s really important to me when I was there. I took it very seriously. I was like, you know, Facebook and Instagram are amazing channels for people to, you know, share their lives and maybe even sometimes make a business out of it, And like, you know, conduct their livelihood on these platforms, right? And that’s a really important thing to take seriously and empower. And, you know, to a large extent, platforms are governing a lot of the attention and traffic, you know, on the internet. And that’s something you take really seriously, right? It’s like, is our platform creating the right incentives? Are we doing the right things? Are we actually empowering people that need to be empowered?
Eric (10:57.833)
Mm.
Jeff Allen (Integ Inst) (11:15.844)
So it’s definitely something internally that we care very, very deeply about, right? And Twitter used to have the phrase, we’re the free speech wing of the free speech party. And many of our members are Twitter employees from that era, when Twitter was the free speech wing of the free speech party. So it’s something that we take really seriously. It’s like you don’t get into technologies that empower communication if you don’t want to empower people to speak to the world and share their stories.
Eric (11:25.395)
Mm-hmm.
Eric (11:44.489)
Okay, all right, all right, good. There’s one other headwind that I feel obliged to bring up. It’s particular to the Integrity Institute. There was a Wired magazine story that identified friction within your organization over the leadership direction and also the political division that a lot of organizations are experiencing over the war in Gaza. How have you handled that at the Institute?
And how’s it affected your ability to have an impact that you want to have on trust and safety?
Jeff Allen (Integ Inst) (12:18.512)
Yeah, so we definitely did have some struggles in the spring with that. think a lot of organizations were having struggles at that time. And at the Integrity Institute, we were very susceptible to it in a certain way in that we’re a very global community. So we have community members from all sorts of different countries and backgrounds living in all sorts of different countries around the world.
And to a certain extent, it was almost maybe inevitable that there would be strife within our communities as geopolitics became part of it. I think it’s also really true that integrity work and trust and safety work demands confrontation of this. When you’re an integrity worker and a trust and safety worker inside of these companies, they were the ones figuring out, are our graphic violence policies actually
Eric (12:55.837)
Mm-hmm.
Jeff Allen (Integ Inst) (13:15.084)
know, properly tuned to address this current crisis. And that means becoming enmeshed in that current crisis, right? Studying it, looking at all the content, you know, figuring out, you know, what are the values of the company and how do we map that onto decisions that we’re making around this kind of content? And so these situations are super acute for our community, right? There’s things that they grapple with, right? And, you know, just like a global community will have differing points of view.
You know, these are large tech companies with tens of thousands of employees. so internally, there’s strife there. And there’s the values of the company, right? And the company leadership. And that doesn’t always align with the workers at the companies. And so there’s tension there. And so to a certain extent, it’s not at all surprising, right? It’s almost inevitable that, like, you know, people who are having to confront this at work are looking for other spaces to talk about this.
Eric (14:05.609)
Mm-hmm.
Jeff Allen (Integ Inst) (14:06.962)
And I think we had a little bit of challenge with that in the beginning. I think we’ve come a long way, to be perfectly frank. We’ve changed our policies. We’ve made it so that it’s easier to create spaces to have conflict or to have conversations that might lead to tension, might lead to conflicts. We’ve had a lot of those conversations inside of the community. We’ve done work around it.
And so, you know, I do think it was sort of a moment for like, yes, you know, the integrity suit has undergone very, very rapid growth, right? So when we first launched, there were something like 15 or 20 members.
But after the layoffs, we saw that grow by 50 to 100 members in some months, know, in individual months. And so I think sort of like at the beginning of this year, we found ourselves with like 400 members and we’re like, you know, we very rapidly discovered like, things are very different when you’re just, you know, 30 people who kind of have all kind of maybe know each other in real life, you know, to.
Eric (15:00.019)
Mm-hmm.
Jeff Allen (Integ Inst) (15:04.806)
suddenly we have like 500 people in this community and we’re having to catch up to a certain extent and build some of that infrastructure that we needed to have in place to make sure that people can have those conversations. But I think in terms of us like preventing or making it challenging to do the work, I mean, I think a lot of that is the work, right? I think a lot of that is, you know, as integrity work and trust and safety work matures as a profession, we’re going to have to figure out how to deal with this, right?
Eric (15:31.657)
Mm-hmm.
Jeff Allen (Integ Inst) (15:33.102)
You know, there’s a lot of extra values that need to be brought to the work of building the social internet and bringing the platform and building platforms. You know, it’s important to value free speech. It’s also important to value keeping people safety. It’s important to value democracy, you know, when you’re building these platforms. And so I think that it’s really important for the profession profession as a whole to figure out, OK, like we are a values, ethics, morals based profession. And so we need to make sure that when individuals bring their values, ethics, and morals, that has an opportunity to be properly reflected in the work.
Eric (16:08.027)
Okay, good. I would say, Jeff, that many organizations, many companies need to handle this kind of political division. is impossible for that, which is so evident in the politics in the U.S., at least, not to seep into the Slack channels at work. And companies can do it better.
Jeff Allen (Integ Inst) (16:28.869)
But I mean, to a certain extent, it’s not necessarily just seeping into it. It’s almost inevitable. And there’s a really great article, and I’m blanking on the author of it. So I don’t know if I can. Well, I’m blanking on the author, but it’s being black while moderating. And this weird dynamic, there’s this really weird tension where it’s like, well, who do you want evaluating whether or not content is hateful or violent?
Eric (16:34.375)
Mm-hmm.
Eric (16:47.719)
Hmm. Hmm.
Eric (16:56.403)
Hmm.
Jeff Allen (Integ Inst) (16:57.306)
you certainly want people who are experts on the ground in these areas, which often means that they are people who are in community with people experiencing violence or experiencing hate speech or being accused of committing hate speech or being accused of committing violence. It’s to a certain extent, the nature of the work almost necessitates bringing in people who are very well versed with tensions around the world, which means that they’ve
probably experience those tensions in their own lives and these are deeply personal issues to them. So it’s like, to a certain extent, you have to have that expertise, you have to have that care and passion about it because it is coming up in the workplace, right? So it’s not necessarily just like a political debate internally, it is, you know, how are we gonna build our platform to ensure that everyone on earth can participate in it.
Eric (17:30.877)
Mm-hmm.
Eric (17:50.791)
Hmm, well, that is, that’s interesting. And God, we could go on forever about this. This is, this is the era we live in. But let’s go back to the work of the Institute. What are the key initiatives you’re working on right now? What are you, where are you most hopeful of making progress?
Jeff Allen (Integ Inst) (17:55.6)
Yeah.
Jeff Allen (Integ Inst) (18:08.486)
Yeah, so there’s generally two big buckets of work that we do at the Integrity Institute. One is on the community side. And that’s of like supporting the profession, building out pathways for professionals to stay engaged in the profession and have that. And the other side is sort of like the research work and think tank work. I think a lot of the work that we do on the community side now.
What are some the needs that are sort of felt in the community? And I think the industry downturn and the big rounds of layoffs is still something that our community is feeling, still something that is occurring, even if it isn’t necessarily getting the big headlines like it was at the beginning of the year and like it was last year. So it’s something that’s still sort of felt acutely by our community. And so a lot of what we do is enabling those professional networking spaces.
Eric (18:51.827)
Mm.
Jeff Allen (Integ Inst) (19:03.25)
You know, I think it’s a real important part of our mission is ensuring that like people who really care deeply about the impact that social the social Internet is having on our societies and having on people We need to make sure that like the people that really care about that are able to still engage in the profession and able to still find opportunities So a lot of what we do is professional networking and creating spaces for that A lot of what we do is also just connecting to outside opportunities.
You know, one really, you know, if there is a silver lining, you know, to the layoffs that trust and safety workers have experienced, it is that they have now gone forth and decided that like, maybe it’s time to take a break from working at a tech company.
Maybe now it’s time to see what else I could do that is in this similar space that could have impact. And so we’ve seen our members become regulators actually and go work for governance. We’ve seen our members go join civil society organizations. To a certain extent, when we first started, we had to build out a lot of.
Eric (19:54.535)
Hmm. Hmm. Hmm.
Jeff Allen (Integ Inst) (20:05.542)
connections to the outside world, right, to civil society organizations and to regulators. And so it was almost a service that we offer to our members of like, hey, like we have this Rolodex that you can use, you know, if you find you have need of it. And we’ve kind of seen over the past year that now actually the Rolodex is our community and our members that have gone on to do, you know, really exciting things in civil society, you know, taking a break from tech for a little while.
Eric (20:16.701)
Mm-hmm.
Jeff Allen (Integ Inst) (20:29.714)
And so making sure that people are aware of those opportunities, they’re being connected to them, they’re being supported in them. That’s definitely a lot of the work that we do on the community side. On the think tank side, on the research side, we are still absolutely in a huge policy moment around the world, really. And yeah, maybe things in the US at the federal level, maybe it’s not looking particularly optimistic that something good will be passed.
But around the world, that is not the case at all, right? And so we’ve had the Digital Services Act be passed in the EU. We’ve had the Online Safety Act be passed in the UK. Both of these are really big, very potentially very impactful bills. And it’s a really critical moment in both of them right now because the regulators have already passed the bills. So they’re in place, but there are just an enormous amount of implementation details to figure out.
Right? The implementation details are really where a lot of the impact is going to be coming from regulation. And so we’re doing a lot of work on the research side to help regulators get that right.
Eric (21:38.349)
Jeff, can I ask you to dig in a little bit deeper on the Digital Services Act, which I believe now is in its second year, but as you noted, there is the devil’s in the details of implementation. Stepping back a bit about the broad sweep of the regulation, do you like it? Do you think it’s wise? And leaving aside the as yet unknown answer to the question about implementation, do think it has the potential to change the…
Jeff Allen (Integ Inst) (21:42.639)
Absolutely.
Eric (22:07.827)
change behavior in the beginning.
Jeff Allen (Integ Inst) (22:10.426)
Yeah, think it does. The Digital Services Act, I think, is a pretty solid framework to give you a sense of actually where our members are. So we did do this policy survey among our members. And we asked, would you like to see more regulation of the social internet? And 85 % of our members said yes. So more regulation is actually very, very popular for professionals and practitioners who have actually done the work on the inside.
We would like to see it. When asked, do you think it’s possible for policy to have a positive impact on creating the right incentives for the companies? The answer was actually 90. I forget the exact number. It was like 90 to 95 percent. Yes, right? So our members really do think that there is an opportunity for policy to have positive impact. And when we ask, here’s a list of policy proposals, you know, rank them according to whether you think they’d have a you know, good, good effect, good impact on empowering people, empowering integrity professionals and empowering empowering trust and safety workers on the inside. So basically we’re like, you know, to what extent do you think this policy proposal would help people who are, you know, trying to fight the good fight from inside of the companies? and at the top of that list was, you know, increased transparency about like the scale and cause of harms online, risk assessments and risk mitigation plans, researcher access and audits.
So at the top of that list was a lot of the things that you see in the DSA. And the overall policy frameworks that are pretty popular for the practitioners are, you’ll see them skew very heavily towards the transparency component of it, towards the risk assessments component of it and auditor and researcher access. This is because we think that if the outside world could…
…see the dashboards that integrity teams are using internally to study risk, and could see the dashboards that the trust and safety teams are using to track the harms and track the risks. We think if we can get more and more of that information transparently out to the public, that is going to dramatically change how the companies are making decisions internally. It’s going to dramatically change how they’re resourcing and staffing teams and how they’re weighing considerations.
It’s not uncommon for safety and growth of the platform to be in tension with each other. We sort of mentioned the, if you want to grow really fast next week, then yeah, reduce the safety of the platform. And so there definitely can be tensions that come up there. And so we do think that increased transparency can help tilt the scales in favor of the people who are sort of pushing for more safety.
Eric (24:35.741)
Yes, right.
Eric (24:42.139)
Mm-hmm.
Eric (24:52.133)
Okay, good. The DSA leans heavily into transparency. Let me focus a bit on solutions. You wrote a very interesting article contrasting your experience on Instagram and its content moderation policies, or I guess was actually around search and the kinds of stories that were served up on Instagram searches related to eating disorders and contrasted that with the
Jeff Allen (Integ Inst) (25:12.912)
Mm-hmm.
Eric (25:21.929)
links that were served up on Google for the same searches. And it was remarkable to me as an outsider to see how different the results were, how much more positive and helpful the results were on Google. Could you explain what happened there, why it was so much worse on Instagram, and what does that say about a way forward?
Jeff Allen (Integ Inst) (25:43.834)
Yeah. So to give the context on that article, wrote it. So obviously I worked on teams that were related to search when I was at Instagram. I wrote that article after Instagram announced that they were changing their policies in January of this year, saying that they were going to stop enabling any searches related to eating disorders, any searches related to self-injury. And so they were, you know, I…
…I don’t know if this is entirely fair, they’re sort of saying, we don’t know how to do these searches correctly, so we’re just going to block them entirely. And I know from my time at Facebook and Instagram, and I deeply believe this, is that like,
There are a lot of great organizations that are trying to help young people and trying to help people who have mental disorders on Facebook, on Instagram. It’s not like it’s impossible for Instagram to help someone who has an eating disorder. It’s not that it’s impossible for Instagram to be a platform that is helpful to someone who has an eating disorder. One of the things, kind of anecdotes I had in my mind when I was writing that was like, yeah, like why?
Why do we live in a world where if a friend came to me and said, you know, hey, Jeff, I want to talk to you. Turns out I have an eating disorder. You know, I’m using Instagram to help like learn how to manage it. I would be like, slap the phone out of their hands. I’d be like, excuse me. No. But it doesn’t have to be that way. You know, if a friend told me like, I was, you know, doing some Google searches to figure out, you know, some steps I could take to manage, you know, an eating disorder.
Eric (27:15.081)
Yeah.
Jeff Allen (Integ Inst) (27:28.658)
I’m like, OK, that’s not crazy. Hopefully you’re also talking to a doctor, medical professional. But OK, I’m not going to slap your phone out of your hand if you tell me you’re using Google search to learn about some steps you can take to manage an eating disorder. But I would do it if you told me that you’re doing it on Instagram. And it’s like, why is that? It doesn’t have to be that way, right? And so a lot of what I dug into was a little bit on the more technical side, but just how is Google search designed?
Eric (27:43.571)
Mm-hmm.
Jeff Allen (Integ Inst) (27:58.162)
How is Instagram search designed? And so there’s the technical aspects of that, of how is it that Google search decides to put this link at the top of their search results? How is it that Instagram decides to put this photo at the top of their search results? What are those systems trying to optimize for? And then what are the company philosophies that are behind that?
And so on the technical aspect of how do these search engines work, how does Google decide what to put at the top, does Instagram decide what to put at the top? Google Search is much more of a quality-focused ranking system, whereas Instagram is much more of an engagement-based ranking system. And Mark Zuckerberg himself actually put out one of my favorite arguments for why engagement-based ranking systems are inherently more risky. In 2018, he actually published publicly.
It was his announcement of the oversight board, the meta oversight board. And in it, he was talking about what he called the natural engagement pattern. And he had a chart in that report, in that article he wrote that said, you know, as content gets closer and closer to being policy violating at Facebook, it tends to get more and more engagement, right?
Eric (28:51.71)
Mm-hmm.
Jeff Allen (Integ Inst) (29:17.542)
And so we had this chart that was a little bit of like a hockey stick kind of chart, right? Where it’s like, as you get closer and closer to policy validity, which means as you get more and more harmful, content will get more and more engagement. Well, the fundamental problem with engagement-based ranking systems is if you are predicting if a user will engage with a piece of content or not and ranking the content accordingly. Well, if those predictions of engagement are doing their job, then predicted engagement should follow true engagement, right?
Eric (29:29.032)
Hmm.
Jeff Allen (Integ Inst) (29:47.602)
And that means that as content gets more more harmful, closer and closer to policy violating, you would expect the predicted engagement scores to get higher and higher. This means fundamentally that engagement-based ranking systems, especially around sensitive searches or sensitive content topics, like eating disorders, like self-injury…
…the bad stuff is going to float to the top. It’s kind of inherent in how those systems are designed, as opposed to Google Search. And here I kind of get into what is the philosophy of Google Search, and what are the missions of the companies, and how do those missions get mapped into how they build the platform. Google’s mission is actually to organize the world’s information and make it widely available and useful.
Eric (30:15.08)
Yeah.
Eric (30:30.566)
Mm-hmm.
Jeff Allen (Integ Inst) (30:39.242)
And I always like to say the most important word in that mission statement is useful. Misinformation is not useful. Content that advocates for eating disorders or self-injury is not useful to people. And so Google has actually done a ton of work to figure out what does useful actually mean. And they have what they call are there search quality guidelines.
And this is basically just like a 80 page document that is like a technical implementation of like how do you assess how useful content is or is not on the internet? You know, and so they have things like content from experts in a particular domain or content from people who have who have Accredentials in a particular field. It’s just inherently more useful than content in that field with for people who don’t have credentials, right?
Eric (31:29.577)
Mm-hmm.
Jeff Allen (Integ Inst) (31:29.586)
And so they have all these systems that are predicting is this piece of content high quality or is this piece of content low quality? As opposed to Instagram where you know all those sort of ranking and recommendation systems are geared towards you know, will you like this post? Will you look at this post? Will you watch this video? Will you comment on it? And we now know and we’ve known since 2018 when Mark Zuckerberg made it public that like those kinds of systems risky harmful content is going to float to the top.
Eric (31:53.885)
Yeah, That is very useful to borrow Google’s phrase distinction. Thank you. You know, the logical leap from there is to follow the advice that I believe is often attributed to Francis Hogg and the Facebook whistleblower is that, well, if all you did was change the algorithmic goal from engagement to chronology, you would solve a lot of the issues at Facebook and Instagram. Is that, it seems so simple. Why is that not really the solution? Or at least why haven’t people adopted it?
Jeff Allen (Integ Inst) (32:35.79)
Yeah, I think like…
Jeff Allen (Integ Inst) (32:40.594)
It is like a little bit of an oversimplification, right? And I think that, you know, really what we need isn’t necessarily like the one true ranking, right? It’s not like there is like this one thing that will fix like all of the ranking problems on every platform, right? What we really need to do is for platforms to like, A, have good values and have a good mission statement, right? And then B, figure out and…
Be sort of accept that how you write content and how you recommend content, that is always an editorial decision. That is always a reflection of values. A lot of these platforms right now tend to try to be making the case of, we don’t have values when it comes to ranking content. We don’t have an editorial stance when it comes to ranking content. And I think one thing we need to get away from is that is just a fallacy.
Eric (33:15.162)
Mm-hmm.
Jeff Allen (Integ Inst) (33:33.722)
Ranking things according to chronology is an editorial statement, is an editorial stance. Ranking content according to whether people will hit the like button or not, that is an editorial stance that has moral and value-based implications to it. So part of it is kind of getting away from this idea that there is such a thing as editorial-free content ranking and recommendation.
And really just appreciating that like no any way of ranking is a reflection of editorial values And you need to take that seriously right and ensure that your ranking system is a reflection of your values and a reflection of your mission, right? You know Google’s mission is to make the world’s content use for world’s information useful And so they figured out okay, like what does that mission mean for how we build a ranking system? Facebook meta, unfortunately, they changed their mission statement very recently, but…
…to one I don’t necessarily love as much, but when I was at Meta, when I was at Facebook, their mission statement was to empower people to build community and bring the world closer together. And I love that mission statement. And it was a statement I loved when I was at a company that had a mission like that. And I always said, I wish we could figure out how to encode that mission statement into our ranking and recommendation systems, because that is a much, much better, those are much better values to guide your editorial decisions than ‘we’ll just show people content that they want to see’. It’s not our problem.
Eric (34:57.331)
Yeah, yeah, yeah. Thinking of other kind of potential simple solutions that have been proposed, one of them would be social media platforms that have different mission statements that are more pro-social. Blue Sky, for example, is often held up as a less toxic version of Twitter. Are those realistic goals?
Jeff Allen (Integ Inst) (35:23.174)
think so, yeah. mean, you know, our vision at the Integrity Institute is a social internet where people, societies, and democracies can thrive. And we believe in that, right? We really do think that, like, there is a positive role for these platforms to be playing in our lives and in our communities and societies. They’re not living up to their potential today.
But it does exist. You know, we’ve seen glimmers of hope, you know, with like the Arab Spring and democracy uprisings. There’s tons of inspirational stories I could tell you of like, you know, creators that have like, you know, had positive impact on people’s lives and wouldn’t have been able to, you know, without the, without online platforms. So it’s absolutely there. We absolutely do believe into it. I mean, there’s definitely like a lot of work that we need to do before we reach this, this sort of, you know, vision, this destination.
Eric (36:05.672)
Mm-hmm. Mm-hmm.
Jeff Allen (Integ Inst) (36:17.874)
But, you know, I don’t think that, you know, it is, it is, we should just accept that it is inherent that social media will always be having, you know, negative impacts on people and negative impacts on societies.
Eric (36:33.755)
All right, good. We’re a month away from the supposed shutdown or sell off of TikTok. Are you in favor of that or against it?
Jeff Allen (Integ Inst) (36:44.914)
I consider that to kind of be a decision about my pay grade. don’t know. The geopolitical question, I think it’s a tricky one. I think we have seen cases like this before. Actually, a little known fact is that Grindr was originally a Chinese-owned company, the dating app that of skews towards LGBT…
…was originally a Chinese company or a company based in China and the government forced us to sell off and so it’s now American based. So it’s like, you know, was that the right thing to do? Again, I kind of considered it above my pay grade, right? It’s like, how does the US and China want to relate to each other in terms of companies and in terms of trade? It’s a tricky one. think like there’s only things to be concerned about it, right?
Eric (37:07.625)
Mm-hmm.
Jeff Allen (Integ Inst) (37:34.91)
It’s definitely not something we want to become any sort of norm, right? That like constantly forcing companies to sell themselves and stuff like that. But like there are valid situations where it’s something to consider.
Eric (37:48.773)
Okay, so it’s not a trust and safety issue. It’s a geopolitics and big power rivalry.
Jeff Allen (Integ Inst) (37:53.042)
It’s kind of a geopolitical, you know? You know? Yeah, yeah.
Eric (37:57.833)
We haven’t even discussed artificial intelligence. So right now, my impression is that the main effect it’s having is that it’s flooding the internet with junk copy and junk images and so forth. That doesn’t seem to be the trust and safety and misinformation cataclysm that was predicted. Do you agree with that assessment of AI to date? And AI is still in its infancy. What worries you most about its future?
Jeff Allen (Integ Inst) (38:27.376)
Yeah, think like, you so heading into this 2024 year of elections, right? I think there’s a lot of concern of like, are we going to see sort of a kind of like a 2016 style, you know, where in 2016, we kind of saw the weaponization of social media, you know, where a foreign power, you know, was able to reach large audiences in the US. I think we’re also similarly scared about like AI and like, is this going to be like the weaponization of AI?
and either local or geopolitics. I think we saw hints that it could be possible, but I think we didn’t sort of see an explosive incendiary event around it. And I think that there’s definitely things where you’re like, OK, cool, we could be heading in this direction. We should definitely still be vigilant and pay attention to it. There was the robo call of Biden, whereas impersonating Biden that was calling people and telling them to stay home and not vote.
You know, there were instances of campaigns actually hooking up chat GPT to texting services and so voters were having conversations about who they should vote for with You know chat GPT, know governed by a campaign that was running it right like that happened We definitely saw things foreign, know, like internationally like in foreign elections, you know, there was you know, faked faked audio That was leaked that may have had an impact
Jeff Allen (Integ Inst) (39:55.154)
We know that the bad actors are using it. So there are reports that yes, Russia and their disinformation operations are utilizing AI. But I think right now it looks like it’s a 10 % or 20 % increase in their operations. It makes them 10 % or 20 % more effective, as opposed to 2x or 3x or 5x more effective, which is what we are worried about. So we haven’t quite seen that explosive moment yet. I still think they’re still…
…very much a good reason to be vigilant about it. A, we don’t fully know everything that happened this year. And then B, I think we’ve seen enough of the early signs of it, of like, yes, this could become a problem in the future. I know I was just looking at the most recent Facebook widely viewed content report. This is a report they put out every quarter of what are the top 20 posts on the platform that were most viewed.
Eric (40:51.571)
Mm-hmm.
Jeff Allen (Integ Inst) (40:53.254)
And it definitely looked like at least two of the posts out of the top 20 were AI generated. So some of the most widely viewed content on Facebook is now AI generated content. And so it absolutely is sort of having an impact on the ecosystem.
Eric (41:08.457)
Okay, well that sounds a bit ominous. But let me end with a question that I often ask people. In light of everything that you’ve seen and your background, are you optimistic or pessimistic about the mission of the Integrity Institute, the ability of humans to steer or create a social internet that will help individual societies and democracies thrive?
Jeff Allen (Integ Inst) (41:39.142)
I think, I mean, certainly there’s some headwinds and some real risks that are coming and things to be concerned about, but overall I’m optimistic. I think when you look around the world, think, you know, societies are taking these problems much more seriously. Regulators are taking action. You know, I like to tell…
…even in the US, there’s lots of reasons to be optimistic about policy in the US, and regulators are much more educated than they have been in the past. I remember I was at Facebook when Mark Zuckerberg was at the hearing when the senator or congressperson asked, how do you even make money? And Mark Zuckerberg was like, Senator, we sell ads. guess it was the Senate hearing. Senator, we sell ads. And I remember all the memes that were made inside of the company around Senator, we sell ads. Even in the US, we’ve come a long, long way. And if you watch the hearings now, the question
Eric (42:16.146)
Right.
Jeff Allen (Integ Inst) (42:28.164)
are way, way better, you know, that Congress is asking of the leaders. So I think in terms of like externally, there’s plenty of reason to be optimistic. And I think, you know, for the profession, right, it’s really exciting to be in this moment at the Integrity Institute watching the trust and safety profession and integrity profession sort of…
…become a real thing. We have conferences dedicated to trust and safety that we go to. And it is amazing, right? And there’s members everywhere running around from the Integrity Institute and coordinating together to have panels and have sessions at them. You can now get a degree in trust and safety. So Cornell Tech actually has a master’s program in trust and safety. So you can get your master’s in trust and safety now, which is really genuinely wild to me.
But it sort of says, yeah, what is the curriculum for trust and safety? What are the things that we need to teach people, ensure that they understand and know when they go off to build the social internet? I think that’s super exciting. We now have a CEO of a tech company who has a background in trust and safety. So OnlyFans, the new CEO at OnlyFans actually came from their trust and safety, the trust and safety part of their organization. So we have trust and safety professionals as CEOs of social internet platforms.
So I think like it is an exciting moment on the professional side of things and the regulatory side as well.
Eric (43:52.349)
Well, that is great. Jeff, thank you. That’s a good place to leave it as I race off to open my OnlyFans account. I thank you for your time and thank you for the work that you do on behalf of making the internet a safer place. We really appreciate it.
Jeff Allen (Integ Inst) (44:10.492)
Thank you.
Created & produced by: Podcast Partners / Published: Dec 19 2024
All episodes are streaming on these platforms:
