
At Quora, we call it “Be Nice, Be Respectful,” or BNBR. It's a code of conduct for using the platform, and most UGC networks and communities have something like it to protect themselves and their users. But how do we enforce these policies amidst increasing threats from spam, harassment, hoaxes, sockpuppets and trolls?
While human reporting and moderation are an essential component of combating bad actors, with global online networks and communities reaching massive scale, we need machine learning to do the heavy lifting. Join us for this invite-only workshop to hear from researchers and practitioners applying the latest ML thinking and technology to protect our shared online spaces.
Aaron Halfaker, PhD, principal research scientist, Wikimedia Foundation
Engineering at the Intersection of Productive Efficiency, Ideology, and Ethical AI in Wikipedia
Wikipedia has become a dominant source of reference information for more than half a billion people every month. Through its improbable rise to popularity, this "free encyclopedia that anyone can edit" has become a synonym for open production communities online. In order to operate at a massive scale (~160k edits per day), Wikipedians have embraced algorithmic technologies that bring efficiency and consistency to the wiki's complex, distributed processes. These algorithms mediate social processes, governance decisions, and editors' perceptions of each other. Specifically, so-called "black box" AI's have proven invaluable for supporting curation activities at scale, but they also have the potential to silence voices and perpetuate biases in insidious ways. Despite Wikipedians' open/auditable processes, that's exactly what's been happening. In this talk, Dr. Halfaker will introduce "ORES," an open AI platform designed to enable Wikipedia's technologists to enact alternative ideological visions and to enable researchers to easily perform audits. He'll share lessons learned from maintaining a large-scale, generalized AI service and discuss a call to action for critical algorithms researchers to take advantage of this platform for their studies.
Dr. Aaron Halfaker is a principal research scientist at the Wikimedia Foundation and a senior scientist at the University of Minnesota. He studies the intersection of advanced algorithmic technologies and social issues in open production communities (like Wikipedia) using a mixture of experimental engineering, data science, and ethnographic methods. His studies of Wikipedia's editor decline and his development of "ORES," an open AI platform, have received substantial attention from the technology press.
Michael McNally, director of engineering, Facebook
Machine Learning Challenges in News Feed Integrity
Facebook has three of the world's largest social network products: Facebook (the “big blue app”) itself, Instagram, and WhatsApp. For each of these, providing "Integrity" — that is enforcing community standards, preventing spam, abuse and fraud, and promoting civil and positive interactions — is a critical aspect of the business. Michael McNally leads News Feed Integrity, which is the newest of several core Integrity teams at Facebook. He will cover how Facebook uses machine learning to address content quality problems, malicious user behavior problems, and the hot-button topics of misinformation and false news. Keys to effectiveness are: obtaining maximal utility from human-provided labels, making effective use of ML algorithms and infrastructure, and adapting our work as ML practitioners to the evolving, adversarial, unbalanced and non-stationary aspects of anti-abuse problems.
Michael McNally is a director of engineering at Facebook, supporting the News Feed Integrity engineering team. Together with partner Integrity teams across Facebook they work to protect online communities from misinformation (“false news”), clickbait, sensationalism, and various forms of commercial spam, hate, abuse and deception. Before Facebook, McNally worked at Google for 13 years, beginning in 2004 with his starter project being the first application of machine learning to Gmail spam filtration. For the second half of his Google career, he led the Ad Traffic Quality and Publisher Quality Teams — protecting the AdWords, AdSense, YouTube, AdMob and DoubleClick ecosystems from ad fraud and invalid traffic, and enforcing content and behavioral community standards. Before that, McNally developed computer games and software for the movie and television industries. He studied artificial intelligence at the UCLA computer science department.
Srijan Kumar, PhD, computer science researcher, Stanford University
Sockpuppets and Hoaxes on the Web
Web platforms enable people to share information and opinions with others. However, some users deceive others by manipulating discussions using multiple accounts called sockpuppets and creating false information. In this talk, Dr. Kumar will present his research on characterizing and detecting sockpuppets in online discussions and hoaxes in Wikipedia. In the first part, we will look at a large scale study of sockpuppetry across nine discussion communities. We study the benign and malicious use of sockpuppets, and develop a taxonomy of deceptive behavior. We create a prediction model to identify whether two accounts belong to the same underlying user. In the second part of the talk, we will discuss the impact, characteristics, and detection of hoax articles in Wikipedia. We will look at what makes a hoax successful and whether humans are effective in detecting them.
Dr. Srijan Kumar is a postdoctoral researcher in the computer science department at Stanford University, working on applied machine learning and social network analysis. He has received several awards including UMD University and Dean fellowships, WorldQuant PhD scholarship, IIT Kharagpur gold medal, and WWW 2017 best paper award honorable mention. He received his Ph.D. in computer science from University of Maryland, College Park and his bachelor's degree in computer science from IIT Kharagpur.
Alana Glassco, anti-abuse engineer, Smyte
Machine Learning and Content Moderation: Common Pitfalls and How to Avoid Them
Online communities are usually governed by a set of policies prohibiting everything from spam to violence to hate speech. These policies are designed to be easily understood by users, and tend to group together concepts that seem similar to a human. However, these policies often do not distinguish between behaviors that are characterized by very different features and would be best represented by separate classes in a machine learning model. We’ll explore strategies for gathering training data in this context, as well as tradeoffs to consider when tuning a model. These include business decisions that need to be made around values such as speed vs accuracy in content moderation, supporting free speech vs maintaining brand safety, and cost.
Alana Glassco graduated from Stanford University in 2011 with a bachelor’s and master’s degree in psychology. She spent the next 5 years working at Google, first on the anti-spam team for Google Maps, and later leading a team of analysts dealing with policy and content moderation at YouTube. Her last project at YouTube before moving to Smyte in May 2017 involved redesigning the manual and automated aspects of YouTube’s entire comment moderation system.
Nikhil Dandekar, senior engineering manager, Quora & Paula Griffin, PhD, product manager for content quality, Quora
Maintaining High Quality User-Generated Content Through Machine Learning
Quora's mission is to share and grow the world’s knowledge. On Quora, people ask questions on a wide range of topics and Quora surfaces those questions to people with relevant credentials and experiences so they can respond with an insightful, helpful answer. Over time, this has led to Quora having one of the largest databases of questions, answers, topics, comments etc. in the world. Any user-generated content platform that works at this scale has to work hard to keep quality from declining and doing so is critical for Quora's success. In this talk, we will discuss how Quora uses machine learning and NLP systems to maintain high content quality. Some of the applications we will cover include classifying and removing spam and harassing content, correcting questions with bad spelling and grammar and identifying duplicate questions. We'll explain how we combine machine learning systems with human reviewers to ensure the Quora you see only consists of high quality content. Finally, we will highlight some lessons learned from applying state-of-the-art machine learning techniques to consumer products at scale.
Nikhil Dandekar is a senior engineering manager at Quora, where he leads the natural language processing team. He joined Quora from Foursquare, where he led the search ranking team. Prior to that, he worked as an engineer and manager on several teams working on web search (Bing) at Microsoft. His expertise is in applying machine learning to internet-scale products.
Dr. Paula Griffin is a product manager at Quora focused on content quality efforts. Prior to her current role, she worked as a data scientist at Quora after completing her PhD in biostatistics at Boston University. Her current work area includes concerns related to moderation, user safety, and overall quality of content on the platform.
Long term parking around downtown Mountain View can be a challenge so if possible, please consider carpooling, public transportation or ride sharing options.
The areas highlighted in orange are places you can park all day, without 2-3 hour restrictions on at least one side of the street. Please double-check the parking signs before parking.
There is also a parking garage that is open to the public in the horseshoe driveway to the right of the parking garage entrance on Church Street (orange square on the map). Availability can be limited during daytime hours.