Background

When users on social media platforms open their app for an engaging and interactive experience, on-platform games have always been an important engagement lever for time-spent and social activities for Meta’s platforms. As Facebook and Reality Labs products have evolved to focus more on mobile experiences, games needed to follow the device usage.

Facebook Games needed to evaluate what the top most performant, fun, and engaging games were in their mobile game catalogue to connect users quickly with fun outside of their friends’ posts and photos. While on the Facebook Games team, I co-led the development and execution of a playtest framework to identify and rank the best games on the platform.

The problem space at hand required an in-depth, scalable approach that identified not only the games that were most fun and engaging for users, but why they were the most played games on the platform in order to showcase top games and identify future investments and guidance for on-platform developers.

What we did

In order to give at-a-glance snapshots of each games’ highlights and lowlights as well as identify themes, we put together a framework of industry-standard playtest metrics and metrics tailored to Facebook’s platform growth and engagement strategies and short survey of questions aimed at why players had (or didn’t have) fun with each game they tested.

Once we had feedback from discovery, developer engagement, and product management teams working in the ecosystem on the metrics and games to test, we launched an unmoderated playtest designed to take 20 minutes of participants’ time to accurately reflect play time on platform and answer questions about their experience. Each game was evaluated by at least 20 Facebook Games users, and games were ranked against the average performance in a dashboard designed to update as more games were evaluated and show the most up-to-date ranking of top performing games on the platform with each round of UXR and DS input.

This allowed us to provide ranking and relevance teams on discovery surfaces with the games qualitatively received well by playtest participants to compare with data-science metrics around time spent in each game as well as structured documentation for Facebook Games developers to help them identify key game mechanics and themes that would help them develop new games for the platform with confidence in their potential to succeed.

The Outcome

The dashboard combining UXR qualitative ranking and feedback and DS collected metrics informed multiple workstreams, from existing game ranking and discovery on the Facebook Games tab in the mobile app, to investment strategy and developer guidance for engaged developers on the platform seeking to make new games, and was presented to VP-level leadership for steer on scaling the program into a recurring practice each business half.

When I moved to the Horizon team, mobile access to Horizon Worlds had just launched, and the framework used on Facebook Games was adapted to follow the same industry standard playtest metrics and adapt to Horizon’s strategic needs. Following the same structure and cadence, Horizon org leaders utilized the data from these playtests to inform creator competition themes to seed more games on the platform native to mobile and curate discovery surfaces like the Horizon Feed with trending collections, top games, and top picks from Horizon in the first few shelves on the Feed.

Both programs continue to run as more games are created and distributed, with strategic shifts reflected in the metrics gathered while meeting industry standards for game evaluations.