Covered in this article:
- What is Cortex?
- How is Cortex different than other podcast metrics platforms?
- What metrics are available in Cortex?
- What should I do if I need help interpreting a metric?
- Where should I leave my feedback?
When we set out to redesign our stats page, we realized our team had to go a step further and build a robust platform that uses mature state digital media data practices and modern data science technologies to sort out non-human traffic, measure all listens across the web and mobile for potential sponsors, and deliver actionable editorial insights for podcasters rather than an endless dump of data. Cortex is that solution. See it on your Dashboard, Episodes page, and Stats page
How is Cortex different than other podcast metrics platforms?
The first thing we noticed when we embarked on this project was the noisey measurement landscape in podcasting. Digital traffic contains “organic” (human driven) sources as well as algorithmic or “bot” driven traffic (most of this is benign web crawlers). These non-human sources inflate numbers if mature classification systems are not used. We’ve always engaged in filtering but we realized we needed to approach it differently for the good of the ecosystem.
Second, we noticed the increasing demands of sponsors. When sponsors buy ads on video and web articles, a high level of accuracy and transparency is the norm. They expect this from podcasting now. Many companies are banding together in the industry to make DIY measurement methods obsolete and advertisers are doing business with sophisticated platforms like ours.
The third challenge was one of design - most data platforms give podcasters an endless dump of data that is not actionable. Many data points nice to know but provide little guidance about which editorial direction a podcaster should take within a realm that they can influence. Some platforms collect data via low-confidence methods like human surveys (prone to sampling error and bias in survey design). Sometimes less is more and you’ll notice areas where we use this design method to help you make decisions with data.
Lastly, we had to build a robust system that scales. A solid traffic measurement system must not only be able to handle the rapidly growing datasets at massive scale, it also has to be smart enough to adapt to a bot landscape that is constantly evolving, oftentimes on a daily basis.
When we embarked on this project, we surveyed the top measurement players in the podcasting space, hoping to find a plug and play solution to solve the measurement part of the equation. However, comparing the prevailing practices to the practices of mature state digital media outside of podcasting, it was clear to us that the gulf between prevailing systems in podcasting and where things had to be was simply too large. This is why we had to set up our own data science and infrastructure team to tackle these problem ourselves back in 2014
Cortex is the culmination of focused, well-researched, and time-tested efforts on this front.
What metrics are available in Cortex? What changes will I see to my Dashboard and Stats page?
Better traffic classification of uniques, organic listens, paid listens, non-human traffic, and an aggregate view of all activity.
- Organic Listens (Unique) - Organic means “likely human/real” traffic as classified by our new traffic classification algorithms that are based on mature digital media data best practices. This segment is the de-duplicated count of all presumed to be human traffic against your content.
- Organic Listens (All) - This segment counts all presumed to be human listens to your content. Even if a user accesses your content multiple times, each time they attempt to listen or download is counted here.
- Paid Traffic – This segment counts all listens that are coming from paid campaigns. This will be zero most of the time.
- Other/Bots – This segment consists of listening events that were classified by our new filtering algorithms as bots or likely non-human listening activity. You’ll notice a similar category in Google Analytics and other mature metrics platforms. Most non-human traffic is benign (for example, many search engines use web crawlers to learn about content and populate search). Check out this Adweek article for a big picture view of bot traffic on the Internet.
- All – this is all of the listening activity our servers saw before any filtering was done – we’ve set this to be the default view for now.
New engagement metrics - Answer the question “Did my listeners finish my podcast?” with our completion and abandonment rates. Use these stats to make editorial choices.
Discussions on content quality are often laced with a great deal of passion and subjectivity, so we’d rather let listeners tell hosts what they think of your content not through surveys and anecdotes which are prone to bias, but through measuring their actual behavior with your content. We’ve developed a means to track how long they stick around. This data is brutally honest and objective. The two initial engagement measures we are launching in this phase are:
- Abandonment Rate – The percentage of listeners that left less than 2 minutes after your content started playing (for all trackable listens of your show).
- Completion Rate – The percentage of those listeners that stuck around for more than 80% of your episode’s duration (for all trackable listens of your show).
New demographics - In March 2016, we launched Cortex, our podcast metrics platform that uses mature data science practices and technologies to sort out non-human traffic, measure listens from across the mobile and desktop ecosystem, and provide insights that help podcasters make editorial and marketing decisions. In phase 2, we added Engagement and Abandonment data.
Now, we’re releasing Demographic and Geographic data. This information is a bit different than the other data points we give you. We use third party services in combination with our own data to estimate these metrics for your listeners. This data should be viewed as an estimate based on data we have about a subset of your listeners. If we feel that we do not have enough data about your audience to make an estimate, we will not populate that section until we have enough data. We want to be as accurate as possible.
To view this data, login to your account and head to your Stats page. Scan past the listens, source, and engagement data to see these three charts:
- Listener age information is designed to give you an idea of the age groups your podcast reaches. Note that we do not provide information for listeners under the age of 18. This data is updated monthly and reflects the previous 30 days.
- Listener Gender information estimates the proportion of listeners that are male and female. This data is updated monthly and reflects the previous 30 days.
- Audience Geography is an estimate of the geographic distribution of your listeners broken down by country. The top five countries of listenership are displayed to the right of the map. This view is a reflection of the past 30 days.
Track total listens bound within a recent time period (instead of just “per episode” listens all time) - Since our platform can give you opportunities to monetize listens across ALL episodes rather than your latest episode, we break down listenership this way.
A new standard of measurement - To help you build trust with advertisers, and to give you measures that you can base your editorial decisions on, we use stringent mature digital media standards to classify your traffic. These numbers will reveal a harsh reality about your traffic, and while you might see higher numbers if you go to a platform that does not apply these mature techniques, our goal is to give you the closest view to reality that we possibly can and to align our metrics to where the industry has agreed, particularly with the efforts of the IAB Podcast Technical Working Group (which we are part of) on podcast measurement. We had pre-empted the industry movements way back in 2014 and are finding that the approach we have taken here is very much in alignment with those broader industry efforts (along with some of our own proprietary techniques to further improve accuracy).
Here’s how you’ll see these changes in your Dashboard, Episodes page, and Stats page:
- BETA summaries of Organic Listens, Abandonment Rate, and Completion rates for your show as a whole.
- A new traffic chart that tracks your DAILY listens across ALL your episodes, with ability to filter using our bot filtering methods as well as between streamed listens and downloads.
- We’ve moved episode metrics from the stats section to your Episode Page so that it is easier to track episode performance
- Each episode will now have the new organic, unique, growth, and engagement metrics
- You’ll now see the same metrics in your dashboard here
- You will also be able to see the composition of your listens for your show as a whole by source/user agent to see where your listens came from
What should I do if I need help interpreting a metric?
After you’ve read this article, if you still need support, send an email to email@example.com. Our support team will collaborate with our Head of Community to post webinars and new support articles.
How can I leave feedback?
Very often, the launch of new measurement tools to a community will open up new insights and learnings unanticipated by designers of such systems. We are quite sure the community is going to learn new things as you start playing round with the new tools. If you have feedback you’d like our product team to consider, email firstname.lastname@example.org - our team looks at this email account before each sprint.