top of page

Of course, search engines like Google and YouTube are some of the most popular places for people to find content. Understanding search engine optimization and knowing how to optimize your own content will help you boost your visibility on the various search engine results pages (SERPs).


In 1996, Microsoft founder Bill Gates wrote a famous essay titled "Content is King". He wrote about the commoditization of content, likening it to the broadcasting and publishing industries. Claiming that the internet will be the primary focus for making money for people who can produce the best content.

He was right. The web has taken over from print publishing and is steadily changing broadcasting. But to still say, over 20 years later, that content is king—the top of the pile—is to miss out on vital elements of your audience's online experience.

Brands need to understand the content discovery process their audience goes through on their website, social media platforms, and other online channels to truly understand their audience’s context. Once you know what the process looks like, you can use this information to improve your own content strategy.

"Brands need to understand the content discovery process their audience goes through on their website, social media platforms, and other online channels to truly understand their audience’s context."

Tweet this

(you can edit before posting)

What is content discovery?

Content discovery is the process of searching through and finding content. This is a process that your customers and leads go through when they find and engage with your brand’s content on your website, social media sites, and anywhere else online.

When you understand the content discovery process that your audience goes through, you gain a better understanding of how people engage with your content. Once you know that, you can find ways to improve your marketing strategy to encourage further engagement.

Audience research: The key to content discovery

How do you tap into your audience's context to find the content you need to create?

There aren't many shortcuts to good content discovery. But how long and thorough you are will depend on the type of content you're planning. Consultant Lauren Pope is the absolute best at explaining content cadences, and you can see her fabulous models for content planning right here.

As a rule of thumb, the longer your content will be around and relevant, the more discovery you should do for it.

Never think you can do enough by just asking a few people in the office some questions. No one in your organization is actually in your audience. You're all too invested and have too much insider information.

Search bar with query input for finding specific content.

Like the best method actors, you need to understand your audience's contexts by carrying out some research, asking some questions, and trying stuff out.

Let's start with the fundamental questions.

"As a rule of thumb, the longer your content will be around and relevant, the more discovery you should do for it."

Tweet this

(you can edit before posting)

Who is your audience?

You can't be all things to all people. Aim for a target, and you're likely to attract plenty of others, too.

Hopefully, you'll have some kind of audience outline in your organization somewhere. Buyer personas or demographic analysis can give you a start, but don't rely on these too much. They're often produced for a different reason than what you’re using them for and can lead you down a narrow path.

If you don't have anything at all, it's worth digging a little:

  • What is the average age of your audience?

  • Are they from a certain location?

  • What kind of device(s) are they using most to navigate content?

  • Where do they go to find great content?

  • What channels or social networks do they use to discover new content?

What does your audience want?

General information about your audience should act as your map. But what will fuel your content strategy is to get an understanding of your audience's motivations and needs.

The best way of doing this is to create user stories. These are short statements that tell you what your audience is looking for and why they want it.

Once you have user stories, you should have a good idea of your audience's motivations. But you might want to drill down a bit further into what people will do to satisfy those motivations.

What's holding your audience back?

Having a good understanding of what your audience wants is the first step in being able to create content that will drive traffic to your brand. But you can’t just focus on their motivations—you also must address what’s holding them back from moving forward.

Your content marketing team will often take the customers’ pain points and misconceptions into consideration when creating content. This information will also help you understand how they consume content.

For example, if you sell cars on a website, your customers will probably need to look around before they buy. That's a temporary state, easily fixed.

What content already exists, and how is that being viewed?

The first step to really understanding the content discovery process for your target audience is to perform a content audit. By auditing the different types of content you’ve published, you can also see how people access and engage with this content.

Categories and tags for organizing and filtering content.

The content audit will also be an opportunity for you to see what content can be improved and what content may need to be eliminated.

When you perform a content audit for your organization, you might also consider auditing your competitors’ content.

Competitors and parallel organizations will have a wealth of relevant content that your target audience is also engaging with. Auditing this will allow you to understand what types of content your audience likes to consume and how they engage with this content.

Need to know: If you need more help, then check out this guide that explains in three steps how to effectively prepare for a content audit.

💡 See Also: Download this free Content Audit Spreadsheet to get started today!

Use this data to understand the content discovery process

Once you’ve gathered information from your actual audience and done a thorough audit of your own content, you’ll want to validate this information. This ensures that you’re not just making assumptions about the content discovery process but that you’re also understanding how your target audience looks for quality content.

Get a mixture of qualitative and quantitative feedback from real people. This is one of the many times when UX, or service design, people can really shine.

If you have them in your team, you've probably already been working together on much of the stuff we've covered. Lean on them more here and urge them to set up user testing sessions, focus groups, and audience surveys.

As a content designer, your job will be to listen out for trigger words or phrases that you can use in your content. Then, make notes. You want to hear what people are saying and figure out what words are being used in any discussions and in response to prototypes or sketches they're seeing.

Listen or watch recordings of live sessions to make sure you've not missed anything. Sometimes it's worth transcribing what's said so you can easily return to it later to find the most commonly used words and discover what’s trending.

Periodically audit your content discovery process

The mistake a lot of organizations make is stopping all content discovery once your content is built or launched. However, you need to keep looking at the numbers coming in to make sure the content is doing what you expect it to do.

If a piece of content doesn’t have the right effect on your audience, or you find that they are not engaging with it in the way they typically do, it’s time to figure out why.

Here are a few potential issues:

  • The content is in the wrong place. Is your audience finding your content? Do they get enough context or information on previous pages or sites before seeing your content? Is it published to the right location and through the right distribution channels?

  • Something about the context has changed. This can be planned such as seasonal weather changes, search algorithm changes, or less marketing spend. It can also be unplanned changes such as political situations, unexpected weather changes, or something like the COVID-19 pandemic.

  • The format isn't right. Would your content be better as a video or an audio file? Is it too long or too short? Maybe a social media post might suit the content better.

  • It hasn't been marketed enough. Can you entice people in, maybe through email or social media? Does your content have the right search phrases in it?

It's a really good idea to start content discovery a few months after launch whether you've hit your targets or not. Reassessing your audience and their contexts will let you know if you have to change your content.

Top content discovery tools

Knowing how to navigate different content discovery tools not only helps you better understand how your audience consumes content, but it can also be helpful for content curation or making content recommendations to your audience.

Here are just a few of our favorite content discovery tools:


Featured content carousel showcasing popular topics.
 Recommended content based on user preferences and history

 Outbrain is a content recommendation platform powered by native advertising. Their goal is to help advertisers get their content discovered on the open web. The process is simple: you buy ads from Outbrain, and they will display your content on other websites to your target audience.


Trending topics section highlighting current popular articles.

 Feedly allows you to organize all of your favorite sources of information in one place. Instead of going to each website and reading the latest articles, you can compile them all in one feed. This tool also features an AI assistant that filters out the noise so you're only seeing what’s relevant to you.


Subscription options for receiving updates on new content.

Now is the best time
As Rob from GatherContent points out in his write-up of lessons learned from his blog redesign project, leaving your content alone and completing a discovery process on everything is a painstaking task.
Evaluating all of your content in one hit will seem like a mountain to climb—especially if this is your first time doing it. But the steps above will give you a starting point for understanding the content discovery process your audience goes through to find your content.
Once you know this, you can find ways to improve the user experience and make it even easier for your customers and leads to find and engage with your content.
Tagged as:
Content Strategy
Content Audit
Audience Research
Content Creation

Feedback mechanism for users to suggest or rate content."

Discovering hidden or private content on a web-server that could lead to new vulnerabilities.

Task 1: What is Content Discovery?

To answer these questions in this task, read and understand the details on this section.

Task 2: Manual Discovery — Robots.txt

The robots.txt file may be a document that tells search engines which pages they are and aren’t permitted to appear on their search engine comes about or ban particular search engines from crawling the site inside and out. It can be common practice to limit certain site areas so they aren’t shown in search engine results.

These pages may be areas such as administration portals or files implied for the website’s clients. This record gives us a incredible list of areas on the site that the owners do not need us to find as penetration testers.

Now, let’s visit the URL with a robots.txt directory. http://<IP MACHINE target>/robots.txt

Task 3: Manual Discovery — Favicon

The favicon is a small icon displayed in the browser’s address bar or tab used for branding a website. Open your web browser and open the given challenge link.

We just have the “Website coming soon…” webpage. Let’s try to view the page source and see if we can obtain anything.

It seems we have something identified as a jpeg image. Let’s follow the href data.

We have something here, but we can not see it clearly. Let’s try to download the ico file and have a well examination on it.

After download, let’s check the md5sum of the favicon.ico file.

From the OWASP favicon database, now let’s try to check the hash of the file and find the framework which it belonged to

There it is!!!

Task 4: Manual Discovery — Sitemap.xml

Unlike the robots.txt file, which restricts what search engine crawlers can look at, the sitemap.xml file gives a list of every file the website owner wishes to be listed on a search engine. These can sometimes contain areas of the website that are a bit more difficult to navigate to or even list some old webpages that the current site no longer uses but are still working behind the scenes.

Let’s take a look at the Acme IT Support website to see if there’s anything we can discover.

Yeahhhp, you have it now!

Task 5: Manual Discovery — HTTP Headers

When we make requests to the web server, the server returns various HTTP headers. These headers can sometimes contain useful information such as the web-server software and possibly the programming/scripting language in use. In the below example, we can see the web-server is NGINX version 1.18.0 and runs PHP version 7.4.3. Using this information, we could find vulnerable versions of software being used.

Let’s run the curl command with the -v flag

Task 6: Manual Discovery — Framework Stack

Visit the given ip address. The website will appear as this one below

Let’s view the page source if we can have any clues hidden there. Viewing the page source CTRL + U

At the bottom of the page, we have a URL given on a comment section. Let’s check it out, and find if we can have any other clues for us.

On the documentation page, we have other clues. As been directed, let’s find them. Login and obtain the flag.

Task 7: OSINT — Google Hacking / Dorking

Read and understand the details in there

Task 8: OSINT — Wappalyzer

“Wappalyzer is an online tool and browser extension that helps identify what technologies a website uses, such as frameworks, Content Management Systems (CMS), payment processors and much more, and it can even find version numbers as well.”

Task 9: OSINT — Wayback Machine

The Wayback Machine is a cool tool from the Internet Archive that lets you travel back in time and see how websites looked in the past! Think of it as a giant archive of the internet.

Task 10: OSINT — Github

Git is a version control system that tracks changes to files in a project. It can be used to delve into code hosted on GitHub. This can reveal details about projects, technologies used, and potential vulnerabilities (especially if the code is for internal systems). GitHub profiles and repositories often contain a wealth of information. Look for bios, links to other social media, affiliations, and even email addresses (be mindful, some might be accidental). Utilize GitHub’s built-in search to find repositories, users, and code snippets based on relevant keywords or phrases. Refine your search using filters like programming language, popularity (stars/forks), and more.

NB: Always be ethical and respectful of privacy when using GitHub for OSINT.

Task 11: OSINT — S3 Buckets

Read about Amazon AWS S3 buckets

Task 12: Automated Discovery

Let’s perform directory brute-forcing using the gobuster tool. You can use the tool as follows;

gobuster dir <IP MACHINE target> -u </path/to/wordlist>

At last!! We have found all the details, first the hidden directory (starting with mo) and the .log file.


bottom of page