Author: Olaf Kopp
Reading time: 23 Minutes

Quality Rater Guidelines 2023: Whats new?

4/5 - (4 votes)

Google released an new version of the Quality Rater Guidelines on 16.November 2023. In this article you can find what’new in the QRG.

What are the Google Quality Rater Guidelines?

The Google Quality Rater Guidelines are a set of criteria that Google uses to guide human evaluators in assessing the quality of search results. These guidelines help ensure that the search results are of high quality, relevant, and useful to users. Here are the key aspects of these guidelines:

  1. Purpose of the Page: Evaluators are instructed to determine the main purpose of a page. This can include providing information, selling products, entertaining users, or any other specific purpose.
  2. Experience, Expertise, Authoritativeness, and Trustworthiness (E-E-A-T): This is a crucial factor in assessing the quality of a page, especially for YMYL (Your Money or Your Life) content, which includes topics like health, finance, legal issues, etc. High-quality pages must demonstrate expertise, authoritativeness, and trustworthiness.
  3. Content Quality and Amount: The guidelines emphasize the importance of high-quality content. This includes the accuracy, originality, thoroughness, and comprehensiveness of the content presented.
  4. Website Information and Reputation: The reputation of the website and the information about the creators of the content are also important. This involves checking reviews, expert opinions, news articles, and other credible sources of information about the site.
  5. User Experience: The overall user experience provided by the website, including its design, usability, and the safety of the user, is a significant factor.
  6. Needs Met Rating: This criterion assesses how well a page meets the needs of users based on their search queries. This includes considering the intent of the search and how effectively the content addresses that intent.

These guidelines are used by Google’s human raters to evaluate search results, but they do not directly impact search engine rankings. However, they provide insights into what Google considers high-quality content, which can indirectly influence how content is optimized for search engines.

Whats new in the Quality Rater Guidelines 2023?

The Google Quality Rater Guidelines from 2023 incorporate several updates and changes, aimed at maintaining relevance and effectiveness in assessing the quality of web pages and search results. The changes are as follow with some examples:

  • Simplified Needs Met Scale Definitions: Added specific guidance for different types of Needs Met ratings.

Part 3: Need Met Rating Guideline

Rating Description A special rating category, which only applies to certain queries and results. All or almost all Fully Meets (FullyM) users would be immediately and fully satisfied by the result and would not need to view other results to satisfy their need. Highly Meets (HM) Very helpful for many or most users. Some users may wish to see additional results. Moderately Meets (MM) Helpful for many users queries with clear intent to find one specific Fully Meets (FullyM) result and the corresponding specific result the user is looking for. A very helpful result for any dominant, common or reasonable minor query interpretation/user Highly Meets (HM) intent. A helpful result for any dominant, common or reasonable minor query interpretation/user Moderately Meets (MM) intent . A less helpful result for a dominant, common or reasonable minor interpretation/user intent Slightly Meets (SM) OR very helpful for some users. Some or many users may wish to see additional results. Slightly Meets (SM) Helpful for fewer users. There is a connection between the query and the result, but not a strong or satisfying connection. Many or most users would wish to see additional results. Fails to Meet (FailsM) Completely fails to meet the needs of the users. All or almost all users would wish to see additional results. Please note that you may assign in-between ratings. a helpful result for an unlikely minor query interpretation/user intent. A result that completely fails to meet the needs of all or almost all users. For example, the Fails to Meet (FailsM) result may be off-topic for the query or address a no chance interpretation of the query.

13.1 Rating Result Blocks: Block Content and Landing Pages

Which part of the result do you rate? It depends on both the query and the result block: If a rating task specifically asks you to rate the Landing Page Needs Met or the Block Content Needs Met of each result, then you should follow the guidance in Section 13.1.1. Unless these specific versions of Needs Met ratings are requested, you should follow the guidance below:

13.1.1

Query and User Intent Result Rating Fully Meets Explanation 13.1.1 Landing Page Needs Met Ratings and Block Content Needs Met Ratings In most cases, you should follow the instructions above when assigning Needs Met ratings. In some cases, though, a rating task may include special instructions that ask you to assign Landing Page Needs Met or Block Content Needs Met ratings. In these cases, follow the instructions below: Type of Needs Met Rating What to Rate Landing Page Needs Met ratings should be based exclusively on the landing Landing Page Needs Met page(s) links in the result block. This applies to both SCRBs and Web Search Result Blocks. The content that appears inside the result block should not factor into Landing Page Needs Met ratings. Block Content Needs Met ratings should be based exclusively on the content inside the result block. This applies to both SCRBs and Web Search Result Blocks. Do not click through to any landing pages when rating Block Content Needs Met; landing page content should not factor into Block Content Needs Met ratings. Block Content Needs Met ● For Web Search Result Blocks, only consider the extent to which the page title, snippet, and any visual content within the block would directly address user need. ● For SCRBs, only consider the extent to which the content within the block would directly address user need. Needs Met Follow the instructions in Section 13.1 for standard Needs Met ratings. 13.2 Fully Meets (FullyM) Fully Meets is a special rating category which only applies when all of the following are true: ● The query interpretation and user intent is specific, clear, and unambiguous. ● All or almost all users would be completely satisfied by the result4users issuing that query would not need additional results to fully satisfy the user intent. In other words, the Fully Meets rating should be reserved for results that are the <complete and perfect response or answer= so that no other results are necessary for all or almost all users to be fully satisfied. There is a specific website, a specific webpage, a specific know-simple fact or other specific result that all or almost all users in the rating locale and user location are looking for. ● All or almost all users in the rating locale and user location would be completely satisfied by the result. Most queries cannot have a Fully Meets result. Most queries do not specify a single website or webpage. Most queries are not specific enough to have a single, know-simple fact as a comprehensive answer. For most queries, different people may want different types of results. For many queries, a single user may need to see many different results.

13.2.

13.2 Fully Meets (FullyM) Fully Meets is a special rating category which only applies when all of the following are true: ● The query interpretation and user intent is specific, clear, and unambiguous. ● All or almost all users would be completely satisfied by the result4users issuing that query would not need additional results to fully satisfy the user intent. In other words, the Fully Meets rating should be reserved for results that are the <complete and perfect response or answer= so that no other results are necessary for all or almost all users to be fully satisfied. There is a specific website, a specific webpage, a specific know-simple fact or other specific result that all or almost all users in the rating locale and user location are looking for. ● All or almost all users in the rating locale and user location would be completely satisfied by the result. Most queries cannot have a Fully Meets result. Most queries do not specify a single website or webpage. Most queries are not specific enough to have a single, know-simple fact as a comprehensive answer. For most queries, different people may want different types of results. For many queries, a single user may need to see many different results.

● The user is looking for a very specific fact or piece of information and the All or almost all users in the rating locale or user location who search for this query would be looking for the same, specific fact or piece of information. The result block provides the information thoroughly, accurately, and clearly, and is from a high quality source. No other results would be needed. Before using the Fully Meets rating for queries seeking a very specific fact or piece of information, you must check for accuracy and confirm that the information is supported by expert consensus where such consensus exists. Fully Meets may apply in other situations as well. Ask yourself whether the result block alone is the perfect and complete result that would fully satisfy all or almost all users. trustworthy source acceptable for all or almost all users. Before using the Fully Meets rating for queries seeking a very specific fact or piece of information, you must check for accuracy and confirm that the information is supported by expert consensus where such consensus exists. Be conservative when using the Fully Meets rating. When in doubt, consider a lower rating. Note: If a result block is very close to being fully satisfying, but the block alone may not be sufficient in order to fully satisfy all or almost all users, a rating of Highly Meets+ may be appropriate. Sometimes you have to think about how specific the user need is and will need to make a judgment call on whether it is specific enough to possibly have a Fully Meets rating. Even queries that seem to have a single right “answer” may benefit from more perspectives to satisfy all or almost all users. When in doubt, use a lower rating. Note: If a result block is very close to being fully satisfying, but the block alone may not be sufficient in order to fully satisfy all or almost all users, a rating of Highly Meets or Highly Meets+ may be appropriate.

Query and User Intent Result Rating Fully Meets Explanation Query: [titanic imdb] The query has clear intent to go User Intent: Go to the IMDb to the IMDb page for Titanic. page for Titanic. Query: [www.yahoo.c0m] Even though this is an imperfect URL query, it’s clear the user User Intent: Go to the Yahoo wants to go to the Yahoo website. [www.netflix.coom] Even though this is an imperfect URL query, it’s clear the user User Intent: Go to the wants to go to the Netflix streaming website, Netflix. website.

Query and User Intent Result Rating Fully Meets Explanation Query: [how to find Mac OS complete and correct answer The result block displays the version] prominently in the result block, in a way that is easy for users to User Intent: Find out how to read, and comes from a high locate the OS version of a quality, authoritative source for complete and correct answer prominently in the result block, User Intent: Find out how to in a way that is easy for users to locate the OS version of a read. The result block Fully Mac. Apple product information. The result block Fully Meets the user need. The result block displays the Query: Meets the user need. The result block displays the Query : [new york city complete and correct answer population 2012] prominently in the result block, in a way that is easy for users to User Intent: Find the read4it read—it Fully Meets the user population of New York City need. back in 2012.

13.3.

13.3 Highly Meets (HM) A rating of Highly Meets is assigned to very helpful results for any dominant, common, or reasonable minor interpretation or user intent of the query. Highly Meets results are highly satisfying and a good “fit” for reasonable interpretations and intents. In addition, they often have some of the following characteristics (depending on the query): ● Entertaining or enjoyable for queries with a reasonable entertainment intent ● Representative of the opinions, perspectives and experience of real people for queries with no single right answer ● For informational queries, very helpful results that, for example: ○ Are easy to understand the content, e.g., helpful images or videos, an easy to understand summary ○ Provide in-depth or insightful content, e.g., a deep dive explanation, an expert analysis ● For queries that benefit from recently created content, fresh results that provide up-to-date content on “what everyone is talking about right now” e.g., breaking news on a topic or the latest viral video or a hot new fashion trend Highly Meets results may be any type of content: articles, short form or long form video, images, social media posts, forum discussions, etc. There is no length requirement for Highly Meets results. Sometimes what makes a result satisfying is that it is short and easy to understand. Other results may be very satisfying because of the depth and richness of content. For some queries, comprehensiveness may be critically important, such as queries about truly complex topics or YMYL queries for which a short answer could be misleading. Accuracy is required for Highly Meets informational results. Highly Meets information pages on YMYL topics must be accurate and trustworthy. Highly Meets medical and scientific information pages must represent well-established scientific/medical consensus unless the user is clearly seeking an alternative viewpoint. For all information pages, you must check for accuracy and confirm that the information is supported by expert consensus where such consensus exists. A query can have many Highly Meets results. Have high standards for the Highly Meets rating. This is especially important for queries with many on-topic results. A query can have many Highly Meets results. Search engines should strive to provide a diverse set of Highly Meets results from a variety of websites in a variety of content types from a variety of people and perspectives. T his is especially important for queries with no single, “right answer”, subjective queries, queries with multiple intents and interpretations, think about how likely the query interpretation is and how helpful the result is. ● A very helpful result for a dominant interpretation should be rated Highly Meets, because it is very helpful for many or most users. Some queries with a dominant interpretation have a FullyM result. ● A very helpful result for a common interpretation may be Highly Meets or Moderately Meets , depending on how likely the interpretation is. ● A very helpful result for a very minor interpretation may be Slightly Meets or lower because few users may be interested in that interpretation. ● There are some interpretations that are so unlikely that results should be rated FailsM . We call these <no chance= interpretations. queries seeking opinions and perspectives, and queries simply looking to browse or explore the web. For most queries, Highly Meets is the highest possible rating a result can receive. Results rated Highly Meets represent the most helpful content available for a given query interpretation and user intent. Some queries have several Highly Meets results because there is a variety of very helpful content available on the internet that addresses reasonable query interpretations and user intents. Other queries may have no Highly Meets results at all because very helpful content doesn’t exist on the internet or search engines are unable to find it. The standard for Highly Meets is very high.

Query and User Intent Result Rating Highly Meets Explanation This is the official website for Query: [trader joes] Trader Joe’s, which has highly authoritative information on the User Location: Charlotte, businesses’ stores, promotions, North Carolina recipes, news, etc. User Intent: There are Note: This result block is not two possible user intents: Fully Meets because users who most users probably want want information to visit the store to visit a nearby location or have to do a bit of work to find go to the website. that information4it information on the businesses’ User Location: Charlotte, stores, promotions, recipes, North Carolina news, etc. User Intent: There are Note: This result block is not two possible user intents: Fully Meets because users who most users probably want want information to visit the store to visit a nearby location or have to do a bit of work to find go to the website. that information—it is not

User Intent: There are Note: This result block is not two possible user intents: Fully Meets because users who most users probably want want information to visit the to visit the car dealership dealership have to do a bit of or go to the website. work to find that information4it is not immediately available. information—it is not located) or go to the immediately available. website. Query: [museum of modern art] The result block shows information about the museum, User Location: and is especially helpful for users Manhattan, New York who want to visit the museum. User Intent: There are Note: This result block is not two possible user intents: Fully Meets— even though there most users probably want is a website link, it is not very to visit the museum (note satisfying for users who want to that the user is located in go to the website because it is NYC where the museum is not as informative as a web located) or go to the search result block with a title, website. snippet, and URL.

User Intent: Find Note: This result block is not information about this Fully Meets4 even though there library or go to the is a website link, it is not very website. satisfying for users who want to go to the website because it is not as informative as a web search result block with a title, snippet, and URL. There is only one library in the user location (Belmont, California) . The result block Query: [belmont library] shows information about the library, and is especially helpful User Location: Belmont, for users who want to visit the California location. User Intent: Find Note: This result block is not information about this Fully Meets4 even though there library or go to the is a website link, it is not very website. satisfying for users who want to go to the website because it is not as informative as a web search result block with a title, snippet, and URL. The user intent is too broad to have a Fully Meets result. However, this is The Query: [decemberists] Decemberists’ official website and has a lot of content including User Intent: Learn more tour dates, music videos, the about the band, The latest album, etc. The result is Decemberists. uniquely authoritative and most users would be very satisfied by this website. The user intent is too broad to have a Fully Meets result. Query: Meets— even though there library or go to the is a website link, it is not very website. satisfying for users who want to go to the website because it is not as informative as a web search result block with a title, snippet, and URL. There is only one library in the user location (Belmont, California). The result block Query: [belmont library] shows information about the library, and is especially helpful User Location: Belmont, for users who want to visit the California location. User Intent: Find Note: This result block is not information about this Fully Meets— even though there library or go to the is a website link, it is not very website. satisfying for users who want to go to the website because it is not as informative as a web search result block with a title, snippet, and URL. The user intent is too broad to Query : [celine dion] have a Fully Meets result. However, this is Celine Dion’s official website and has a lot of User Intent: Learn more content including recent news, User Intent: Learn more official website and has a lot of about the singer Celine content including recent news, Dion. videos, etc. The result is uniquely authoritative and most users would be very satisfied by this website.

  • Modernized Examples: Focused on popular platforms and video formats, removing outdated examples.
  • Expanded Rating Guidance: Specifically for forum pages.
  • Consistent Language Updates: Across various sections.
  • Refinement of Page Quality Rating: Focused on Main Content Quality, Website and Content Creator Reputation, and E-E-A-T.

14.0

Here is some guidance about assigning Needs Met and Page Quality ratings: ● Useless results should always be rated FailsM , even if the landing page has a high Page Quality rating. Useless is useless. ● On-topic, helpful, but low Page Quality results should get lower Needs Met ratings than on-topic, helpful, and high Page Quality results. The Needs Met scale encompasses all aspects of <helpfulness,= and many users find low Page Quality results less helpful than high Page Quality results. Your ratings should reflect this. ● The HM rating should be given to helpful, high Page Quality pages that are a good fit for the query. The HM rating may also be used for results that are very helpful, medium quality, and have other very desirable characteristics, such as very recent information. ● The HM rating may not be appropriate if a page has low Page Quality or has any other undesirable characteristics, such as outdated or inaccurate information, or if it is a poor fit for the query. We have very high standards for the HM rating. ● SM is often an appropriate rating for low quality but on-topic pages. However, a page can have such low Page Quality that it is useless for nearly all queries. Gibberish pages are a good example of pages with low Page Quality that should be rated FailsM . An The HM rating is not appropriate if a page is untrustworthy or has any other undesirable characteristics, such as outdated or inaccurate information. There are very high standards for the HM rating. ● A page can have such low Page Quality that it is unhelpful or even harmful, such as gibberish pages or scams. These types of extremely low quality pages should be rated FailsM because the Page Quality issues interfere with helpfulness. (An exception to this is queries with clear website intent, where the target website should be rated FullyM even if the page has low Page Quality. ● Remember that if a page lacks a beneficial purpose, it should always be rated Lowest Page Quality – regardless of the page’s Needs Met rating or how well-designed the page may be. Please review Section 4.1 for a summary of other types of Lowest Page Quality pages.  Query and User Intent Result Block and LP, Needs Met Rating, Page Quality Rating Explanation Needs Met : This page would be helpful for few users. Page Quality : Read this content out loud. The article on the LP is shallow and unintentionally humorous. The MC has four paragraphs of text, but it basically says that electric cars use Query: [how do electric electricity and have to be cars work] charged. The page also lacks E-E-A-T. Low is an appropriate User Intent: Find rating. information about how electric cars work. Needs Met : The query asks how electric cars work, but the article on the LP is about mobility scooters. Page Quality : There is a lot of helpful MC on the landing page on Wikipedia.org. The page has high E-E-A-T. Medium+ to High is an appropriate rating. is queries with clear website intent, where the target website should be rated FullyM . People are allowed to search for any website and search engines should honor that intent when it is clear.)

Query and User Intent Result Block and LP, Needs Met Rating, Page Quality Rating Explanation is a very helpful result for the query. Page Quality: This is a YMYL topic. This page is on a highly authoritative medical website and has a lot of reliable and accurate MC. This page is very trustworthy.

15.2.3.

15.2.3 Needs Met Rating for Clear Porn Intent Queries For very clear porn intent queries, assign a rating to the porn landing page based on how helpful it is for the user. Even though there is porn intent, the page should still be assigned a Porn flag. Do not simply rate all porn pages for porn queries as MM or HM . Even though the query is porn and the result is porn, the page must fit the query and be helpful to get a high Needs Met rating. Pages that provide a poor user experience, such as pages that try to download malicious software, should also receive low ratings, even if they have some images appropriate for the query. Query User Intent Landing Page Rating Porn Flag? [freeones], English (US) Navigate to the Freeones Example: – this page is porn. FailsM Yes 15.2.3 Needs Met Rating for Clear Porn Intent Queries For very clear porn intent queries, assign a rating to the porn landing page based on how helpful it is for the user. Even though there is porn intent, the page should still be assigned a Porn flag. Do not simply rate all porn pages for porn queries as MM or HM . Even though the query is porn and the result is porn, the page must fit the query and be helpful to get a high Needs Met rating. Pages that provide a poor user experience, such as pages that try to download malicious software, should also receive low ratings, even if they have some images appropriate for the query. Query User Intent Landing Page Rating Porn Flag? Example: [freeones], English (US) Navigate to the Freeones

15.4.2

Videos are often an example where foreign language pages are helpful and desired. Think about user intent and what pages are good for users. If the query <asks= for a foreign language song, band, film, sporting event, etc., then a video of the song, band, film, sporting event, etc. is helpful since it can probably be understood or enjoyed even though it is in a foreign language. For these types of queries, foreign language results are often expected. If the video is someone talking about the song, band, film, or event, the result probably cannot be understood or enjoyed and should be assigned a FailsM rating and the Foreign Language flag. Note: If you are unable to evaluate the Page Quality rating of a Foreign Language result, you do not need to assign a Page Quality rating and can leave the slider at N/A.  “asks” for a foreign language song, band, film, sporting event, etc., then a video of the song, band, film, sporting event, etc. is helpful since it can probably be understood or enjoyed even though it is in a foreign language. For these types of queries, foreign language results are often expected. If the video is someone talking about the song, band, film, or event, the result probably cannot be understood or enjoyed and should be assigned a FailsM rating and the Foreign Language flag. Note: If you are unable to evaluate the Page Quality rating of a Foreign Language result, you do not need to assign a Page Quality rating and can leave the slider at N/A.

16.0

16.0 Rating Queries with Multiple Interpretations and Intents Some queries really only have one meaning. Consider the query [iphone], English (US). There may be different user intents for this query (research iPhones, buy an iPhone, go to the iPhone page on Apple’s website), but all users are basically referring to the same thing: the phone made by Apple, Inc. Some queries truly have different possible meanings. Consider the query [apple], English (US). Some users may want to find more information on the computer brand or the fruit. We refer to these different meanings as query interpretations. When giving Needs Met ratings for results involving different query  16.0 Many Types of Helpful Results Some queries may have many different possible meanings, or people may be looking for different things. Make sure to give Highly Meets ratings to very helpful results for all reasonable interpretations and intents, including reasonable minor interpretations. ● Many queries have different possible meanings or interpretations. Don’t assume – do query research! Results for a diverse set of reasonable interpretations may be very helpful. ● For any given query interpretation, different users may have different intents. For example, different people who search for [avatar] with the movie in mind could be seeking different kinds of information about the movie. Some may be seeking reviews, others could want to see what people are saying on social media, while others still could be interested in the cast or ways to watch the movie, etc. Results that satisfy different reasonable intents may be very helpful to different users. ● It’s also possible for a single user to have many different intents for any given query interpretation. That is, one person searching [avatar] could be broadly interested in the movie, and that person might want to browse reviews and social media posts, see the cast, and find out how to watch the movie. A diverse set of results that satisfy different reasonable intents may be very helpful to individual users, too! Results of different types may be helpful for many kinds of queries, including queries with a single, clear interpretation or intent. Consider what kinds of results are helpful for all queries you rate, including: ● Different formats (e.g., text, video, image, audio) ● Different experiences, perspectives and opinions (e.g., a personal blog or a social media post) ● Different content depths (e.g., the 10 second summary video and the 1 hour deep dive documentary) ● Different types of sites: corporate, not for profit, government, social media, forums, personal blogs, etc. ● … and many more! Make sure all types of very helpful results get a Highly Meets rating. Some queries may benefit from a wide variety of helpful results: ● Queries that seek experiences, opinions and perspectives, such as [should i get a dog], [what does love feel like] ● Queries that are subjective, such as [best music to run to] ● Exploration and browsing queries – users may need to see many different options, such as [ideas for bathroom remodel], [black suit], [inexpensive engagement rings] ● Queries with no single, right answer, such as [how to ask your boss for a raise]

17.0

Query and User Intent Result Block & LP, Needs Met & Page Quality Ratings Explanation Even though there are many options, a page for an individual company that offers different cards could be very helpful. Needs Met : This LP is more specific than the query, but could still be a very helpful result because Visa is a popular credit card company. Page Quality: Visa is a credit card company with high E-E-A-T that offers services for credit cards, banking, etc. and has a good reputation. High+ to Highest is an appropriate rating. Even though there are many options, a page for an individual company that offers different cards could be very helpful. Needs Met : This LP is more specific than the query, but could still be a very helpful result because Discover is a popular credit card company. Query: [credit cards] Page Quality : Discover is a company with User Intent: Users are high E-E-A-T that offers services for credit probably looking to sign cards, banking, etc. with a good reputation. up for a credit card High+ to Highest is an appropriate rating. Quality: Discover is a company with high E-E-A-T that offers services for credit User Intent: Users are cards, banking, etc. with a good reputation. probably looking to sign High+ to Highest is an appropriate rating. up for a credit card online, or want to This page offers a list of top credit cards in research credit cards a variety of categories. before signing up.

  • Refreshed YMYL (Your Money Your Life) Focus: Concentrated on high-accuracy topics to prevent significant harm.

2.3

2.3 Your Money or Your Life (YMYL) Topics Pages on the World Wide Web are about a vast variety of topics. Some topics have a high risk of harm because content about these topics could significantly impact the health, financial stability, or safety of people, or the welfare or well-being of society. We call these topics <Your Money or Your Life= 2023 10 2.3 Your Money or Your Life (YMYL) Topics Pages on the World Wide Web are about a vast variety of topics. Some topics have a high risk of harm because content about these topics could significantly impact the health, financial stability, or safety of people, or the welfare or well-being of society. We call these topics “Your Money or Your Life” or YMYL.

13.3

13.3 Highly Meets (HM) A rating of Highly Meets is assigned to results that meet the needs of many or most users. Highly Meets results are highly satisfying and a good <fit= for the query. In addition, they often have some or all of the following characteristics: high quality, authoritative, entertaining, and/or recent (e.g., breaking news on a topic). In order to receive a Highly Meets rating, information pages such as encyclopedia articles and news articles must be accurate and highly credible. 2023 110 13.3 Highly Meets (HM) A rating of Highly Meets is assigned to very helpful results for any dominant, common, or reasonable minor interpretation or user intent of the query. Highly Meets results are highly satisfying and a good “fit” for reasonable interpretations and intents. In addition, they often have some of the following characteristics (depending on the query): ● Entertaining or enjoyable for queries with a reasonable entertainment intent ● Representative of the opinions, perspectives and experience of real people for queries with no single right answer ● For informational queries, very helpful results that, for example: ○ Are easy to understand the content, e.g., helpful images or videos, an easy to understand summary ○ Provide in-depth or insightful content, e.g., a deep dive explanation, an expert analysis ● For queries that benefit from recently created content, fresh results that provide up-to-date content on “what everyone is talking about right now” e.g., breaking news on a topic or the latest viral video or a hot new fashion trend Highly Meets results may be any type of content: articles, short form or long form video, images, social media posts, forum discussions, etc. There is no length requirement for Highly Meets results. Sometimes what makes a result satisfying is that it is short and easy to understand. Other results may be very satisfying because of the depth and richness of content. For some queries, comprehensiveness may be critically important, such as queries about truly complex topics or YMYL queries for which a short answer could be misleading. Accuracy is required for Highly Meets informational results. Highly Meets information pages on YMYL topics must be accurate and trustworthy. Highly Meets medical and scientific information pages must represent well-established scientific/medical consensus unless the user is clearly seeking an alternative viewpoint. For all information pages, you must check for accuracy and confirm that the information is supported by expert consensus where such consensus exists. A query can have many Highly Meets results. Have high standards for the Highly Meets rating. This is especially important for queries with many on-topic results. A query can have many Highly Meets results. Search engines should strive to provide a diverse set of Highly Meets results from a variety of websites in a variety of content types from a variety of people and perspectives. T his is especially important for queries with no single, “right answer”, subjective queries, queries with multiple intents and interpretations, think about how likely the query interpretation is and how helpful the result is. ● A very helpful result for a dominant interpretation should be rated Highly Meets, because it is very helpful for many or most users. Some queries with a dominant interpretation have a FullyM result. ● A very helpful result for a common interpretation may be Highly Meets or Moderately Meets , depending on how likely the interpretation is. ● A very helpful result for a very minor interpretation may be Slightly Meets or lower because few users may be interested in that interpretation. ● There are some interpretations that are so unlikely that results should be rated FailsM . We call these <no chance= interpretations. queries seeking opinions and perspectives, and queries simply looking to browse or explore the web. For most queries, Highly Meets is the highest possible rating a result can receive. Results rated Highly Meets represent the most helpful content available for a given query interpretation and user intent. Some queries have several Highly Meets results because there is a variety of very helpful content available on the internet that addresses reasonable query interpretations and user intents. Other queries may have no Highly Meets results at all because very helpful content doesn’t exist on the internet or search engines are unable to find it. The standard for Highly Meets is very high.

  • Device Agnostic Language: To be applicable across all device types.
  • General Minor Updates: Including language, examples, and fixing of typos.

These changes reflect Google’s ongoing efforts to improve the relevance and quality of search results, emphasizing the importance of content quality, authoritativeness, and user experience. The 2023 guidelines show a greater emphasis on modernization, with specific attention to popular platforms and video formats, and a simplified approach to Needs Met ratings. The 2022 guidelines focused more on broadening the applicability of the guidelines to various types of websites and refining the core pillars of Page Quality Rating​.

Summary of the Quality Rater Guidelines

The Google Quality Rater Guidelines provide comprehensive instructions for evaluating the quality of web pages and search results. Here’s a detailed summary of the key sections:

  1. Introduction: The guidelines begin with an overview of the purpose of the document, emphasizing its role in improving search results by providing human feedback to Google’s algorithms.
  2. Understanding Webpages and Websites: This section instructs raters on how to evaluate a website’s purpose, its content, and the context of the information provided.
  3. Page Quality Rating: It delves into the criteria for assessing the quality of a page, focusing on factors like experience, expertise, authoritativeness, trustworthiness (E-E-A-T), and the quality of content.
  4. Needs Met Rating: This part focuses on how well a page meets a user’s search intent. It includes various scenarios and examples to illustrate different levels of user satisfaction.
  5. Understanding User Intent: The guidelines emphasize the importance of understanding what users are looking for when they perform a search and how this affects the evaluation of search results.
  6. Mobile User Needs: Given the increasing use of mobile devices for internet access, this section addresses the unique needs and challenges of mobile users.
  7. Additional Rating Tasks and Instructions: This segment covers various other tasks that raters may encounter, such as rating dictionary results, understanding the user location, and handling misspelled queries.
  8. Appendix and Additional Resources: The document concludes with an appendix that provides additional resources and explanations to assist raters in their tasks.

Throughout the document, the emphasis is on providing accurate, unbiased ratings based on the guidelines, with numerous examples and scenarios to guide raters in various situations. The guidelines also highlight the evolving nature of web content and the importance of staying current with new types of content and changing user behaviors.

Different Versions of the Quality Rater Guidelines

Here are different Versions of the QRG to download:

  1. Quality Rater Guidelines 2014
  2. Quality Rater Guidelines 2015
  3. Quality Rater Guidelines May 2017
  4. Quality Rater Guidelines July 2017
  5. Quality Rater Guidelines 2022

About Olaf Kopp

Olaf Kopp is Co-Founder, Chief Business Development Officer (CBDO) and Head of SEO & Content at Aufgesang GmbH. He is an internationally recognized industry expert in semantic SEO, E-E-A-T, LLMO, AI- and modern search engine technology, content marketing and customer journey management. As an author, Olaf Kopp writes for national and international magazines such as Search Engine Land, t3n, Website Boosting, Hubspot, Sistrix, Oncrawl, Searchmetrics, Upload … . In 2022 he was Top contributor for Search Engine Land. His blog is one of the most famous online marketing blogs in Germany. In addition, Olaf Kopp is a speaker for SEO and content marketing SMX, SERP Conf., CMCx, OMT, OMX, Campixx...

COMMENT ARTICLE



Content from the blog

What is the Google Shopping Graph and how does it work?

The Google Shopping Graph is an advanced, dynamic data structure developed by Google to enhance read more

How Google can personalize search results?

The personalization of search results is one of the last steps in the ranking process read more

LLMO: How do you optimize for the answers of generative AI systems?

As more and more people prefer to ask ChatGPT rather than Google when searching for read more

The dimensions of the Google ranking

The ranking factors at Google have become more and more multidimensional and diverse over the read more

How Google evaluates E-E-A-T? 80+ signals for E-E-A-T

In 2022 I published an overview of E-E-A-T signals for the first time, which Google read more

E-E-A-T: More than an introduction to Experience ,Expertise, Authority, Trust

There are many definitions and explanations of E-E-A-T, but few are truly tangible. This article read more