SEO (search engine optimization) has traditionally been all about keywords. Keywords play an essential role in how information is retrieved. Therefore, it was all logical: search engines scanned, found, indexed, and classified content based on keywords.
SEO still revolves around keywords, but to a lesser extent. A piece of content’s relevance is ranked by how often it mentions the keyword – and all other signals included – on a page.
The term frequency is referred to as “TF”.
There is a problem in that stop words like “the,” “a,” “and,” “in,” “at,” and so on are far more frequent. A different approach is needed. In other words, it evaluates how often a term is used when compared to the frequency of all other terms.
This prevents stop words from being classified as more relevant.
Known as “inverse term frequency” or “IDF,” this formula looks at the frequency of keywords as well as that of other terms across different pieces of work (such as a site’s blog). We eliminate stop words that occur more frequently than other words.
This formula, called TF*IDF, gives a keyword a certain numerical weight based on inverse document frequency (multiplied by) term frequency.
A keyword is weighed by how many times it appears on a page. As a result, the page with the most frequency will score – and be ranked higher – by weighing the keyword across all the other pages.
To find out how important a document is within a group of documents, the formula can be applied. This can be applied to the relevance of a page regarding a particular keyword in comparison with the relevance of all the other pages within a website, for instance.
As a result, for a very long time, SEO focused not only on keywords, but also recommended a primary keyword for each page. These guidelines are still followed by many of the SEO tools and plugins used by content creators.
TF-IDF is greatest benefit when compared with a large amount of data – like the whole Internet – to determine the weighting of any particular keyword. It was the basis of how search engines would rank certain pages from multiple websites. Search engines used this ranking method to determine the ranking of certain pages from multiple websites. To some degree, that remains the case.
As I mentioned, the formula is a bit more complex than what I’m explaining here, but fundamentally speaking, whichever page ranks higher has a greater weight.
It has remained the standard method of retrieving and evaluating information since 1972. To put it simply, it gives sophisticated software a base to work from in order to build artificial intelligence (AI).
This formula, however, poses a problem by itself.
Getting high rankings started in the primitive days of the Internet, when writing pages with lots of keywords was the most straightforward and simplest way of achieving this. Even if you add an extra keyword here and there, it doesn’t matter.
The problem became worse as more website owners discovered the trick and began abusing it. It rendered search engine results unbearably messy by bringing up keyword-stuffed pages that did not make any sense, and it seriously affected user experience.
TF-IDF assesses relevance, but it fails to indicate quality – that is, whether the content in which the keyword appears is of high quality. Consequently, search engines could now determine how relevant a page is beyond keyword relevance as well as the meaning of its pages. Due to major Google algorithm updates, keyword-driven content plays a smaller role (or should we say, is less reliant on) now, making TF-IDF less relevant.
Only keywords are considered in the TD-IDF. It doesn’t take into account keyword variations, semantically related terms, or keyword relationships. Furthermore, it ignores synonyms and terms that are related by themes or context.
Take, for example, the keyword “soap.” Bathing soap, dishwasher soap, laundry soap, cleaning soap, shaving soap, and so on are all available. Handmade soap, perfumed soap, baby soap, medicated soap, glycerin soap, and other varieties of soap are available.
When you include different types of soap, such as shampoo liquid, laundry detergent, shower gel, shaving cream, bubble bath, and so on, the problem becomes even worse. “Soap operas,” “Soap” TV shows, “to soap” (to flatter), and SOAP (Simple Object Access Protocol).
The options are practically limitless.
Keywords alone can be misleading if you don’t consider the context, such as keyword variations, their position in the content, keyword interrelatedness, and how keywords fit within and link to the rest of the page.
2. It disregards the significance of the keyword.
The TF-IDF tries to figure out how relevant a keyword is, but it doesn’t take into account how meaningful that relevance is. What if a keyword that is more relevant but appears less frequently is more relevant? What if a different website, with different keywords, provides higher value for the same keyword?
In other words, the TF-IDF fails to take into account not only how keywords fit into the context of the page, but also how they fit with the rest of the site. As a result, other pages may include the same keywords, but their content may be more relevant and suited to the issue.
To put it another way, relevance is not the same as importance.
Although a keyword may be judged significant because it is used more frequently, this does not imply that it raises the page’s worth. Regardless of the keyword’s frequency or TF-IDF score, other keywords, let alone other pages, may be more relevant.
When compared to other pages, a page featuring the term “medicated soap” has a better relevancy score. However, a less relevant page might go into further detail about antibacterial, antifungal, and antimicrobial soap, which is more topically relevant.
Other keywords and keyword variations (as well as related but different keywords) may be located on other pages that are more relevant to the user than the document TF-IDF is analysing, comparable to the preceding limitation.
Finally, there’s the most crucial factor to consider. In fact, it isn’t even considered in the TF-IDF equation. And there you have it, the user.
TF-IDF can help you figure out what the page is about. However, it may be either vague or too specialised for the user’s needs. TF-IDF may also be comparing it to entirely other pages that serve different consumers or achieve different objectives.
All of the other pages are thrown into the mix, regardless of their intended purpose. For example, TF-IDF might compare a term in a blog post to a keyword on a shopping page, a FAQ page, or a page aimed at a completely different industry.
Fortunately, frequency is simply one of several factors considered when evaluating keywords. In addition, other ranking variables come into play when deciding how relevant a keyword is.
Machine learning and a method known as “natural language processing” (NLP) have changed the way we think about keywords in recent years. While keywords will continue to be essential, new algorithms will examine and attempt to understand them on a deeper, more complex level.
To solve the three issues highlighted above, search engine software is now attempting to comprehend keywords by studying how they are used in normal language. It accomplishes this by taking into account the terms’ usage, context, and relationship to one another.
As a result, the software can discern the nuances and intricacies of a keyword. Technically, they’re still keywords, but in the field of NLP, they’re referred to as “entities.”
Because they transform the way we think about SEO, entities are becoming increasingly significant, especially in digital marketing. We can no longer optimise content just based on keywords because we’ve given them meaning that changes based on many aspects.
A keyword’s meaning might vary greatly depending on its context, usage, and placement in the content (i.e., its surrounding text and other on-page elements). It can also completely change the meaning of a paragraph or piece of content, giving it a completely different tone.
As the proverb goes, and it absolutely applies to today’s SEO:
Without context, content is worthless.
Entities are words that have a specific meaning based on the context. They can be “names,” “types,” or “attributes,” among other things. They can connect to other concepts. They help to identify a specific person, item, or event by grouping them.
“Antibacterial soap,” for example, is one thing, whereas “hand sanitizer” is another. Although the latter is not a separate type of soap than the former, they are nonetheless related. Both are various forms of “disinfectant cleansers” depending on the situation (another entity).
To expand on this example, “antibacterial soap” has a different meaning in an article about “COVID-19” (also an object) due to the context. As a result, the term “antibacterial soap” has little meaning. It does, however, have value, importance, and purpose as a separate entity.
Rather of thinking about keywords as being linear or on a spectrum, consider them as existing in a cluster of linked concepts, similar to a hub-and-spoke wheel. (Google refers to them as “branches” and “nodes,” and groups them into a “Knowledge Graph.”)
Consider the words “head” and “shoulders.” These are two distinct keywords. A different keyword is “head and shoulders.” However, “Head & Shoulders” is a trademark. It’s a thing. Also, “dandruff,” “shampoo,” and “anti-dandruff shampoo” are all separate entities that are connected.
Entities are significantly more sophisticated than what I’m attempting to explain in this paper, and they have far more consequences and applications than I’m able to adequately discuss.
The crucial thing to remember is that, like it or not, search behaviour has changed. As a result, SEO has changed and will continue to change. As a result, it stands to reason that the practise and practitioners of SEO, as well as the beneficiaries of SEO services, are all connected.
Previously, search queries were usually made up of a single or a few words linked together. However, the intricacy of human language was not taken into consideration in the search results.
The search results were disjointed. It was primarily a game of luck to get the response you wanted. (In fact, it’s probably one of the reasons why Google created the “I’m Feeling Lucky” button, which allows you to skip all of the search result pages, or SERPs.)
Users would add more keywords to their queries in an attempt to achieve better results. However, this frequently backfired: Google would examine certain keywords within the query and provide different results for each. Then, regardless of r, it would rank everything.
Keywords have become intrinsically meaningless since the emergence of entity-oriented search (a term used by former Google researcher Kriszrtian Balog). Or, to put it another way, focusing on keywords and their ranks has become pointless.
Keywords are still useful in certain situations, such as performing research. However, optimising content with specific keywords — and attempting to rank for them — is rapidly becoming obsolete.
Chasing precise keywords is meaningless today, with digital assistants, intelligent devices, and voice search allowing users to ask long, sophisticated, and nuanced questions.
Keywords have become intrinsically meaningless since the emergence of entity-oriented search (a term used by former Google researcher Kriszrtian Balog). Or, to put it another way, focusing on keywords and their ranks has become pointless.
Keywords are still useful in certain situations, such as performing research. However, optimising content with specific keywords — and attempting to rank for them — is rapidly becoming obsolete.
Chasing precise keywords is meaningless today, with digital assistants, intelligent devices, and voice search allowing users to ask long, sophisticated, and nuanced questions.
Focus on subjects rather than keywords.
The term “anti-dandruff shampoo” is more than just a keyword in the previous example. It’s a subject. It might be a subtopic of an article about different types of “shampoos,” or it could be an umbrella subject about dandruff control. In any case, context is crucial.
Focusing on topics eliminates the need to compete for individual keywords. It’s no longer necessary to perform backflips in order to cram irregular, strange, and frequently misspelt keywords into content only to rank for them because they’re popular.
Trying to put “best covid soap toronto” into a sentence as is, for example, is pointless, to say nothing of mind-numbingly tough.
While keyword research is still necessary, it’s more important to understand what themes the user wants to learn about, what topics have already been covered (or haven’t), and what topics to write about that will provide all of the information needed to increase search signals.
To begin, identify a reader's problem, a question they're asking, or a topic they're interested in - one they could be investigating themselves.
Compare and contrast the findings that appear. For instance, look at the many forms of content that already exist on the subject (or how they fail to cover it adequately).
Above all, establish a content aim that is clear to both the reader and the website. Then go over the subject with the user as well as the goal in mind.
Finally, insert related keywords that will occur naturally and smoothly throughout the text, utilising the topic as a guide (rather than a specific term as a goal).
Additional actions can certainly help, but they aren’t required. Select the most frequently searched keywords that fit inside the topic’s umbrella, for example. Then use these keywords in the page’s headings and subheadings, as well as the HTML.
Everything else, including the correct keywords, will fall into place organically if the topic content reflects what the reader is truly interested in and searching for. All that’s left to do from an SEO standpoint is make sure the material is properly formatted.
What matters is the interaction between subjects and material. Some themes are more broad and inclusive than others. Others could be subtopics or connected subjects.
In two ways, a piece of content can cover an umbrella topic. To ensure that it covers the issue thoroughly, it may first divide it down into subtopics on a single page. It could also involve many pieces of material, each of which covers different subtopics that are linked together.
Topical clusters are like wheels, with hubs and spokes, similar to the map of nodes and branches outlined above with entities and the Knowledge Graph.
Keywords were formerly categorised and organised into categories or silos. While this may still be effective for arranging content, it is linear and does not reflect how subjects (and their relationships) work. Consider a mindmap as an example.
Whereas old-school SEO was centred on keywords and their search engine popularity, today’s SEO is based on themes and how valuable they are (to the reader).
The former required authors to develop material first for search engines and then for people. Because the search engine is now the user, it has been not only flipped but also streamlined.
In other words, machine learning algorithms are assisting search engines in becoming more intelligent by allowing them to learn and understand language in the same way that humans do. As a result, writing for search engines is no longer a viable option. It’s pointless.
It’s like attempting to translate something that will inevitably be translated back. As a result, this process is not only unnecessary, but it can also be harmful because information might be lost in translation.
In the end, writing for the user is preferable. Concentrate on making them happy. Provide them with the best possible material as well as the best possible experience while consuming it.
When you write for your target audience, you’re also writing for Google. If you do this, you’ll be sending all the appropriate search signals. Keywords will be used, links will be earned, mentions will be gained, authority will be built, word of mouth will be generated, you will rank well, and traffic will be driven. Naturally.
That’s SEO in the modern era.