In the past, search engine optimization (SEO) was an exercise in keyword stuffing. What this means is that in the beginning stages of the processes that later became known as “search engine optimization,” the search engines themselves were quite crude in the factors that actually made up the rankings. The most gratuitous examples of this is that one of the major factors that used to decide position within SERPs of a website was the sheer number of times that a term was utilized within the text of the site. Google would count the number of times that a specific term was used within the crawled pages and put that website at the top positions for searches for that specific term. When it was discovered that this is the process that Google was using, people who wanted their websites to be at the top of the results for a specific term simple used that term as much as possible within the site. This became problematic, as natural speech and writing patterns are completely opposite, and would not be very usable if a word or phrase was simply repeated over and over again. The text might trick the search engine into placing the site at the top of results, but the text on the site itself would do little to help the person who performed the search to find out the information they were searching for. This lead to “offpaging” and “whitetexting” which are simple code tricks to hide the repeated words, either positioning them in a place that is not directly displayed when the site loads on a computer, or by making the repeated text be the same color as the background, so as not to display it to readers. These tricks once again did the job of getting a website to the first page of a result, but provided poor results for users. People must understand that Google’s motivation is to provide the results that satisfy customers the most, because their business is providing information that satisfies customers. If they begin providing results that are not satisfactory, they will lose customers to competing search engines.
Google moved quickly away from using the simple analysis of text as the only ranking factor, and began experimenting with judging a website on the actual value it provides to human beings. This was accomplished using a “crowd sourcing” model, where the theory that if something is important to people, they will bookmark it in some way in mass. Google found that if information is important on a subject, then other websites that are providing information on that same subject would naturally link to it, or specifically to that content. The combination of text analysis and inbound links provided good results for searches for a period of time, until search engine optimization companies realized that they could manipulate the inbound link profile of a website as well. Search engine optimization became an exercise in not only putting keywords into a website but also creating links into that website from other websites. This was typically accomplished through the use of blogs, where users had the ability to comment directly on a piece of content. The process of building thousands of “signature links” was the easiest method of accomplishing this goal, essentially using software to crawl the internet seeking blogs that would allow a comment to be posted along with a link to a website in the signature field of the comment poster. By placing the information into the software, a search engine optimization company could create millions of links to a website in a short amount of time, the majority of them being placed on blogs that had nothing to do with the subject. Crackdown by Google on this method lead to the actual creation of thousands of websites using free services, essentially due to the fact that if Google can recognize that the signature link is the way the SEOs are manipulating things then to devalue those types of links. Google placed more value on actual blog articles that included links instead of signatures, which lead to the process of “blog farming.” This method of manipulation was the utilization of massive collections of blogs that had been created and were willing to sell space within their posts to people who needed links to a website. They would generally require a minimum amount of text, a link and a payment of a certain amount of money to post the content within their space. While this was quite effective for a long period of time, it was destroyed as a process when Google suddenly identified the major blog networks and not only devalued the links coming from them but actually penalized websites who used them.
Fast forward to today, and the technology that is being employed by Google and other search engines in order to provide the results that you see when you search for something. While inbound links and text on a website are still the major ranking factors, the analysis of these two factors has become incredibly effective at weeding out those who are simply trying to manipulate results and those who are legitimately trying to provide information. High tech analysis techniques using AI (artificial intelligence) and machine learning are intersecting with crowd sourcing of satisfaction to form a state where search engine optimization is almost impossible to mechanize in a natural way. Essentially, being “natural” in the presentation of the text and links that are being manipulated is the most important aspect of SEO, not only to increase rankings but also avoid penalties. Google has analyzed and interpreted these factors to identify what the patterns of those who are attempting to simply gain higher positions look like, and devalue anything created in these patterns. As an example, the term “content is king” is important because it reflects the shift of Google’s analysis of the importance of a website away from the sheer volume of inbound links and onto the text within a site as the most important aspect. This is because the machines of Google are becoming better and better at analyzing that text in search of patterns that will illustrate that it is actually manipulation and not presentation of good information. If you simply need to write a bunch of text using a keyword and you will rank for that keyword, then the pattern of those who want to manipulate things is to do as much as possible with the least amount of work. This will probably mean large amounts of pieces of content within a website that utilize a minimum amount of words, whatever that minimum is perceived as. You will see lots of articles in websites now that are around 400 to 500 words, simply because that is the perceived minimum that Google is looking for in order to not raise a red flag that this content is simply for manipulation. Google now prefers longer articles, statistically showing that articles that are over 1500 words long will provide better value than shorter ones in terms of ranking. The reason this is effective is that it is easy to create short articles that have very little information for a cheap price and a small amount of time spent writing them. It is far more difficult to create a 1500 word piece, or one that is even longer.
Google has now employed a method of understanding the natural utilization of specific terms within content in order to identify “keyword stuffing.” What this means is that Google analyzes the content, counts the words and compares it to an understood pattern of how many times a specific term would be utilized within an article of that length. It also looks for known synonyms, or alternative versions of that phrase or term, which is typical in a natural pattern of writing. Few people concerned about quality of information are going to use the same term over and over, and will instead use natural variants of that term when it is appropriate. Google now understands that a variation of a term still counts towards the amount of times that the specific term is utilized, leaning towards natural forms of writing over keyword stuffed articles. TF*IDF stands for “term frequency–inverse document frequency”. This means that Google is looking at your content and analyzing the number of times a specific term or a variation of that term is utilized, compared against the total number of words within that document. It then compares it against inverse document frequency, which measures how common a term is across all known documents on the internet. This way, Google can crowd source an understanding of typical writing patterns on a subject in order to determine if content is being written for information or for rankings. Documents which are created in order to provide valuable information on a subject are given priority and those created to manipulate rankings are devalued.
So where does this leave us with regards to SEO? There are those who now say that SEO as a process is dead, and these new technologies make the process have no value. I personally do not believe this, due to the fact that if there were no process that can manipulate rankings, then rankings would be random. SEO as a process is now based upon creating the content that provides valuable information on a subject, then letting the internet know the content is out there. Instead of trying to create links to content in a mechanized way, concentration on creating valuable content is the process, allowing the linking, social media mentions and shares, and bookmarks to follow naturally the way Google wants it to. This process becomes “search engine optimization” when it is utilized specifically to create better rankings for a website. For example, if you want to get a dentist’s website to rank higher, instead of trying to use the word “dentist” over and over again in short articles on the site, it is far more effective to create content explaining the processes, techniques, and methods that the dentist utilizes in order to treat patients. Google understands that this website is important for dentistry, places it into results for searches for “dentist” and then potential customers find the site when they Google that term. This is the modern version of SEO, essentially looking at what Google wants and then providing it for a client. In a nutshell, Google wants a website to provide the best information on a subject, and the modern SEO’s job is to make that website be the best on that subject.