[Developer]

How Search Engines Operate


Different Search Engines follow different algorithms and criteria to optimize websites. But the basics of these algorithms and criteria do not differ too much. In the beginning search engines give importance to Meta tags but now most of the search engines don't take it an important element in ranking a website. Only a few search engines give it importance in the ranking of a website.

Search engines are continuously developing their technologies to read the content of a website. Search engines crawl the websites more deeply and return relevant results to the user. If you optimize your website according the crawl and read method of search engines then you can get thousands of visitors and a lot of attention. On the other hand if you are not following these methods and algorithms then your website can be escaped or bury deep in the search results of a search engine where the chances of getting visitors is minimum. SEO techniques help us to boost rankings on search engines and to place content where searchers will more readily see it. SEO is becoming increasingly important in the world of competition to get advantage. When a user tries to find some kind of information by using search engine then search engine performs a list of operations to provide relevant web results to the user. A user finds its required results in fractions of a second but the search engine undergoes many calculations and work to provide these required results.

Crawling Web

A web crawler / web spider / web robot is a program or automated script that uses the hyperlink structure of the web to crawl the pages and contents in a methodical, automated manner that make up the World Wide Web. There is an estimate that search engines have crawled between 8 and 10 billion web pages out of 20 billion web pages.

The process is called web crawling or spidering. Web crawlers create a copy of all the visited pages for later usage or processing by the search engines to make the search faster.

There are some situations in which web crawling is very difficult:

• When there is large volume
• When the rate of change is fast
• When the page generation is dynamic

Indexing Documents

After crawling, search engines index contents of the websites and store them in a huge database of documents that make up search engines index. This index is managed so well that requests can be fulfilled after searching and sorting billions of documents in fractions of a second. This indexing saves a lot of time of a user in matching the query with the indexed documents and provides him / her accurate results with respect to his / her query.

Processing Queries

When a user sends a request to search engine for information the search engine retrieves all the documents from its index that match the request or search. The match of search word or query depends on the way a user types it. For example if a user searches for internet marketing magazine at Google, it returns 8,780,000 results, but if the user searches the same phrase in quotes "internet marketing magazine".




In the first way the search engine used the "find all" mode. Google returned all pages which had the words "internet", "marketing", "magazine". But in the second search only those documents were returned which had the exact phrase "internet marketing magazine. With the help of other advanced operators, the search engine results what it considers to be the best match according to the query?

Ranking Results

After matching the query with the search engine indexed documents, it prepares the results on the base of the algorithm used by the search engine. The algorithm used by the search engine makes calculations on each of the results to find which page is the most relevant to the query. Then the final result is compiled by sorting them on results pages in order from most relevant to the least relevant. From these search results the user can make selection according to his/ her choices. The process of search engine is not lengthy, because these search engines are run on some of the most powerful computers in the world. They use the complex and process intensive computers which manage millions of calculations in each second and provide results to the users after sorting and implementing algorithm to find out the most relevant results.

[roddick]

Basically, search engines work by crawling, indexing, processing queries, and ranking results. Your hope as a small web business owner is to get into the top 30 ranking results of the big three search engines: Google, Yahoo!, and MSN. The coveted, cream of the crop rankings are those which fall on the first page of the search engine results in the top ten positions. And, because Google enjoys the biggest market share, most small businesses see most benefits from ranking well in Google. Google is the main search engine on which we concentrate.

[Lumber Joe]

The most sophisticated thing is ranking. Crawling and indexing is simple, even small SE do that, but ranking (relevant ranking I mean) is really a complicated task.