{"id":231,"date":"2018-06-14T19:05:00","date_gmt":"2018-06-14T19:05:00","guid":{"rendered":"https:\/\/pressbooks.ccconline.org\/bus3060\/chapter\/ch14-2\/"},"modified":"2026-02-17T19:23:57","modified_gmt":"2026-02-17T19:23:57","slug":"ch14-2","status":"publish","type":"chapter","link":"https:\/\/pressbooks.ccconline.org\/bus3060\/chapter\/ch14-2\/","title":{"raw":"14.2 Understanding Search","rendered":"14.2 Understanding Search"},"content":{"raw":"
\r\n
\r\n
\r\n
\r\n

Learning Objectives<\/span><\/p>\r\n\r\n<\/header>\r\n

\r\n

After studying this section you should be able to do the following:<\/p>\r\n\r\n

    \r\n \t
  1. Understand the mechanics of search, including how Google indexes the Web and ranks its organic search results.<\/li>\r\n \t
  2. Examine the infrastructure that powers Google and how its scale and complexity offer key competitive advantages.<\/li>\r\n<\/ol>\r\n<\/div>\r\n<\/div>\r\n \r\n\r\n<\/div>\r\n

    Before diving into how the firm makes money, let\u2019s first understand how Google\u2019s core service, search, works.<\/p>\r\n

    Perform a search (or query<\/a><\/span>) on Google or another search engine, and the results you\u2019ll see are referred to by industry professionals as organic or natural search<\/a><\/span>. Search engines use different algorithms for determining the order of organic search results, but at Google the method is called PageRank<\/a><\/span> (a bit of a play on words, it ranks Web pages, and was initially developed by Google cofounder Larry Page). Google does not accept money for placement of links in organic search results. Instead, PageRank results are a kind of popularity contest. Web pages that have more pages linking to them<\/em> are ranked higher.<\/p>\r\n\r\n

    \r\n

    Figure 14.4<\/span><\/p>\r\n

    \r\n\"The\r\n<\/a><\/p>\r\n

    The query for \u201cToyota Prius\u201d triggers organic search results, flanked top and right by sponsored link advertisements.<\/p>\r\n\r\n<\/div>\r\n

    The process of improving a page\u2019s organic search results is often referred to as search engine optimization (SEO)<\/a><\/span>. SEO has become a critical function for many marketing organizations since if a firm\u2019s pages aren\u2019t near the top of search results, customers may never discover its site.<\/p>\r\n

    Google is a bit vague about the specifics of precisely how PageRank has been refined, in part because many have tried to game the system. In addition to in-bound links, Google\u2019s organic search results also consider some two hundred other signals, and the firm\u2019s search quality team is relentlessly analyzing user behavior for clues on how to tweak the system to improve accuracy (Levy, 2010). The less scrupulous have tried creating a series of bogus Web sites, all linking back to the pages they\u2019re trying to promote (this is called link fraud<\/a><\/span>, and Google actively works to uncover and shut down such efforts). We do know that links from some Web sites carry more weight than others. For example, links from Web sites that Google deems as \u201cinfluential,\u201d and links from most \u201c.edu\u201d Web sites, have greater weight in PageRank calculations than links from run-of-the-mill \u201c.com\u201d sites.<\/p>\r\n\r\n

    \r\n
    \r\n

    Spiders and Bots and Crawlers\u2014Oh My!<\/h4>\r\n

    When performing a search via Google or another search engine, you\u2019re not actually searching the Web. What really happens is that the major search engines make what amounts to a copy<\/em> of the Web, storing and indexing the text of online documents on their own computers. Google\u2019s index considers over one trillion URLs (Wright, 2009). The upper right-hand corner of a Google query shows you just how fast a search can take place (in the example above, rankings from over eight million results containing the term \u201cToyota Prius\u201d were delivered in less than two tenths of a second).<\/p>\r\n

    To create these massive indexes, search firms use software to crawl the Web and uncover as much information as they can find. This software is referred to by several different names\u2014software robots, spiders, Web crawlers<\/a><\/span>\u2014but they all pretty much work the same way. In order to make its Web sites visible, every online firm provides a list of all of the public, named servers on its network, known as domain name service (DNS)<\/a><\/span> listings. For example, Yahoo! has different servers that can be found at http:\/\/www.yahoo.com, sports.yahoo.com, weather.yahoo.com, finance.yahoo.com, and so on. Spiders start at the first page on every public server and follow every available link, traversing a Web site until all pages are uncovered.<\/p>\r\n

    Google will crawl frequently updated sites, like those run by news organizations, as often as several times an hour. Rarely updated, less popular sites might only be reindexed every few days. The method used to crawl the Web also means that if a Web site isn\u2019t the first page on a public server, or isn\u2019t linked to from another public page, then it\u2019ll never be found1<\/sup>. Also note that each search engine also offers a page where you can submit your Web site for indexing.<\/p>\r\n

    While search engines show you what they\u2019ve found on their copy<\/em> of the Web\u2019s contents; clicking a search result will direct you to the actual Web site, not the copy. But sometimes you\u2019ll click a result only to find that the Web site doesn\u2019t match what the search engine found. This happens if a Web site was updated before your search engine had a chance to reindex the changes. In most cases you can still pull up the search engine\u2019s copy of the page. Just click the \u201cCached\u201d link below the result cache<\/a><\/span>\u00a0refers to a temporary storage space used to speed computing tasks).<\/p>\r\n

    But what if you want the content on your Web site to remain off limits to search engine indexing and caching? Organizations have created a set of standards to stop the spider crawl, and all commercial search engines have agreed to respect these standards. One way is to put a line of HTML code<\/em> invisibly embedded in a Web site that tells all software robots to stop indexing a page, stop following links on the page, or stop offering old page archives in a cache. Users don\u2019t see this code, but commercial Web crawlers do. For those familiar with HTML code (the language used to describe a Web site), the command to stop Web crawlers from indexing a page, following links, and listing archives of cached pages looks like this:<\/p>\r\n

    \u2329META NAME=\u201cROBOTS\u201d CONTENT=\u201cNOINDEX, NOFOLLOW, NOARCHIVE\u201d\u232a<\/p>\r\n

    There are other techniques to keep the spiders out, too. Web site administrators can add a special file (called robots.txt) that provides similar instructions on how indexing software should treat the Web site. And a lot of content lies inside the \u201cdark Web<\/a><\/span>,\u201d either behind corporate firewalls or inaccessible to those without a user account\u2014think of private Facebook updates no one can see unless they\u2019re your friend\u2014all of that is out of Google\u2019s reach.<\/p>\r\n\r\n<\/div>\r\n \r\n\r\n<\/div>\r\n

    \r\n
    \r\n
    \r\n
    \r\n
    \r\n

    What\u2019s It Take to Run This Thing?<\/h4>\r\n

    Sergey Brin and Larry Page started Google with just four scavenged computers (Liedtke, 2008). But in a decade, the infrastructure used to power the search sovereign has ballooned to the point where it is now the largest of its kind in the world (Carr, 2006). Google doesn\u2019t disclose the number of servers it uses, but by some estimates, it runs over 1.4 million servers in over a dozen so-called server farms<\/a><\/span> worldwide (Katz, 2009). In 2008, the firm spent $2.18 billion on capital expenditures, with data centers, servers, and networking equipment eating up the bulk of this cost2<\/sup>. Building massive server farms to index the ever-growing Web is now the cost of admission for any firm wanting to compete in the search market. This is clearly no longer a game for two graduate students working out of a garage.<\/p>\r\n

    Google\u2019s Container Data Center<\/p>\r\n\r\n<\/div>\r\n

    Take a virtual tour of one of Google\u2019s data centers.<\/p>\r\n\r\n<\/div>\r\n

    The size of this investment not only creates a barrier to entry, it influences industry profitability, with market-leader Google enjoying huge economies of scale. Firms may spend the same amount to build server farms, but if Google has nearly 70 percent of this market (and growing) while Microsoft\u2019s search draws less than one-seventh the traffic, which do you think enjoys the better return on investment?<\/p>\r\n

    The hardware components that power Google aren\u2019t particularly special. In most cases the firm uses the kind of Intel or AMD processors, low-end hard drives, and RAM chips that you\u2019d find in a desktop PC. These components are housed in rack-mounted servers about 3.5 inches thick, with each server containing two processors, eight memory slots, and two hard drives.<\/p>\r\n

    In some cases, Google mounts racks of these servers inside standard-sized shipping containers, each with as many as 1,160 servers per box (Shankland, 2009). A given data center may have dozens of these server-filled containers all linked together. Redundancy is the name of the game. Google assumes individual components will regularly fail, but no single failure should interrupt the firm\u2019s operations (making the setup what geeks call fault-tolerant<\/a><\/span>). If something breaks, a technician can easily swap it out with a replacement.<\/p>\r\n

    Each server farm layout has also been carefully designed with an emphasis on lowering power consumption and cooling requirements. And the firm\u2019s custom software (much of it built upon open source products) allows all this equipment to operate as the world\u2019s largest grid computer.<\/p>\r\n

    Web search is a task particularly well suited for the massively parallel architecture used by Google and its rivals. For an analogy of how this works, imagine that working alone, you need try to find a particular phrase in a hundred-page document (that\u2019s a one server effort). Next, imagine that you can distribute the task across five thousand people, giving each of them a separate sentence to scan (that\u2019s the multi-server grid). This difference gives you a sense of how search firms use massive numbers of servers and the divide-and-conquer approach of grid computing to quickly find the needles you\u2019re searching for within the Web\u2019s haystack. (For more on grid computing, see Chapter 5 \u201cMoore\u2019s Law: Fast, Cheap Computing and What It Means for the Manager<\/strong>\u201d, and for more on server farms, see Chapter 10 \u201cSoftware in Flux: Partly Cloudy and Sometimes Free<\/strong>\u201d.)<\/p>\r\n\r\n

    \r\n
    \r\n

    Figure 14.5<\/span><\/p>\r\n

    \r\n\"The\r\n<\/a><\/p>\r\n

    The Google Search Appliance is a hardware product that firms can purchase in order to run Google search technology within the privacy and security of an organization\u2019s firewall.<\/p>\r\n\r\n<\/div>\r\n<\/div>\r\n

    Google will even sell you a bit of its technology so that you can run your own little Google in-house without sharing documents with the rest of the world. Google\u2019s line of search appliances are rack-mounted servers that can index documents within a corporation\u2019s Web site, even specifying password and security access on a per-document basis. Selling hardware isn\u2019t a large business for Google, and other vendors offer similar solutions, but search appliances can be vital tools for law firms, investment banks, and other document-rich organizations.<\/p>\r\n\r\n<\/div>\r\n

    \r\n

    <\/h4>\r\n<\/div>\r\n<\/div>\r\n \r\n
    \r\n
    \r\n
    \r\n
    \r\n

    Trendspotting with Google<\/h4>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n
    \r\n

    Google not only gives you search results, it lets you see aggregate trends in what its users are searching for, and this can yield powerful insights. For example, by tracking search trends for flu symptoms, Google\u2019s Flu Trends Web site can pinpoint outbreaks one to two weeks faster than the Centers for Disease Control and Prevention (Bruce, 2009). Want to go beyond the flu? Google\u2019s Trends, and Insights for Search services allow anyone to explore search trends, breaking out the analysis by region, category (image, news, product), date, and other criteria. Savvy managers can leverage these and similar tools for competitive analysis, comparing a firm, its brands, and its rivals.<\/p>\r\n\r\n

    \r\n
    \r\n

    Figure 14.6<\/span><\/p>\r\n

    \r\n\"Google\r\n<\/a><\/p>\r\n

    Google Insights for Search can be a useful tool for competitive analysis and trend discovery. The chart above shows a comparison (over a twelve-month period, and geographically) of search interest in the terms Wii, Playstation, and Xbox.<\/p>\r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n \r\n\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n

    \r\n
    \r\n

    Key Takeaways<\/span><\/p>\r\n\r\n<\/header>\r\n

    \r\n