Mikhail Simkin and Vwani Roychowdhury, electrical engineers from both the University of California Los Angeles and NetSeer.com, have constructed a model of the Web using the traffic statistics of just three Web pages:
http://reverent.org/true_art_or_fake_art.html ,
http://reverent.org/sounds_like_faulkner.html ,
http://ecclesiastes911.net/disumbrated_art.html . (Traffic patterns from a dozen other Web pages the researchers studied were very similar.) Using several years of data from these three pages, the researchers show how the Internet overall reaches a self-organized critical (SOC) state with long-lasting traffic.
“One of the main implications of our findings is that traffic and [the corresponding] fame is a prolonged phenomenon instead of a one-time fling, and recurs in a spasmodic fashion,” Roychowdhury told
PhysOrg.com.
Most of the time, traffic to any single Web page is relatively low and steady, where visitors come from search engines, Web directories, online encyclopedias, and other constant sources. But these long periods of low traffic are interrupted by bursts of heavy traffic that follow a power law, usually the effect of numerous blog entries linking to those pages.
The researchers use a branching model to describe the probability and extent of these bursts. Basically, there’s a certain probability that a viewer will post a blog entry with a link to that Web page, and then a certain number of viewers who will visit the Web page via the blogger’s link. The product of these two variables determines whether or not a Web page will reach the critical value of 1, which determines if the branch keeps growing or dead-ends.
“A system is in a critical state if a single movement in an individual constituent element leads, on the average, to the movement of precisely one other element in the system,” Roychowdhury explained.
If a system is in a super- or sub-critical state, movement of one element leads, on average, to the movement of either more or less than one other element, respectively. That means that a signal generated in a super-critical system should increase forever, while a signal in a sub-critical system eventually dies out.
“But in a critical system, something very interesting happens,” Roychowdhury said. “Almost all signal cascades will die out, but some of them can last for a long time and can cover a large area. Clearly, sub- and super-critical systems are not that interesting unless we want a system that is either not that responsive or a system that explodes at the slightest provocation. Critical systems, on the other hand, allow for a responsive system to exist without it being blown apart. Many physical systems naturally gravitate towards a critical state, and this phenomenon is termed SOC.”
As the researchers explain, competition for viewers and links is a driving force of the Web, and this competition pushes the entire Web into an SOC state. Based on their data, the researchers determined the values for the two variables above for the “true art or fake art” site that closely produce its traffic patterns: they found that its link probability of 0.01 and referral number of 95 visitors per link results in a slightly sub-critical value of 0.95 for that particular Web page.
But since some Web pages are more interesting than others, some pages will achieve the critical value of 1 or even surpass it.
“To explain how the Web evolves into the SOC state, we need to use the concept of Darwinian fitness, which is a scientific measure of digital fangs and claws that help the Web page to fight for links with its competitors,” Simkin said. “The success in this competition depends not only on the Web page's own fitness, but also on the average fitness of other pages currently discussed in the blogosphere, with which our Web page competes.”
If this average is low, Simkin explained, then the fittest papers are super-critical. This means that, with time, they increase their share of the blogosphere. But in turn, this leads to the increase of the average fitness. The process continues until the fittest pages become exactly critical.
“One finding that is important for Webmasters is that our work disproves the so-called fifteen minutes of fame paradigm, according to which things can get popular soon after release and quickly become forgotten,” Simkin said. “One, of course, knows that this paradigm is manifestly wrong for immortal classics. However, our work shows it to be wrong not only for great creations, but for anything which is of any intrinsic (not created by advertisement) merit.”
The researchers found that the traffic to a Web page with fixed content can persist for at least several years.
“So one should not hurry to delete old Web pages,” Simkin said. “When there is enough – say a year – of access statistics, our model can be used to infer a page's fitness and predict the average volume and fluctuations of future traffic.”
Roychowdhury is a cofounder and Simkin a consultant for a start-up company called NetSeer.com that focuses on next-generation Internet advertising. The company utilizes similar physics-based modeling of the Web, though not the direct results of the present study.
The researchers add that the Web is just one of many complex systems that exhibit self-organized criticality, with other examples including evolutionary patterns, earthquakes, and citations in research papers. They suggest that their model could also be used to explain the spreading of cultural elements, like movies, books, and fashion styles.
More information: Simkin, M. V. and Roychowdhury, V. P. “A theory of web traffic.”
Europhysics Letters, 82 (2008) 28006.
Copyright 2008 PhysOrg.com.
All rights reserved. This material may not be published, broadcast, rewritten or redistributed in whole or part without the express written permission of PhysOrg.com.
Related stories:
Chumby puts a host of widgets at your fingertips
Never let it be said that I don't listen to readers. In April, after I favorably reviewed a wireless product that elegantly displayed baseball scores, Don wrote: "How could you recommend that, especially at the price of $200 for a unitasker? This isn't worth $20 for what it does. What a joke. Please check out Chumby for a slightly cheaper and much more functional alternative."
Probing Question: What is colorblindness?
Midnight Blue, Burnt Orange, Aquamarine. Since 1903, Crayola crayons -- with their fanciful names and hundreds of hues -- have introduced generations of American children to the nuanced beauty of the color spectrum. Imagine the public's surprise when Crayola's senior crayon maker, Emerson Moser, who molded a record-breaking 1.4 billion crayons in his 37-year career, announced upon his retirement that he was color blind.
MIT develops lecture search engine to aid students
Imagine you are taking an introductory biology course. You're studying for an exam and realize it would be helpful to revisit the professor's explanation of RNA interference. Fortunately for you, a digital recording of the lecture is online, but the 10-minute explanation you want is buried in a 90-minute lecture you don't have time to watch.
Kamchatka volcano blows its top
Klyuchevskoy (pronounced Kloo-shef-skoy), a stratovolcano located in the north central region of the Kamchatka Peninsula, is blasting ash up to 32,000 feet in the air, and has diverted air traffic headed toward the Far East. This is the largest eruption to occur in the North Pacific in a decade, and is providing students at the University of Alaska Fairbanks a unique opportunity to collaborate with scientists, as well as state and federal agencies.
IronPort Revamps Security Monitoring Site
IronPort Systems has revised its Internet traffic monitoring Web site, a resource for IT staffers searching for a real-time view into security threats.
LiveOffice Launches Web-based Admin for E-discovery
In new hosted e-mail management service, security and compliance features are provided by Secure Computing's IronMail to go with LiveOffice's search and archiving capabilities.
Mighty Nanofibers Could Mean Stronger, Lighter Materials
Bigger may be better, but tinier is stronger. So say scientists at the Technion-Israel Institute of Technology, who have shown that tiny polymer nanofibers become much stronger when their diameters shrink below a certain size. Their research, published in the January issue of
Nature Nanotechnology, could make possible stronger fabrics that use less material.
Study shows internet to be resilient against terror attack
Researchers have simulated what would happen to Internet reliability in the United States if terrorists were able to knock out various physical components of the network.
