DATA: Sheesh, I'm starting to get dozens of spam messages a day now that have nothing but a link to a Google Group. Anyone else out there in the cloud dealing with this? Google, please crack down on the Google Spam Groups!

Labels: google groups, spam

INFO SCIENCE: Technology Review reports:

An ambitious project to create an accurate computer model of the brain has reached an impressive milestone. Scientists in Switzerland working with IBM researchers have shown that their computer simulation of the neocortical column, arguably the most complex part of a mammal's brain, appears to behave like its biological counterpart. By demonstrating that their simulation is realistic, the researchers say, these results suggest that an entire mammal brain could be completely modeled within three years, and a human brain within the next decade.

The article's visualization is fascinating and gives me renewed respect for the people trying to reverse engineer the brain.

Labels: brain, science

DATA:
Over on Ask Edward Tufte there's an interesting thread about "representing scale in concrete, understandable terms." It's a great read.

Something I was told twenty-five years ago (when RAM was a dollar a byte) was that the Bible was 5Mb as unformatted text. There are a million seconds in (roughly) eleven-and-a-half days, and a terabyte of seconds in something around 32,000 years.

However, a good-quality large art book (ignoring file compression) could have half-a-gigabyte of illustrations, and a collection of maps as image files easily larger, so a terabyte would be somewhere around the fine art section of a big-city bookstore, or a reasonably sized personal library in a private house that included maps or illustrated books. "Library of Congress" may only be good guide if you leave out formatting and illustrations.

A terabyte is not big at all. If I could contain all the information I know about one other person in one byte, and their information about me similarly, then two people's knowledge of each other equals two bytes. Three people's knowledge is six bytes (factorial 3). By the time you reach fifteen people you are past a terabyte.

If, rather than a byte, you take a terabyte as the information you know of any person - one percent of a brainful say (estimates of the number of information connections in the brain vary from say a hundred terabytes to a terabyte squared) - and say six billion people on earth each knowing one hundred other people, then a terabyte is to the sum of human knowledge as a single atom is to the square of the number of atoms in the known universe - give or take a few.

UPDATE: After re-reading the last two paragraphs, I realized that the author made a mistake in his math. So, I submitted this post to the thread, but it may not appear since it is moderated.

Response to How big is a phone book, and other ways of illustrating size
Earlier in this thread, Martin Ternouth noted:

"A terabyte is not big at all. If I could contain all the information I know about one other person in one byte, and their information about me similarly, then two people's knowledge of each other equals two bytes. Three people's knowledge is six bytes (factorial 3). By the time you reach fifteen people you are past a terabyte."

However, this isn't the case. If we visualized this example as a graph, each person would be a node, and known information about other people would be an edge between two nodes.

In the provided example, a complete graph (in other words, a connection between each node on the graph -- see http://en.wikipedia.org/wiki/Complete_graph) of 15 nodes, would not be 15! bytes (over a terabyte) as the author states, but would simply be 15(15-1), or 210 bytes. The author appears to have assumed this number would grow at a factorial rate since his example of three nodes satisfies this equation: 3! = 3(3-1), however at 15 this is not the case: 15! > 15(15-1).

Labels: scale, tufte

DATA: Those ubergeeks at Google (can't we go a day without saying or typing Google anymore?) released an interesting paper on observed hard drive failures in their data centers. It's called, aptly enough, Failure Trends in a Large Disk Drive Population. While it doesn't name names and point out which brands are the best and worst -- it does provide a lot of interesting analysis on when the drive is most likely to fail along with the optimal operating performance. Give it a look.

Labels: gear, google, harddrives

INFO VISUALIZATION: I just found a great new blog called Strange Maps. The title says it all -- give it a look!



Astute readers may notice that I've been deeplinking images on my last few posts. For shame! I know -- it's a nasty practice. Unfortunately I'm sans-Photoshop on this computer and I haven't been able to move the images over and format them correctly. I'm going to try and revisit soon from my home computer and update the links.

Labels: blogs, maps

INFO VISUALIZATION: This isn't a perfect visualization -- but it's an interesting attempt at showing the spread of today's religions over Eurasia. I'd love to see a more comprehensive look animated as well some day. Kudos again to Wikipedia.

INFO VISUALIZATION: Well, here I am, back from the dead. :-)

Work, work, work and some of my projects (a new source code community wiki, a place to buy historical stock quote data en masse (still being built), and a place for historical price data for consumer goods (still in the early stages)) have been keeping me from blogging for almost a year.

BUT, here I am, to show you a great new map of the US Interstate system. Enjoy!

INFO VISUALIZATION: Reuters has some illustrative visualizations on birth rates and population growth.



Pop quiz. Biggest pensions in the world + flat population growth =

A. European economic crisis.
B. No worries d00d! Das no problem.
C. European economic crisis.

INFO VISUALIZATION: Two new visualizations for you this morning:

• Cabspotting: An animated map showing the location and speed of San Franciscan cabs.
• World Population TreeMap: Another treemap -- this one using (I think) Treemap 4.0 in a Java Applet.

INFO VISUALIZATION: Here are some charts I created for the New York City Demographics page on Wikipedia. For your viewing pleasure.

Key:

• New York City
• The Bronx
• Brooklyn
• Manhattan
• Queens
• Staten Island

Percentage EDIT: Rate of New York City borough population growth, decade over decade.


New York City population, total and by borough, from 1900 to 2000. Figures in millions.


New York City population, total and by borough, from 1790 to 1890. Figures in millions.


New York City borough population from 1790 to 2000. Figures in millions.


New York City population, total and by borough, from 1790 to 2000. Figures in millions.

DATA: Check out this great paper called Cross-National Correlations of Quantifiable Societal Health with Popular Religiosity and Secularism in the Prosperous Democracies. It shows that religion doesn't have the effect on a nation that many people suspect it has.


Specifically, it looks at crime rates v. religiosity and finds that, at best, the relationship is random and, at worst, religion has the effect of increasing crime rates. (Correlation does not imply causation though. It could be random, or perhaps people turn to religion when the streets around them are crime ridden?)

INFO VISUALIZATION: The debate continues.

DATA:Here's an interesting peice from Morgan Stanley about the current issues brewing in the economy. Again, it's the same things I posted on last time.

• Energy prices up


• National / Personal debt and Trade Deficit up


• Real Estate bubble


• Savings null

Here's the kicker paragraph from the article:

Just prior to the two oil price spikes of the 1970s, discretionary spending by US households had also gone to excess. The GDP share of consumer durables and residential construction -- the latter being a proxy for the discretionary demand for shelter -- was running at peak levels of around 14.5%. In the aftermath of those two earlier energy shocks, discretionary spending collapsed -- with the combined share of consumer durables and homebuilding falling to 11.5% in the mid-1970s and 10.5% in the early 1980s. These were the most severe consumer-led recessions on record in the United States. In the current expansion, discretionary household spending has moved into a similar zone of excess. The combined share of consumer durables and residential construction has averaged 14.3% of GDP over the past year -- virtually identical to peak shares hit just before the two energy-shock-induced consumption collapses of the 1970s. In other words, just as the energy shocks of the 1970s hit US households at a point when their spending behavior had gone to excess, the same is the case in the present climate. Yet unlike those earlier periods, today’s asset-dependent, overly-indebted American consumer is lacking any semblance of a backstop of income-based saving to shore up the downside. It would be one thing if American consumers were committed to defending modest lifestyles. It is another thing altogether in today’s era of excess -- there is much more room and greater urgency for consolidation.

The only investment Dan and I have in non-cash assets right now is investment in international bonds (Templeton Global Income Fund) because I just have a bad feeling about economy as a whole right now.

When is this government really, really going to get serious about energy policy? And, no, the latest pork-fest energy bill doesn't count.

DATA: So, I'm not an economist.

However, I've been reading more and more about the housing bubble. And I've seen it play out in real life; none of my friends are looking to buy houses, we all understand that they are overpriced. Today, I read a very interesting article by the good ol' bear Krugman, called Safe as Houses.

I used to live next door to a Russian émigré. One day he asked me to explain something that puzzled him about his new country. "This place seems very rich," he said, "but I never see anyone making anything. How does the country earn its money?"

The answer, these days, is that we make a living by selling each other houses. Since December 2000 employment in U.S. manufacturing has fallen 17 percent, but membership in the National Association of Realtors has risen 58 percent.

I've thought for sometime, even back in Seattle, that housing prices were unsustainable. Now that I'm in New York, I know they are. (Living in a city with a per-capita income of $22,402 but where the average apartment closing price is now $1,300,000.00 will do that to a person.) Meanwhile savings rates have fallen to zero, interest only mortgages are proliferating, and the twin deficits are rising.

Despite my generally bullish take on the US economy and future in general, I think it may be time to switch into hedge-mode and short a few housing related stocks.

Another Op-ed piece ties the economy to Bush's slowly falling approval ratings:

[P]eople are feeling insecure because they understand that today's economy is built on shaky fundamentals. Average Americans may not sit around fretting about America's outsized budget and trade deficits, and its unprecedented foreign indebtedness. But many of them - as buyers, borrowers and employees - are concerned about the increasingly bubbly housing sector.

The economy's shortcomings are nowhere more obvious than in the job market. Nearly four years into an economic expansion, job growth is still substantially slower than in previous recoveries. Wages for 80 percent of the work force are barely keeping pace with inflation, and aid for the workers hurt by global trade is paltry. Because Mr. Bush fails to acknowledge the lackluster job and wage growth, he fails to respond appropriately. The administration's insistence that the economy is getting better all the time - a stance that is based on statistical aggregates that are often divorced from individuals' actual experience - only intensifies the anxiety that people feel.

DATA: Straphangers just released their latest NYC Subway reportcard. I guess I've got a good mix with my commute using the R or the V (two of the shitty lines) and sometimes the N or the W (even worse), however I always transfer to the 6 at 59th Street or 51st Street, which gives me a taste of the good life every morn' on the city's best train.

Gothamist has a good conspiracy theory:

N riders aren't surprised their line sucks; Gothamist likes how one person the Post interviewed was "sweating profusely" as he waited for a train, because we've noticed that many N stations (Times Square, Herald Square, Union Square) are like saunas. If Gothamist were a conspiracy buff, we'd say the 6 is the best line because Mayor Bloomberg usually takes it from his townhouse, but we're not anything like that. But perhaps we'll change our nickname for the N from the "Never" to the "Nasty" train.