Net Neutrality.  Up to now, the conventional definition of the concept has been that internet service providers shall be prohibited from “blocking or slowing content from some applications or companies” (as quoted from a recent NetworkWorld article).  Arguably, the definitive infraction against this particular notion of Net Neutrality was Comcast’s recent ‘managing’ of Bit Torrent traffic via the insertion of spurious connection reset packets.

However, the whole issue the issue of Net Neutrality (at the last mile between the ISP and the consumer, at least) is rapidly becoming a moot point: in preparation for the expected explosion of demand for longer-form video over IP, most major carriers are now scrambling to assemble and/or acquire proprietary content delivery networks (CDN)s to avoid the ever more congested and unpredictable system of routers out there in the public cloud (a recent post about just what Google, Microsoft, and Verizon are up to can be found here).

So while your neighborhood ISP might maintain a commitment to Net Neutrality itself, the real action is well upstream, as major corporations join already established CDN players such as Akamai, Edge Networks, and Yahoo’s Cloudfront to distribute and/or cache digital media content out along the edge of the cloud, in effect forming competing private mini-clouds to minimize the role of the public internet itself.

Put another way, in the purest sense of the term, Net Neutrality has already become something of an anachronism – not due to any localized slowing down of unfavored packets at the ISP level, but due to a globalized speeding up of favored packets on CDNs, before they ever reach the ISP.   A recent Wall Street Journal article touches on just this nuanced distinction: according to Google, their recent proprietary internet/CDN initiatives “do not rely on the carrier’s unilateral control over the last-mile connections to consumers, and also do not involve discriminatory intent“ - and even the independent public interest organization Public Knowledge (whose directors include internet academic and Obama advisor Lawrence Lessig) now maintains that “caching in no way is a part of the Net Neutrality issue.”

I’m of the opinion there’s considerably more gray area here.  But no matter - since the public internet will simply not scale to meet the anticipated bandwidth demand once short-tail (mainstream) premium digital media over IP becomes widespread, both carriers and content owners will increasingly invest in proprietary content delivery networks - and as consumers buy into the mass-market internet video offerings made possible by these high-performance CDNs, the very concept of Net Neutrality will seem increasingly quaint - and the “internet” as a whole will come to resemble the American health care system: multi-tiered and largely privatized.

So to the extent long-form video over IP ultimately enjoys widespread mass-market success,  the innocent ideal of a truly egalitarian and fundamentally neutral internet is destined to end, no matter what your local ISP’s policies are.

Don’t shoot the messenger…  

One of the most daunting technological challenges we face is scaling up this old internet of ours to meet the bourgeoning consumer demand for bandwidth-intensive applications such as streaming media, telecommuting, and cloud computing - and as internet video demand moves from short-form/long-tail/low quality content to long-form/short-tail/high quality content (i.e. from YouTube to Hulu to movies), internet video only becomes a larger and larger part of this equation.

Just a heads up: as of now, the public internet will cease to be able to meet bandwidth demand by 2012, according to an excellent study by research firm Nemertes (if you read just one of the external sources linked to from this blog, make it this one).  Citing independent research from the University of Calgary, the study finds that the three largest US content providers (Microsoft, Google and Yahoo) have already “built out dedicated infrastructures” in advance of just such as scenario - in Google’s case, as evidenced by their recent pursuit of ‘dark’ (unused) transcontinental fiber.

In short, a trend towards content providers “investing in technologies to accelerate traffic to their sites ahead of that on the regular Internet” over higher-performing paid or private “overlay” networks.  Put another way, the egalitarian net-neutral internet we of today will become just the lowest rung of a multi-tiered system composed of competing proprietary networks – nothing less than the commercialization albeit by technical necessity) of the internet.  “The oligarchy, in other words, is devolving into individual city-states” is how the Nemertes study puts it.

So that’s what’s up on the content provider side – what about the service provider?

It turns out ISP industry recognizes the trend as well, and moves are well underway.  Because they lack the resources of a Google or Microsoft to implement their own parallel physical internet backbone data links, proprietary  CDNs (content delivery networks) are the direction they’re going in instead.  What’s a CDN? Through both network optimization/caching software and the brute-force deployment of multiple content-caching servers placed at strategically positioned and geographically diverse locations along the edge of the internet, CDNs are networks designed to circumvent the increasingly messy core of the publicly routed internet to provide the higher bandwidth, lower latency and increased scalability required to meet the challenges of the future.  The third-party wholesale CDN market is relatively mature (industry leader Akamai now maintains over 34,000 servers located in 70 countries), but several large ISPs have recently opted to ‘roll their own’: in June 2008, AT&T announced it was building out a CDN of its own using  software licensed by several smaller firms (ExtendMedia, Qumu and Stratacache), and just last week at a conference, I was handed a press release from Verizon announcing a CDN partnership with UK CDN firm Velocix.

Why have two of the largest US carriers now decided to buck the outsourcing trend and create their own CDNs, especially during such challenging economic times?  There are several reasons:

• While most residential ISP customers are on (ostensibly) all-you-can-eat plans, CDN usage is metered – so while the ISP’s per-user revenue has remained largely static, their costs from the 3rd party CDNs they’re currently contracting with have been steadily going up.  This makes the prospect of cutting out the middleman increasingly attractive.
• On a technical level, the synergy of an integrated CDN/last mile solution offers potential performance advantages an external wholesaled CDN would have difficulty matching – and as the last mile becomes faster due to increased presence of fiber (i.e. FIOS) and/or Docsis3.0, the integration of the CDN with the last mile makes even more sense.
• To the extent an ISP is able to build a demonstrably better mousetrap on their own, the quality of that company’s proprietary CDN could well become a primary competitive differentiator driving subscription growth – especially if, as expected, long-form internet video usage continues to grow.
• Unlike AT&T, Verizon is planning to leverage the price/performance advantages of their new proprietary CDN on the content owner side as well, and has already contracted directly with Starz Entertainment to offer Starz content to Verizon customers (in this way, the in-house CDN could very well resuscitate the largely failed ‘walled-garden’ model).

Are 3rd party CDN wholesalers Akamai and Limelight losing sleep?  Probably not – deploying an effective CDN is an incredibly huge undertaking, and as such will be a realistic option only for the AT&Ts, and Verizons of the world.  But for the combined residential CDN/ISP, does being in both businesses concurrently present some interesting antitrust/conflict of interest issues?

Regardless of how it plays out between content providers’ pivate backbones and service providers’ private CDNs, what is clear is that the landscape will likely look profoundly different in five years – the internet as we know it could well become the equivalent of the public post office - while the equivalent of a parrallel  higher-performing ‘Fedex’ internet emerges, for the sole purpose of getting you that HD internet video stream or workplace desktop session you require.

On a purely technical level, it remains an exciting future – but will the incorporation of the internet (and the de facto end of net neutrality) happen at the expense of innovation?

On Wednesday morning last week, a few more people than usual were probably interested in picking up the morning paper.  However, the New York Times was nowhere to be found at many new stands here in Manhattan.   The new parallel online economy was making itself felt - papers were bought in bulk throughout the city long before rush hour, and now that historic front page can be yours, suitable for framing - Buy It Now for only $99.99.

Similarly, a CNN story this morning aired regarding the online sale of Obama inauguration tickets ,  which are intended to be distributed free of charge through members of the incoming congress (currently, VIP tickets are going for the low five figures).

These two end-runs around the intended distribution mechanism put me in mind of the situation my wife faced earlier this summer when attempting to surprise me with birthday tickets to the much in-demand iMax showing of The Dark Knight during the first week of its release (I know: best. wife. ever.)  It turns out she had been trying to buy them for days via AOL’s Moviefone service , but strangely the tickets (released in maximum lots of 5) were constantly sold out: regardless of how often (or when) she checked, there were just never any tickets available - for that particular movie, the AOL site was essentially broken.  She then checked craigslist and found out where all those tickets went: a thriving business in (drastically marked-up) iMax Batman tickets.  ‘Entrepreneurs’ were grabbing all tickets within seconds of release and were reselling them on craigslist for 3 to 4 times the original price - depending on your point of view, either a complete hijacking of the system by a unnecessary additional layer of middleman, or a shining example of capitalism at its finest.

To someone who’s just paid $80 for two tickets to a movie, it seems more the former - although of course, with the power and convenience of the internet comes the occasional disruptive gaming of the system - so chances are, this kind of thing will be with us for a while.

I still like my internet.

We try to keep it light here at digitalmissive.com: adventures in iPhone home repair, subway advertisements, that kind of thing…  But today’s news of numerous computers at both Barack Obama’s and John McCain’s  campaign offices being hacked earlier this year reminded me of a particularly frightening Business Week article I listened to the podcast of a few weeks ago while practicing my backhand.  So I just now did a little additional research - consider this brief timeline:

November 6, 2008: US Newsweek reports that several computers on the campaign office networks of both Obama and McCain were compromised during the summer - Trojan malware sends an unknown amount of data detailing candidate policy positions to a “foreign entity”.  FBI launches investigation.

June 11, 2008: US Virginia Representatives (and longtime China human rights critics) Frank Wolf and announces that four of his Capitol Hill PCs were compromised by malware which copied and transferred an unknown amount of data.  FBI announces attack originated from China, declines to comment further.

December, 2007:  UK Director-general of British intelligence agency MI5 sends letter to 300 British companies warning that their networks are under attack.  Announcement explicitly names “Chinese state organizations” as source.

December, 2007: US 37,000+ attempted attacks on both government and private networks reported for the year, US Congress is informed that Chinese espionage represents “the single greatest risk to the security of American technologies”.  New 40,000 person US Air Force unit created to combat problem.

September 24, 2007: US FBI announces that the  Department of Homeland Security network had been attacked by malware originating (and communicating with) China.  Although from an “unclassified” network, an unknown amount of data copied and transferred  over the past 2 (!) years.  IT contractor Unisys denies any fault,  FBI investigates.

August 27, 2007: Germany While on a state visit to China, German newsweekly Der Spiegel reports that “many” computers in  Chancellor Angela Merkel’s office (as well as those in several other ministries) were found to be infected with trojan malware communicating back to Chinese-registered URLs. A 160GB data transfer stopped in progress, how much data lost previously remains unknown.  China denies involvement.

Whew.  Nervous yet?    The most common attack method used here is ‘trojan’ malware: software embedded into files of common Microsoft Office applications such as Word or PowerPoint.  The file arrives as an attachment in an email “spoofed” to appear from a trustworthy source, and the malware executes when the user opens the file (under certain conditions malicious javascript can also launch exploits merely from the unsuspecting user visiting a malicious website, although browser and OS upgrades help prevent this).

How can it all be done so anonymously?  What helps hide the actual source of these attacks is a technology called “Dynamic DNS”.  The ‘real’ (internet-wide) DNS system is what allows human-readable names (like digitalmissive.com) to map to an actual IP address - and to geek out for a second here, it is a marvel of a distributed database (it’s really amazing how well it works).  However, most non-enterprise ISPs don’t need to worry about their customers needing URL names, so they just maintain a pool of interchangeable IP addresses that they swap around to their customers as needed - meaning they’re often changing.  How then to publicly contact a computer (i.e. from the internet)  whose address is non-static?  Enter Dynamic DNS (because who doesn’t want to host a website from their home PC? …I kid).  Anyway, think of Dynamic DNS as an additional independent layer of DNS – a private service out on the internet has a valid public name reserved for you, and a little piece of software on your PC (or your home router) regularly calls out to that service and says ‘hey this is my IP address as of right now!’   Problem solved – but that additional layer is entirely private and uncontrolled, and so can be used in a ‘rogue’ fashion.  Add to that China’s non-transparent approach to all things internet, and conditions are favorable for these anonymous attacks.

Rightly, we know little of the highly classified political and technical goings-on to address the issue – I just have to assume that it’s on the good guys’ radar and that they’re on it, even if it’s not quite on the general public’s radar yet.  Something called ‘Reverse DNS’ can help a lot, and I’m sure MS is continuing to tighten up the security of the Office file formats (although there are serious legacy-compatibility issues there) - so we’ll see.

Apologies if this is all a little upsetting - my next post will be on the new colors the iPod Nano comes in.

While we’re on the subject of record long tail voter aggregation and its impact on democracy, the unprecedented accumulation of small-size incremental financial contributions during the current US presidential campaign marks another success story for the long tail of citizen ingenuity.

According to OpenSecrets.org, over 90% of an impressive $640 million raised by the Barack Obama campaign came from individuals rather than corporations or entrenched interest groups.

And the bulk of that was contributions under $200.

Amazing what a lot of a little can accomplish in its aggreate value - to the benefit of all.

On that note, for all you digital media marketers out there (opportunistically speaking, of course), the current long tail campaign donation phenomenon clearly demonstrates the significant power of consumers’ take on ”motive and opportunity”.

From digital video and online books to for-pay widgets and Twitter posts, monetizing the long tail of any of these things depends on whether they truly matter to people and their lifes.

Meaning, if “the cause” is right, wallets open up.

It clearly worked during the recent Presidential campaign. What does that mean to future branded product campaign designs?

To be sure, way way before Chris Anderson’s pointed Wired article (re)discovered the right side of the curve for us, something as old, tried and proven as democracy knew to utilize the long tail phenomenon all along; to ensure that all, not just a select few partake in shaping government at large.

So, in many ways, we’re only coming full circle here.

Who knew? Democracy as an ingenious grass-root marketing campaign.

Glad it worked so well this time.

As it stands, this country is about to (re)discover the power of long tail voting.

Not since the 60s has the United States seen this kind of voter turnout. Individuals previously ignorant to the democratic process are expected to come out and vote in what clearly is a historic election.

We, members of the digital technology and media industry, have used the long tail idea gladly and often, at least since Chris Anderson’s highly recognized Wired magazine article about “the few that dominate”.

We have since flocked to the long tail concept to describe how the aggregate number of individuals previously ignored by commercial systems can populate and popularize anything from micro blogs to amateur-produced snack-size videos, or en-gross selling of long forgotten books on amazon.com.

The same “saftey in numbers” phenomenon may now be just what it took to change the direction of an entire country.

While the latter remains a promise until proven, no matter who you vote for tomorrow morning, the former is happening as we speak.

Already a record number of those previously discouraged or put off by politics have returned from their voting duties; young, first-time voters, african-americans, hispanics, immigrants, all joined by millions of others in a common believe that its worth standing in line for hours on end, convinced that the time and cause is right.

Of course our industry’s arsenal of lingo would be incomplete if we couldn’t add to the long tail moniker all sorts of related terms.

Think discovery, collaboration, and sharing. Add hyper-targeting and monetization to understand how much the past 21 months of presidential campaigning have benefited from their first dabble with Web 2.0.

Not to mention the unprecedented number of supporting broadband connections that helped to fuel the national debate.

Still, it is not entirely clear whether any side has gained on the other in its particular ability to leverage the long-tail power of the Internet.

It just might simply be a zero-sum game.

But hey. Who’s counting?

For now, let’s go and vote!

PS: For those of you eager to combine your long tail capabilities until the very last second of the democratic process, go to Current TV and fire up your Twitter and your Digg account.

PS II: To be sure, while both campaigns leveraged the long tail power of the Internet, they also knew that a linear TV feed was still a medium key enough to agree to last minute Saturday Night Live appearances or to buy millions of dollars worth of traditional broadcasting airtime to get the message out. Shelly Palmer’s recent post makes several salient points on the deliberate old-school-ness of those decisions. Be sure to check it out.

…for an important message from some celebrities.

Just off a subway ride on the 6 line, from 38th street to Time Square.

While on the train and others prior, I notice again and again, people don’t seem any happier since jumping on the consumer gadget band wagon.

Although plugged into countless iPods, Kindles, and Zunes, typing into Blackberries, and watching videos on PSPs and iPhones, if New York subway riders are at all indicative of the larger mass - despite all our digital consumer innovation might - it just doesn’t seem to have made anyone genuinely more content compared to, let’s say, reading the newspaper or a hard cover book (as many might have in the past).

If that’s true, what user experience element is it that our consumer digital technology industry keeps ignoring?

Asking the experts might help.

Just back from this week’s Web 2.0 Expo in New York, Intel’s Director of User Experience, Genevieve Bell, gave a purposely simple but rather powerful talk about the art of truly matching our every day lives with the appropriate digital technology.

Amazing how different a role consumer digital innovation plays depending on where you go around the globe.

Back to the us, the New York subway, in all fairness, the news these days must add to anyone’s level of discomfort. I concur.

From political battles and hurricanes, to mortgage woes and banks collapsing, OK, so maybe there’s good reason to stay somewhat bleak and subdued while hasting from one hectic place in Manhattan to another, never quite “catching up”.

Still, isn’t it exactly right here where the US consumer electronics market is making substantial money? In large cities like New York, rich with affluent first adopters and technology aficionados looking to buy into personal devices designed to make it all easier, more bearable, dare I say it - more fun!

Looks to me, mission yet to be accomplished.

Off I go, my Blackberry in hand, trying to catch the next train out.

The New York Times recently ran Mattathias Schwartz’s powerful article about the popular online message board /b/ and a subgroup of its netizens, so-called “trolls”, that have turned into serious pranksters both online and off.

While largely unharmful, a number of past troll posts and activities have had serious, even deadly consequences - at least for some of the message boarders.

Think pseudonymous posters publishing violent fantasies about female college students.

Think humiliating details about people’s online dating activities publicly revealed to cause some to loose their job.

Think what seemed to have been the mother of a girlfriend posting unnamed online messages ultimately leading to the suicide of a young girl unable to cope with the public humiliation caused by the claims made online.

My questions: Do any of these troll activities happen because of or despite the Internet’s popular existence? Is the Internet as a “public square” at all capable, let alone intended, to prevent certain social interaction from occurring online.

My take on this: The Web is a ubiquitously public “application” residing on the Internet itself.

As such it was never designed to regulate human behavior. Asking the Web to begin vetting engagement, under the current structure, it wouldn’t work. Not to mention the implications for basic civil liberties such as free speech.

Instead, attempting to curb personal behavior online would be as if anyone taking the railroad or public highway would have to adhere to specific moral, intellectual, or cultural code outside ones own in order to qualify for its regular use.

Ubiquitously available public platforms are just not set out to perform that task.

Good for pranksters everywhere, this is probably why there’s no “thou shalt not humiliate” signs posted anywhere.

Not at one’s local train station, nor shoulder-side down the highway, or anywhere on the World Wide Web.

This is because if such signs existed, they certainly would neither change pranksters’ behavior nor would they make it any easier to catch them after the fact.

Instead, when using any of our shared and (fairly) organized public systems, the large majority agrees to certain ground rules.

Some are rather obvious (such as basic societal laws), other require added sensibility and discretion. Never intended to be foolpoof, that “system” mostly works.

As to the Web, Schwartz, in his article, points to Postel’s Law: “Be conservative in what you do; be liberal in what you accept from others”.

Originally intended as a fundamental ground rule for all of us online, it seems a pretty good guideline for other public platforms as well.