Three Puzzle Pieces

The triad of digital infrastructure

The triad of digital infrastructure

Stacey Higginbotham wrote an interesting piece over at GigaOm, called “Something’s gotta give when big data meets broadband.” She writes,

Scientists aren’t worried about storing or processing all data, according to an article written by Mari Silbey for A Smarter PlanetSmartPlanet. Instead they’re worried about shipping that data from Chile to everyplace else it will be wanted. Basically it’s not a big data issue, it’s a broadband issue.

My feeling is that, while this may be a current bottleneck, if we removed either of the other two pieces, the same scientists would have nightmares figuring out how to store their massive volumes of data or how to crunch the petabytes of information. I don’t think the scientists disagree; they are more concerned that the broadband piece is lagging behind storage and compute capacity.

But there is progress in the broadband arena that will surely support these researchers. Notably, a rather under-ballyhooed demonstration taking place this week at the GENI Engineering Conference in Boston.

The demonstration is called Slice Around the World, and you won’t find much about it on the Internet, ironically.

In the Slice Around the World demonstration, software – rather than manually configured network resources – will dynamically direct network and computation high volume data streams over the ultra-high-speed CANARIE and BCNET networks, as well as other research-and-education networks around the world.

Ordinarily, network switches and routers are proprietary hardware, manually configured to direct traffic over specified paths, much as a railroad switches tracks to direct trains onto specific, defined routes. With a programmable network, traffic flow can be controlled by software dynamically with almost instant response to dramatically increased traffic needs or network congestion. These highly programmable OpenFlow-based networks are considered to be the next dramatic leap in network technology innovation. The Slice Around the World demonstration is a critical step forward in proving the capabilities of this approach, which launches an era of ubiquitous high performance networking and computation that is as accessible and easy-to-use as the web is today.

With the software approach, the flow is much more like a river flowing out to a delta: the water (data) flows where there is least restriction without manual intervention.

Could this be the solution to the scientists’ concern about data bottlenecks? Who knows. But advanced networks in Canada and globally are already delving into the problem.

What are your thoughts?

One Response to Three Puzzle Pieces

  1. I fell over myself laughing after reading this in the article. “So maybe it’s a method of deduplicating data before it travels over the congested networks or maybe it’s …. reducing the unnecessary data …. so scientists can pick only the data sets that are relevant to their research”.

    “De-duplicating”. What a wonderful term for the future world of research. Looking at this particular example of big data, I wonder why it isn’t obvious to put the user apps in the data “access points/centers”. We already know people will only require “a portion of the whole, or the whole of a portion” of a data set, so they would naturally gravitate to, in this case, one of the 2 complete libraries; either of which provide access. So if bandwidth is the weak link, locating the user apps in these computer sites would, logically, solve the problem. It would probably also show up the duplications in user apps (and the degree of duplication in simialar research)

    The problem goes deeper than this. As you say, it’s ironic that this particular GENI “slice of heaven” doesn’t appear on the (let’s say) web. This illustrates the main problem; so much duplication. All these engineers working on similar things in isolation and the only time yu hear about it is when a journo (like yu) gets it mentioned, or the researchers, who have studiously ignored their outside worlds, want to make an announcement. E.g. My friends in Europe and oz were not at all interested to hear that “Ryerson University leads development of global campus HD IP broadcast network”, because they know their research is so much more “leading”. (Hopefully to another grant)

    So i think you’ve hit it on the head with your last line = “advanced networks in Canada and globally are already delving into the problem. Just take out the “and globally” because they are still working on the basis on which they were funded. Bit like this blog really. No I’m not having go at you (nice blog), just making the same point I make with my friends in Europe. So much (national) duplication, so little (global) collaboration.

    There was an interesting video by a Jason Morgan on that Gigaom page, about “the next generation of collaborative tools”. Interesting. Maybe we should create a slice around the world for NREN members. Getting them to share might be a bit of a challenge though.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: