An eduroam Sunday in Canada

One aspect of being the eduroam operator in Canada is how to maintain the high quality of service eduroam is known for and respond to issues that may arise before end users encounter them. To do this, CANARIE has taken a proactive stance to monitoring the health and performance characteristics of eduroam in Canada and works with domestic eduroam sites to improve the service or diagnose issues. An offshoot of this investment is being able to better understand how the service is used through the performance monitoring data and feed this information back in to improve on the eduroam environment.

This is not a small undertaking as eduroam interconnects 48 sites across Canada as well as to the international eduroam system of 60 plus countries.

What piqued our interest and this blog post is a particular Sunday in May had higher traffic than traditionally expected, much like a weekday, and we wanted to see if we could find out why.  As we dug into our numbers we visualized the data in a number of dimensions, by time and by location to better depict how eduroam is being used.  It shows how mobile users can be anytime and anywhere when they are connected with CANARIE and CAF.

The Stats

The daily report for Sunday that shows the overall hourly traffic that the Canadian eduroam servers observe.  Traffic is exclusively those who are mobile and not at their home institution and are sign ons, not individual users.  A sign on can be a single user, but from multiple devices or from a single device as they move around.

The graph below shows traffic across Canada on Sunday May 5^th, 2013.  The stats behind the line are from our sites across Canada.  The Success line is self explanatory, but a ‘Fail’ entry may be due to a variety of reasons and be very intermittent as shown in the spike just before 4pm below. Being able to identify these is always a work in progress to keep the quality of service as high as possible. This one looks transient and quickly goes away.

The above graph illustrates individual traffic by domain.  We have masked the domains and will focus analyzing stats for the highest line in the graph.

So where are Canadian mobile users benefiting from eduroam?

Short answer: All over the world!
Zooming in on traffic for this one domain and overlaying it on a map we get an idea just how mobile this domain’s users are. Remember this data spans the 24hrs on Sunday.

Here are a few views from other regions around the world for this domain:

Central and northern Europe:

 

Inside Canada

Eastern Canada and North East USA:

Western Canada

Now in the Eastern Ontario area:

Observations

All in all, very interesting just how mobile users are in this one domain. It also drives home the point that being connected is not only important  across Canada, but worldwide and not just Monday to Friday 9-5 either, but 7 days a week, 24×7. You can see this in the first graph at the top of this post as the successful sign ons  never goes to zero even at 4am Eastern  —  somewhere in the world there are people from this domain using their home institution credentials.
While obviously a great benefit for eduroam users to be on wifi around Canada and the world, behind the scenes it allows eduroam site operators to avoid issuing guest accounts and/or  having to running open WiFi access points to support mobile users all the while maintaining the appropriate level of certainty of who is using their network.

I would be remiss if I did not extend the invitation to those not connected to CAF to join and expand the coverage of eduroam even further in Canada. We would love to have you connected!

Postscript

I will be presenting at CANHEIT 2013 in Ottawa on All Things Eduroam as well as our work on simplifying participating with CAF for eduroam and Federated Single Sign On(SSO) in my Leveraging the Cloud to Deliver Identity Services talk. See you there!

Some eduroam improvements behind the Scenes

Things have been quiet as we’ve been heads down working and are ready to share some of our progress. One item the CAF team has been working on is improving the eduroam health monitoring infrastructure behind the scenes. This is in response to intermittent reports where eduroam doesn’t work as well as it should have.  By enhancing the monitoring infrastructure it allows us to better assess how well (or not) eduroam in Canada is performing and identify any improvements that can be made.  This helps maintain the quality of the eduroam service as good as it has been or better as eduroam spreads further across Canada.

What does the end user see?

For the most part, end users experience a reality of either it works or doesn’t and as a rule,  things appear to work  smoothly. This is sometimes deceptive, but unintentionally so.  Some devices mask the number of retries they make attempting to get online and all you may see is the checkmark beside the eduroam SSID indicating you are connected.  What is not seen though are that some devices retry aggressively multiple times anywhere between 5 to hundreds of times in the span of a few minutes to get online. Multiply this by the number of devices you carry (laptop, phone, tablet etc) and maybe a wrong password in one device and you can get a glimpse at what the problem could be if not handled well.

While the end user doesn’t see or realize this is happening under the hood,these transactions are visible at the Canadian eduroam servers — of course only for traffic originating in Canada.  This style of activity is taken into consideration and is part of the monitoring practices and metrics we track.  We don’t always have a lot to go on other than the destination and origin due to the encryption of the traffic but that is enough for us to engage and inform the target sites that something may be going on or has occurred.

Analyzing the Data So Far

With over a million successful monthly sign-ons since November 2012, we’ve had a lot of data to analyze! As a starting point, we are looking at requests that result in a ‘No Reply’ response in our logs at our root Canadian eduroam servers, which would indicate that a participant’s RADIUS server is temporarily offline.

Right now the traffic patterns show a 10% ‘No Reply’ overall rate for RADIUS authentication requests.  These requests appear in spikes like the above graph of 24hr of eduroam traffic. It may be that this is an artifact of the UDP based protocol or potentially how ‘chatty’ mobile devices could be but either way our goal is to understand what it means and how we reduce the problem from current levels and in turn improve the eduroam service.

What Canadian eduroam Sites May See Next

CANARIE will be analyzing log files a few times a week and may reach out to individual eduroam site contacts to clarify anomalies as we encounter them. We know time is precious and diagnosing a transient issue is difficult so if we do contact you we will try and provide a detailed report about the time period in question. We use Splunk, a commercial log analysis tool with our custom reports that can pinpoint the issue and timeframe in question save diagnosis time.  Even with tools like Splunk we still manually assess when to escalate to a site to ensure that it’s worth digging into and appreciate your help to go the ‘last mile’ with your local RADIUS  and network logs.

Préservation et expansion du savoir humain

Écrit par Jim Ghadbane, directeur de la technologie, CANARIE

Sheldon Cooper mis à part (ce personnage de The Big Bang Theory aussi intelligent qu’insupportable qui semble tout savoir), personne ne peut à lui seul fabriquer d’automobile. Par là, je veux dire qu’aucun de nous ne sait comment tirer de l’acier du sol, comment faire du verre avec du silicium, comment produire du caoutchouc, du nylon, du tissu, des semi-conducteurs, de l’antigel et ainsi de suite.

Bon an mal an, l’humanité fabrique pourtant au-delà de vingt millions d’automobiles par année, soit environ 145 000 par jour ouvrable!

Les automobiles construites à la chaîne il y a un siècle n’ont ni l’aspect, ni la performance ou la sûreté des voitures modernes. Certes, elles ont, comme elles, été réalisées grâce au savoir et aux découvertes d’êtres humains qui ont malheureusement disparu depuis longtemps. L’humanité a néanmoins préservé l’essentiel de leurs découvertes. En effet, sauvegarder les connaissances du passé s’avère indispensable à la réalisation de nouvelles découvertes.

La première grande étape franchie pour préserver le savoir humain fut l’écriture. Lire revient un peu à laisser l’auteur d’origine nous inculquer directement son savoir. Avant l’écriture et la lecture, les connaissances étaient transmises par tradition orale et par mnémotechnie (il y a trente jours en septembre, avril, juin et novembre, par exemple). Essayez un peu d’apprendre à votre enfant à bâtir une Ferrari avec cette méthode!

La percée qui a suivi dans la rétention du savoir fut l’imprimerie. En rédigeant un ouvrage, l’auteur ou un petit groupe d’auteurs parvenait à atteindre la population entière – d’un individu à une multitude. L’invention de la presse d’imprimerie a contribué à augmenter le nombre de volumes, donc à faire rayonner les connaissances d’une façon sans précédent : des bibliothèques ont vu le jour localement, l’alphabétisme s’est propagé à une vitesse phénoménale, et le savoir collectif a crû de manière draconienne. Les connaissances se sont multipliées plus vite, de plus en plus d’humains recourant au savoir acquis dans le passé.

Une vitesse incroyable

La création d’informations progresse actuellement à une vitesse incroyable. Cette croissance alimentera l’expansion du savoir dans des directions que nous parvenons à peine à imaginer. Le moteur de cette croissance est l’Internet. Pour nous, techniciens, l’Internet est l’équipement – les routeurs, les lignes DSL, les fibres, les pylônes, etc. – à la base des systèmes qui constituent les réseaux. Pour le reste de la planète, l’Internet est la puissance de calcul, le stockage, Facebook, Google, Wikipedia, Yahoo, MSN, Twitter, et j’en passe. C’est là que l’action se déroule, le niveau inférieur n’étant que la plomberie qui permet à l’eau de circuler.

Contrairement aux deux premières étapes qui ont révolutionné l’expansion du savoir, l’Internet permet à chacun d’apprendre du passé et d’enrichir le bagage de connaissances de l’humanité. Il autorise la diffusion de l’information et la collaboration entre n’importe qui et qui que ce soit, entre un grand nombre et une foule d’individus, et de toute autre manière imaginable. Il permet aussi au savoir de parvenir sur le bureau ou dans la paume du commun des mortels – les jours où vous croisiez les doigts en espérant qu’un autre n’ait pas emprunté avant vous le livre dont vous aviez besoin à la bibliothèque sont révolus. Ce livre, vous l’avez déjà en main!

Est-ce qu’on pourrait créer un cochon volant?

Qu’arrivera-t-il une fois que l’humanité entière aura accès à la totalité du savoir amassé au fil des siècles? Le temps seul nous permettra de répondre collectivement à cette question, et la réponse nous surprendra à coup sûr. La surprise viendra en partie de la diversité des valeurs culturelles qui forgent l’humanité. Impossible de prévoir ce qui se passera quand les sociétés auront toutes accès au savoir universel.

Mon but n’est absolument pas de suggérer qu’il faut craindre les conséquences éventuelles d’un partage du savoir collectif. Cependant, nous devons nous préparer à vivre dans un monde où le savoir sera exploité d’une manière qu’on peut difficilement prévoir ou contrôler. On pourrait, par exemple, synthétiser de l’ADN afin de créer des cochons volants (fameux adynaton!), comme on pourrait y recourir pour faire luire des arbres dans le noir et ainsi combattre la pénurie d’énergie.

CANARIE fait sa part

CANARIE fait sa part pour que les Canadiens ne traînent pas de l’arrière dans cette expansion phénoménale du savoir humain et de la rétention des connaissances. Notre infrastructure numérique doit absolument continuer de s’adapter et d’évoluer afin que les Canadiens demeurent compétitifs dans l’économie du savoir.

À présent, essayons de voir comment mettre fin aux embouteillages – ou fabriquer des automobiles qui volent!

Human Knowledge Retention and Growth

By Jim Ghadbane, CTO, CANARIE Inc.

Other than the really smart (and obnoxious) guy, Sheldon Cooper on The Big Bang Theory, who seems to know everything, no single human can make a car. By that I mean no single human knows how to make steel from rock, how to make glass from silica, how to make rubber, nylon, cloth, semiconductors, anti-freeze, on, on, and on.

Yet the annual estimated worldwide production of cars by humanity is over 20 million – approximately 145,000 per calendar day!

Cars mass-produced 100 years ago don’t have the look, performance, or safety of the modern automobile.  Like current cars, the early cars were built on knowledge and discovery by humans who are sadly long gone.  Yet the critical information of their discoveries has been retained by humanity. Retaining past human knowledge is crucial for new discovery.

The first major leap in knowledge retention came with writing. Reading is akin to the original author imparting their knowledge directly to us. Before reading and writing, knowledge was retained with spoken mnemonics – e.g., 30 days hath September, April, June and November…  Try using that technique to pass on to your child how to make a Ferrari from scratch!

The next major break-through in knowledge retention was the printing press. Books are written by an author, or co-authored by a small set of people, yet have the ability to reach everyone – one to many. The increased volume of books, enabled by the printing press, made it possible for the unprecedented outreach of knowledge: libraries became local, literacy rates soared – the collective human knowledge grew dramatically. Its rate of growth increased as more humans were enabled to build on past knowledge.

Unprecedented Growth

We are currently experiencing an unprecedented rate of growth in information creation. That growth will fuel knowledge growth in ways we are only beginning to comprehend. The enabler for that growth is the Internet. For us techies, the Internet is the hardware – routers, DSL, fibre, radio towers, etc. that make up the systems which make the physical networks.  For everyone else, the Internet is online computing, storage, Facebook, Google, Wikipedia, Yahoo, MSN, Twitter etc. It’s this second part where all the action is, the first part is just the plumbing to get to it.

Unlike the first two revolutionary steps towards knowledge retention, the Internet allows us all to learn from the past, and to make a contribution to the collective knowledge of humanity. It enables any to any, many to many or any other ad-hoc form of information dissemination and collaboration. It also enables knowledge outreach to our desktop or to the palms of our hands – gone are the days of hoping that someone else hasn’t already checked-out the book you need – it’s already in your hand!

Will pigs fly?

Where will it lead?

Where will it lead us when all of humanity has access to our accumulated knowledge? Over time we will collectively answer that question; and the answer is almost guaranteed to surprise us. The surprise will come, in part, as a result of the wide diversity of values in the cultures which make up humanity.  When the collective world knowledge is accessible to all societies, the outcomes will be highly unpredictable.

This is not to suggest that we should fear the possible outcomes of shared collective knowledge. However, we must be prepared to live in a world where knowledge is exploited in ways we have little ability to predict or control. For example, DNA generation could be used to actually make pigs that can fly (so much for that adynaton!), just as it could be used to make trees that glow in the dark to deal with energy shortage.

Doing our Part

We at CANARIE are doing our part to ensure Canadians aren’t left behind in the most significant growth of human knowledge and knowledge retention ever. Continuing to adapt and grow the digital infrastructure is essential to keeping Canadians competitive in the knowledge economy.

Now let’s all figure out how to make traffic go away – or cars that can fly!

Canadian Access Federation: A model that

Canadian Access Federation: A model that works http://owl.li/ecpF4 #CANARIE_CAF

Dispatches from REFEDS and VAMP2012

Good morning from Utrecht, NL where I am attending as CANARIE’s CAF representative for REFEDS & VAMP2012

I’ve found that this September is an inflection point for change; back to school kicks in, summer holidays recharge the batteries and give a chance to step back and take stock. To this end, I’m going to experiment with a  more brief communication model with this blog.  There may be the occasional essay like post because complicated topics need their due depth, but I would rather have more frequent postings to avoid the TL;DR (Too Long; Didn’t read) and see which ones to go deep on as people express an interest in them.

Why REFEDS?

Research and Education FEDerations is one of the few locations dedicated to the interests of  CAF and our peers.  It is also a forum for advancing Federated Identity topics and collaborating on workplans to the benefit of multiple federations.  One topic that has been incubating in the REFEDS environment are recommendations for Service Providers for Federated ID sign on and discovery which are based in part on a NISO Espresso Report.  An interested and comprehensive document at 35 pages. More to come on this front soon.

As the REFEDS meeting progresses more will be posted to this blog entry.

In the mean time, I would like to point you to the 2 day VAMP2012 agenda and encourage you to post comments or questions that you would like to hear me bring forward.

A great day for digital innovators everywhere!

The sunny smiles inside matched the sunny skies outside this morning at the University of Ottawa, where Minister of State (Science and Technology) Gary Goodyear announced CANARIE’s renewed mandate and three-year funding of $62 million.

During the event, which representatives of the university and research community attended, the Minister was clear about the importance of CANARIE to Canada’s innovation ecosystem. He spoke eloquently on how CANARIE (together with its partner networks in the provinces and territories), enables Canadians to participate, and lead, in global initiatives that have a tangible impact:

“CANARIE is the bedrock for new advancements in science, health care and other important disciplines. These advancements improve our health, our environment, our economy and our future.”

The Minister also noted other initiatives that CANARIE is pursuing in order to meet the evolving needs of Canada’s innovators, researchers and educators:

“But beyond its core mandate as Canada’s premier research and education network, CANARIE has been making a valuable contribution to other government priorities, such as the economy and Canada’s global competitiveness.

“Last year, CANARIE launched the Digital Accelerator for Innovation and Research program, or DAIR. This pilot program supported a key element of Canada’s digital economy strategy—namely, helping to grow our information and communications technology industry.

“DAIR gave small and medium-sized digital technology firms access to CANARIE’s reliable, ultra-fast network to design and test their products, share massive amounts of data and accelerate innovation, giving them a first-to-market advantage.

“CANARIE is changing with the times in other ways as well. Traffic over the network has increased nearly 600 percent in the past five years. In response, CANARIE has been able to adapt and will continue to do so to meet this burgeoning user demand.”

Minister Goodyear’s enthusiasm and support of CANARIE and of the Canadians whose work it supports was obvious, and he was particularly eloquent in his closing remarks:

“CANARIE has helped Canada’s greatest minds accomplish incredible things, and we look forward to that track record of success continuing.”

Thank you Minister Goodyear, and the Government of Canada, for your continued support of CANARIE – a fundamental element of Canada’s digital infrastructure for research and education.

Read the full speech here in English, or here in French.

Virtualize The Network

The firmware of network switches and routers (control plane) has traditionally remained proprietary, locked and under the control of the companies that manufactured the equipment. Software defined networking seeks to change this oligarchy.

VMware’s recent acquisition of Nicira allows the virtualization giant to become the “industry leader in software-defined networking.” Nicira’s software acts as an intelligent layer that sits on top of data-center servers and manages and controls the resources and capacity of networking gear.

Interesting fact: Nicira’s founders invented OpenFlow. Today OpenStack uses Nicira’s API for its network interface.

Combined with VMware’s market proven expertise virtualizing systems, software-defined networking is well positioned to become a data-center game changer.

Video: Nicira: Creating Virtual Networks in Cloud Data Centers

Venture Beat says Big Data will fuel a new age of TV.

Venture Beat writes, “The data revolution uses real-world metrics to inform business-critical decisions. The result is reduced costs, added value, and additional revenue.”

What do you think? http://owl.li/cqNPp

Introducing Neptune Canada’s Digital Marine Life Field Guide e-book

Introducing  Neptune Canada’s Marine Life Field Guide Useful!

http://owl.li/cmi0M