Thursday, May 30, 2013

SDN and Network abstraction take 2

Ok a while ago wearing a network hat and arrived with that view of network abstraction on top of the network virtualization as a consequence of SDN (see previous blog). Some of the results looked more like a classic approach on exposing network assets as APIs and can actually be done without using SDN. But looking at the first category of abstraction, I said to myself that there must be something disruptive and game changing by implementing SDN, so wearing a IT hat I arrived with a more complete thinking about network abstraction :

Major internet service providers like Google already have very distributed core network of IT resources (based on commoditized blades) and they have realized that this network had to be very dynamic, programmable, and low cost and therefore using SDN/Openflow or equivalent is a way to handle these problems.

They also are spending a lot of efforts in expand this core network with remote IT resources by:
  • giving SDN racks to be installed in other networks, (viewed as a CDN solution for Youtube)
  • providing home/enterprise gateways to be attached to operator networks or within enterprise networks (viewed as a TV solution (GoogleTV) or as a search optimization solution (Google Appliance)
  • providing downloadable code to end user devices viewed as a browser/device bases service platform (chrome) or as search expansion (google desktop)
  • and adding openflow like functionality in android which means that any android devices can be viewed as an IT resource....
Each of these activities provide a service to the user/enterprise (TV, CDN, optimization etc…)  and at the same time drastically expand their core network of IT resources.  This pushes the network providers to become a forwarding plane managing opaque tunnels (I will prefer canals term see why later) carrying high level/value signalling/payload between the remote IT resources and their core network.

While this situation is basically unavoidable and could appear as yet another form of disintermediation, network operators can also change the game as a consequence of implementing SDN. Network operators are implementing SDN to reduce cost by performing two actives:
  1. Virtualize the hardware of each network element (base stations, firewalls, routers, switches etc…)
  2. Create distributed controllers to make the network easy to configure (ultimately software based), cheap to manage.
The second activity is core since there is a need to quickly and dynamically establish paths (route, vpn etc..) in the network but this will be completely hidden to developers. The virtualization activity can be used to create IT resources (using a de facto abstraction standard like AWS specs) that can be used by developers to run their machine images or implement storage buckets. The abstraction  is implemented as an extra workload on the network element virtualized hardware and a broker (similar to the brokers for hybrid cloud implementations) is needed to find the IT resources either in an explicit way of based on rules/policies or even analytics.  A cloud watch equivalent must also be implemented as part of the abstraction to give IT resource feedback to the broker.

This means that the network abstraction result for using SDN is a massively distributed data center with very interesting IT resources called: edge IT resources.  Of course the network based IT resources will have constraints and therefor will makes them different from the data centre based IT resources, but having different IT resources is not a problem for developers (e.g. AWS has more than 17 EC2 instance types).

The network based IT resources and more specifically the edge IT resources are game changing because: 
  • Only a network operator can implement this type of edge IT resources.
  • We are dealing now with IT resources that are 1ms away for the devices hence reducing the need of running workloads in the device while still  maintaining a very responsive user experience hence avoiding unnecessary power and bandwidth consumption.
  • Offload the core network of IT blades of the small workloads that are created to pre-empt activities performed on mobile devices (these small workloads have similar size of overhead than the large workloads and with the proliferation of mobile devices (e.g.: Kindle for Amazon) there is an important increase of the ratio "total overhead"/"total effective workload" thus reducing the value generated by the core network of IT resources. This is a similar problem known by the network providers with the header versus payload ratio.
  • Network operators are not just canals handler, but a part of the IT supply chain which increases their visibility of the high value activities at the software service level.
In this situation network operators are not just canals handler, but a part of the IT supply chain which increase their visibility of the high value activities at the software service level.

From an infrastructure perspective, the developers will see a continuum of IT resources from the back end to the device including the network instead of having to deal with the network as an IT no man’s land which today has a great negative impact in particular on mobile or M2M solutions.

The way software services need to be developed in order to fully benefit of the continuum of IT resources should actually not be a problem for cloud developer since they are already handling the following:
  • API: developers are already familiar with consuming API (internal or 3rd parties) in order to develop their solutions and therefore they know how to develop solution based on loosely coupled software elements.
  • Elasticity: cloud developers are already aware of the complexity/benefits when dealing with a cloud infrastructure by using elasticity. It appears that there are different sophistication levels on elasticity:
    • Level 1: request or release of IT resources to match the load (available services)
    •  Level 2: use of elasticity to handle IT resource failures while still coping with the load  (resilient services)
    • Level 3: broadening the criteria to trigger elasticity process (dynamic services)
    • Level 4: depending on a hybrid cloud broker to get IT resources in different part of the accessible continuum of IT resources (nomadic services)
    •  Level 5: analytic based criteria to trigger the elasticity process and analytic based broker to get the IT resource (liquid service),
    • Many cloud developers are already pass beyond level 3.
  • API discovery: when software elements move (e.g. software element following roaming devices), an explicit or implicit “DNS” like system pattern may be needed to discover where the new API endpoint. This  is similar to what has been developed and deployed by Apigee API Exchange.
  • Resource discovery: data access also needs to be handled appropriately in order to make sure that the data can follow the software elements. XRD based index can be used for that. This may not solve completely the consistency problem of the data but developers are already familiar with dealing with inconsistent or almost consistent data by the time the level of consistency is known.
Today because of the IT no man’s land, canals have to be built thru the network in order to let the liquid services move closer to the devices. Making the network (operator/enterprise)  a massively distributed data center as an abstraction consequence of implementing SDN, will be like opening levies and letting the existing and future liquid services expand up to an edge that is very close (1ms) to the devices and therefore to the user providing human touch like user experiences which are advertised by 5G networks. This may imply that 5G will actually not be solved only by improving the radio/network but also by a completely different IT approach and a maturity in distributed software towards liquid services.


  1. The canal metaphor is a useful one to highlight the "liquid" nature of future applications and services. Reminds me of Christophe Longépé's town planning/urbanisation metaphors for Enterprise IT Architecture...

    Urban planning decisions (good or bad) affect activities like public infrastructure maintenance like roads, utility rights of way, canals(!) etc. Operational spending...

    SDNs are appealing because of the promise of automation and hence reduced operations costs - do more routine tasks with less people. But this really only works in practice if the architectural approach guides it in the right direction.

    I've seen brilliantly thought out Enterprise IT Architectures, things of beauty which required a cast of thousands to operate - rigid, slow, expensive. Just like some towns which have great infrastructure and clear rules but cost billions to run every year!

    So, I guess the key here to cracking the SDN nut is to work out ways to empower companies to do more with less and have a meta-architecture to influence their own infrastructure implementation choices?

  2. Michel- I posted this on my blog last week, I would love to have a discussion with you about the content.

    Networking the Application, exists with or without SDN

    There is a great deal of focus on Software Defined Networks (moving control plane to the server, virtualizing the router and switch) these days. People are of the belief that SDN will someday allow “Networking the Application”. The fault in this is that the architecture continues to leave data and applications susceptible to attacks in layer 2 and 3.
    Application centric networking is not a function of a device but rather of a protocol, including where in the stack the protocol resides. Application centric networking decouples network-dependent applications from the network and enables parts of an application or a group of related applications to be distributed safely around the network using XMPP to tie the components together and enable them to act as a single unit.

    This capability is accomplished by moving the networking control and forwarding functionality to Layer 7 (application layer in the OSI stack). Each end-point is assigned a domain address, which is used for identification as opposed to traditional IP addressing; XMPP endpoints are tunneled to those within the domains they are assigned to. Additionally, the presence feature of XMPP is a powerful concept that can be used for nodes in the application network to be aware of the states of the other nodes.

    XMPP also enables a very secure coupling between the application nodes. A compromise of the physical network, or even the software defined network (SDN), does not mean a compromise of the application. The application network may be disrupted by denial of service (DoS) but the data stream will not be compromised if an attacker is able to hack a router or attempt a man-in-the-middle attack.

    Additional benefits of using a protocol for networking is the ability to
    • Close all incoming ports on the firewall for less intrusion points on a network
    • Reduce need for external Load Balancing appliances, some commercial platforms accommodate
    • Consolidate your SOA, ESB platforms into the communications platform for reduction of overall infrastructure savings in power, space, HVAC etc.
    • Diminished need for intelligent routers and switches allows reduction in costs of expensive equipment

    The need for secure network communication between end points, man or machine, is not accomplished by throwing more points of failure/intrusion points into a network. The functionality needs to be in the protocol itself that is allowing the connection between endpoints and bi-directional transport of the data.

    1. I agree with you that networking application is not dependent of SDN and protocol like XMPP will be a great way to pass information from one software elements to another. Basically it is saying that the Layer 7 of the OSI model is actually more distributed than the network itself and actually represents a network where the nodes are the software elements and the links are the exchange of semantically rich information. Ir will actually be very interesting to figure how many layers (embedded in the layer 7) this network has. My view is that with SDN you have a better continuum of IT resources to play with than the classic data center, client situation .... which has two consequences:
      1 make use of IT resource that are one hop away form the client ...
      2 put the operators back in a more favorable situation of having a better visibility on what is happening in this new network, which today is completely hidden.

  3. Great post. I hope you can write more good stuff like this article.

    Boston Network Services

  4. custom software development and providing software maintenance services. They build up variety of software mostly focus on CMS, website development, Desktop & web application, Iphone apps, mobile apps for android, internet marketing and many more.


Comments and suggestion are always welcomed !!!