API Exposure Platform

Introduction

APIs have become the new hot topic and, while still being quite immature, may have reached the tipping point to full recognition as a key component in any software project. It is important to recognized the three different environments involved in the API supply chain:

1. On the bottom there is the API implementation environment which represents the set of systems/platforms/frameworks that actually makes the APIs work.
2. In the middle there is the API exposure environment which represents the system/platform that helps the APIs to be available to developers (short tail, long tail) and also handles all the aspects associated with making an API commercially viable.
3. Finally, on top there is the API consumption environment which represents the set of systems/platforms/frameworks that facilitate the consumption of exposed APIs in order to create new solutions.

What makes this segmentation particularly complex is the fractal nature of APIs since when a solution is created by developing a piece of software that consumes API's, it is possible for that piece of software to actually implement an API and while part of the consumption environment the same piece of software and the systems/platforms/frameworks that support it, are simultaneously part of the implementation and consumption environment. However for matter of simplicity, one approach is to only work at the first level of the fractal but understand that this pattern exists, How many time the recursion happens may be bound by three constraints/observations:

• not every solution actually implements an API, 
• not every software solution that implements an API is made by composing API's (which may then be called a canonical API),
• composing API's recursively may lead to complexities like functionality echo.

This paper will describe the different environments and focus on the functionality and specific characteristics of the exposure environment by describing the API exposure platform.

The implementation environment

This environment is represented by a wide variety of technologies/architecture since implementing a canonical API may follow a session driven paradigm, an enterprise architecture paradigm, use a application server or may be based on a completely homegrown solution depending on the domain in which the API is implemented. Looking at the telco industry for example we can identity four implementation domains:

1. Network,
2. BSS (Business Support Systems),
3. Products and Services,
4. Infrastructure,

and each of these domains generally follows a different paradigm well represented by the framework/platform used to implement the solutions within these domains. IMS (IP Multimedia Subsystems) in the Network domain has a session centric architecture, while SDP (Service Delivery Platform) in the BSS domain follows a more classic enterprise architecture with a enterprise service bus that mediates all the events/messages that are exchanged between the different sub-systems attached to the bus. Products and Services domain follows a more internet centric architecture where the base platform called service platform is actually a set of individual tools specialized in federating many other internal or external components. Finally the Infrastructure domain follows a broker based approach (analytic or policy driven) to handle the different IT/network resources available either within data center or in the network.

The consumption environment

This environment is where an exposed API is being consumed. There are many ways to consume an API: directly, or via an intermediary (SDK, adapter, broker, platform, framework) and this with a full IDE or a simple text editor. The consumption can happen at multiple levels from the device that delivers an experience (user or not), the front end technologies that support the device, or the back end systems (broker, platforms, frameworks, or application servers..) used to developed software services.

This consumption domain has followed a natural evolution of the software development which started by using local subroutines, to libraries from local to distributed, to remote objects accessible via local stub, to software services that implement APIs. This evolution has introduced multiple new levels of complexity inherent to distributed systems like concurrency, deadlock, service latency, network failure, availability constraint, elasticity, on demand access of base resources which have generally lead to the following realizations:

• Never hide distribution to the consumer of an API: latency, network error will happen and API consuming solutions should have a way to handle these problems. (Jim Waldo interview on distributed systems), introduction of DTN (Delay Tolerant Network) by Vint Cerf.
• Never hide elasticity but exposed it via API and continuously expand the criteria to trigger it. Expand the criteria for refining the on-demand access of base resources and expose this mechanism via API. The API consumers will hve the logic to either deal with the elasticity triggers and will make the requests to get or rekease resources.
• Dealing with a API controlled by an independent party implies (especially for mission critical solutions) more discipline around system management (FCAPS (Fault, Configuration, Accounting, Performance, Security) model will apply to an API or the implementation of it).

Today most of the API consumption is best effort and some of the previously described issues are not yet addressed but more and more it will be important to handle system management issues (one of the role of the exposure environment) and ultimately develop fluid solutions.

The exposure environment

Also know as API exposure,  the exposure environment is an important step in the API supply chain since it allows a consumer (developer, partners, etc..) to find the available API's and interact with them in order to build simple best effort applications or complex mission critical systems..

Platform functionality

While there are many ways to implement and consume an API, exposing an API can and should be handled thru one platform (eventually with many logical instances), by the time special attention on not being API semantic specific is constantly taken care of.

Beyond the classic aspects of a community based platform including blogs, review, forums and even some form of gamification to recognized members of the community, from a top down perspective, the API exposure platform must address the following:

• Developer registration: a developer can be a single person consuming an API as a hobby or can belongs to a large corporation developing enterprise grade solution. It is possible to have many level of developers (gold, silver, bronze...) and different structures (single developer to groups of developers). While many aspects of the platform should be accessible without registration, it is important to create a structured community of developers by registering and categorizing these developers.
• Solution registration: based on the state of development, a solution (application, software service etc..) that consumes an API need to be registered and this for analytic purpose or for business model perspective. The developer could be used for that but it will be important to know more details about the solution that actually consumes the API. This is borderline with the consumption domain, but at least the meta data of the solution stored during the solution registration gives enough information of the context in which the API is being used to the API exposure platform.
• API authentication/authorization: the developer and or the solution have to be authenticated in order to be allowed to use an API (may be only at scale). This may actually be dependent on the business model and at which stage of development the solution is. A more complex aspect that also needs to be handled is the authentication of the user that has downloaded the application or the consents that user has given for a solution to act on behalf of the user.
• API description and usage management: is the facility to make the understanding and access of the API easy and quick: hypertext documentation, code repositories (with optionally the associated binaries), sandbox to understand the API by trying it, are tools that are been used to make the API successful. Based on the state of the solution that consumes API/based on the developer, throttling rules may be applied on the API calls for either protecting the implementation or avoiding misuse of the API.
• API business model: is the commercial part of the API exposure since it describes what are the commercial conditions to use an API like:
• usage volume limit (per time period for example),
• usage volume before contract,
• cost model on API, on assets,
• revenue share if any.
There are many business models that have been already described by John Musser.
• API analytics: provides that ability to feed many other functionality (like the business model) but also is a way to adapt the APIs to the demand by understanding the patterns of usage or non usage. API exposure can become a important source of information since it will describe the popularity of the business assets.
• API exposure state:while the API implementation can be at different states of development (simulated (downloadable code), simulated end point, alpha, beta...), the exposure itself can be at different states. For example the API may not have a business model, or the API may be just a specification. It is very important to explicitly and independently describe the state of development of the API implementation and the state of the API exposure to the developer.
• API for management: is a new aspect of what API exposure platform should enforce. It represents the different management aspects an API should have in order to be able to be used in enterprise grade solution. Normalization around common management semantic, ability to assess the health of the API are example of concepts an API must have to be part of a controlled environment. This concept is not new, and in distributed systems (network for example) there are models like FCAPS to cover the different attributes network elements need to have in order to make a network, composed of a large variety of these elements, manageable. A similar problem exists on API and while this may influence the API implementation, this can be enforced/facilitated by the API exposure platform.

All of these operations can be done independently from what the API semantic is and therefore can be handled by a single platform. However due to the inherent complexity and lack of flexibility of the some of the implementation platforms, which makes changes difficult or expensive to do, real implementations generally lead to API transformations which may leak in the exposure environment. These transformations must be clearly decoupled from the core functionality since it may create scalability and maintenance issues on the API exposure platform itself and ultimately should be migrated into the proper environment. Here is a list of transformations that is addressed by the API exposure platform and the environment in which the transformations should happen:

• API adaptation (implementation): often the API is not ready to be exposed as is and needs to be adapted to fit more the consumer demand. Protocol adaption (SOAP to REST for example), resource adaptation (grouping multiple methods) are examples of type of adaptations that can be performed on APIs before exposure
• API customization (consumption): dealing with a large partner may lead to a negotiation between different parties in order to define what will be the information exchange and how it will happen. An API customization may need to happen in order to comply to the negotiated interfaces between the different parties.
• API specialization (consumption): while an API should as much as possible be agnostic to the API consumers (that API can then be called a wholesale API), it may be necessary to specialized the API in order to fit the specific requirements of an API consumer (that API is then called a retail API). Protocol optimization, usage of legacy protocols, message grouping to minimize chattiness are examples of activities to specialized a wholesale API to become a retail API.
• API profiling(implementation): an interesting case of transformation is modifying the response of an the API based on the level of the developer/application. For example a bronze level developer may have access to different precision than a gold developer when using a location API. While the definition of the levels is in the exposure domain this transformation of the API should be in the implementation domain but often leaks in the exposure domain.
• API versioning (implementation): it is important not to hide versioning and used the proper design pattern as described by Brian Mulloy. From a system point of view the versioning should always be addressed by the implementation environment when it is a significant change but also when it is a revision which changes the behavior of the API. A policy based revision may be handled in the exposure environment.

API exchange

By providing an intermediary endpoint to the real endpoint of the implementation, the API exposure platform allows a more dynamic bindings between the API consumer and the API implementation itself. This property allows the API consumer not to be affected even if the conditions at the API implementation level may have changed.

However, there are conditions where it may be necessary to set up an API context at the API consumer level in order to make sure that the correct API is being called. This is particularly useful when a group of companies have agreed on a specific API semantic and instead of having to create an API aggregator which will act as the endpoint for the API instead of each member of the group, a API exchange (a good example has been implemented by Apigee) is being used in order to implicitly or explicitly make sure that the correct API of one member of the group is being discovered and called.

• explicit discovery: very much like DNS, the API consumer makes a call to the API exchange and as a return receives a context  containing the specific API from one of the group member to call.
• implicit discovery: the API consumer makes a call to the API of one of the group members. The group member cannot successfully answer the call, calls the API exchange and redirects the API consumer to the proper API from another group member based on the response of the API exchange.

The API exchange works between group members that have commonly agree on a specific standard either on the functional APIs or more probably on the APIs for management. 

Other aspects that are resolved by the API exchange are:

• the settlement aspects between the members of the group in case of cross billing handling,
• the developer term and conditions which will actually allow either the API discovery and/or the API redirection,
• part of the user consent to allow restricted information to be used by the API consumer.

The API exchange can either be defined as a stand alone service (like DNS) or could be defined as an add-on of the API exposure platform. 

Conclusion

Many companies are jumping in the exposure environment either because they are already covering specific aspects of the API exposure or because it may be a natural evolution of their existing products. However the tendency to merge  different environments like implementation and exposure or exposure and consumption, in order to grab a bigger part of the API supply chain, leads to over-complex systems that look more like a new silo than a proper architecture and limit the richness of each of the merged environment. However the worse today is to merge implementation and consumption to create tighly coupled end to end solutions not allowing the creation of a rich ecosystem of solutions on top of a platform business, expressed by a serie of exposed APIs, mandatory to any modern business.

API analytics is a core aspect of the exposure environment and should always be treated as a priority since:

• it allows measurement  for defining success,
• it acts as a tools to refined the API and the business models to fit with the demand,
• it prioritizes the core assets for implementation improvements,
• it can become a source of revenue in indirect business models,

but to arrive at full maturity, an increased focus on API for management is necessary.

Creating a proper API exposure platform that:

• enables the creation of a rich ecosystem of internal/external solutions that consume exposed API,
• allows a rich ecosystem of API implementations, representing the core assets of a business, to be used,

has become, in the digital world, the key for the success of any business and is the core transformation mechanism for the struggling incumbent businesses.