Open Service Mesh — Microsoft’s SMI based Open Source Service Mesh Implementation
Microsoft’s Open Service Mesh is an SMI-compliant, lightweight service mesh being run as an open source project. Backed by service-mesh partners including HashiCorp, Solo.io, and Buoyant, Microsoft introduced the Service Mesh Interface last year with the goal of helping end users and software vendors work with the myriad choices presented by service mesh technology by providing a set of specification standards. OSM can be considered as a reference implementation of SMI, one that builds on existing service mesh components and concepts.
Open Service Mesh data plane is architecturally based on the Envoy proxy and implements the go-control-plane xDS v3 API. However, despite the fact that Envoy comes with OSM by default, using standard interfaces allows it to be integrated with other reverse proxies (compatible with xDS).
SMI follows in the footsteps of existing Kubernetes resources, like Ingress and Network Policy, which also do not provide an implementation where required interfaces to interact with Kubernetes are facilitated for providers to plug their products. The SMI specification instead defines a set of common APIs that allow mesh providers to deliver their own implementations. This means mesh providers can either use SMI APIs directly or build operators to translate SMI to native APIs.
With OSM, users can use SMI and Envoy on Kubernetes and get a simplified service-mesh implementation. The SMI ecosystem already has multiple providers like Istio, Linkerd, Consul Connect, now Open Service Mesh etc. some of them have implemented SMI compatibility using adaptors (Istio, Consul Connect) and others (OSM, Linkerd etc.) consume the SMI APIs directly.
OSM implementation is very similar to Linkerd which also directly consumes SMI APIs without any need for an adaptor like Istio, but one key difference is that OSM uses Envoy for its proxy and communication bus, whereas Linkerd uses linkerd2-proxy (rust based — lighter than Envoy).
Architecture & Components
OSM control plane comprise four core components. All these four components are implemented as a single controller entity (Kubernetes pod/deployment), this is much lighter in weight when compared with older versions of Istio where there are 4 control plane components (Istio-1.6 introduced istiod which unifies all the control plane components into one binary).
OSM Data Plane — Uses Envoy as reverse-proxy by default — similar to most other Service Mesh providers (Linkerd is unique in this case which uses ultralight transparent proxy written in Rust). While by default OSM ships with Envoy, the design utilizes interfaces (An interface type in Go is kind of definition. It defines and describes the exact methods that some other type must have), which enable integrations with any xDS compatible reverse-proxy. The dynamic configuration of all the proxies is handled by OSM controller using Envoy xDS go-control-plane.
Mesh Specification is a wrapper around the existing SMI Spec (traffic spec, traffic target, traffic split and traffic metrics). Service mesh interface implementation provides two different approaches for providers to consume SMI spec. One, using adapters: requires a separate entity to convert from the SMI APIs to provider specific configuration. Istio and Consul Connect use adapters. The second approach is to directly consume the SMI API’s alongside the controller implementation where there is no need for an extra layer. Open Service Mesh, Linkerd, Maesh, Meshery etc. use this approach.
Mesh Catalog is the central component of OSM. MC interacts with all other components gathering individual outputs and provides an aggregated structure which are then transformed into proxy configuration. The proxy configuration is sent to all listening/registered proxies via the proxy control plane.
The Certificate Manager entity provides unique X.509 certificates for individual proxies. A Kubernetes service object is the core entity referenced in SMI spec for enforcing traffic policies. All traffic policies are enforced by the target service’s proxy, all the endpoints under a service are augmented with a proxy. Each service will be provided with a service certificate which will be used for service-proxy control plane mTLS connectivity.
The component proxy fronts a mesh service process (container or binary running on Kubernetes or a VM), All proxies establish an mTLS gRPC connection to the Proxy Control Plane (OSM Controller) and the controller using Envoy xDS go-control-plane implementation sends configuration updates using xDS protocol buffers.
OSM relies on SMI spec to reference services that will participate in the service mesh. OSM ships out-of-the-box with all necessary components to deploy a complete service mesh spanning multiple compute platforms.
SMI Spec — Overview
Service Mesh Interface (SMI) specification includes four API objects: traffic spec, traffic target, traffic split, and traffic metrics. Each API object is extended as a Kubernetes CRD (Custom Resource Definition).
A detailed explanation on SMI Spec and Istio SMI implementation is provided in the article below:
OSM Control Plane — Kubernetes
The OSM Command Line Interface (CLI) installs the OSM control plan into Kubernetes using this Helm chart. Under the hood, ‘osm’ uses Helm libraries to create a Helm ‘release’ object in the control plane Namespace. The Helm `release` name is the mesh-name (this name is used to label namespaces for enabling sidecar injection ). The ‘helm’ CLI can also be used to inspect Kubernetes manifests installed in more detail.
The whole ecosystem is controlled by a single controller and other components constitute CRD’s, monitoring and telemetry (Prometheus, Grafana and Zipkin). All the components are deployed in osm-system namespace.
OSM controller and related components:
Open Service Mesh controller initializes all the required informers (The client-go package comes with a subpackage that makes getting events easy: k8s.io/client-go/tools/cache. It allows users to easily add functions that will be called when certain events come in. It also allows users to store all of the objects in memory easily which is called a Store) to watch for creation/changes on the current state of Kubernetes objects.
Open Service Mesh controller initializes informers related to the SMI components (TrafficTarget, TrafficSplit, TrafficSpec etc.). This enables the controller to identify and process all the SMI specifications on the cluster. As the logic is embedded in the controller there is no need for a separate adaptor (like Istio) to translate the SMI spec.
Open Service Mesh uses mTLS for encryption of data between pods as well as Envoy and service identity. Certificates are created and distributed to each Envoy proxy via the SDS (Secret Discovery Service) protocol by the OSM control plane. Open Service Mesh supports 3 methods of issuing certificates: using an internal OSM package, called Tresor (This is the default for a first time installation), using Hashicorp Vault and Azure Key Vault.
OSM ecosystem requires two types of certificates:
- Certificates used for Envoy proxies to connect to xDS control plane — identifies the proxy and pod connecting to xDS.
- Certificates used for service to service communication (one Envoy connects to another) — identifies the services connecting to each other.
A Prometheus deployment is created alongside the controller to constantly poll for OSM related metrics — Control Plane and Data Plane (service-service, pod-service). It helps gather timing data needed to troubleshoot latency problems in service architectures using a specific trace_id.
A Zipkin deployment is created in the osm-system namespace which can be used for distributed tracing.
A Grafana deployment is created alongside Prometheus to visualize the metrics collected by Prometheus. OSM project provides default dashboards stored in a configmap (osm-grafana-config).
Required config maps are created in osm-system namespace which include the global OSM config, grafana config along with dashboard definitions and datasources and prometheus configuration.
Config Map ‘osm-config’ stores the global configuration used by the OSM controller.
Required secrets are created in osm-system namespace. These include the certs required for mutating webhook (for sidecar injection). The helm secret stores the release information of the deployed OSM version.
All SMI CRD’s are created as part of control plane boot strap, these are all SMI specific CRD’s where there’s no specific configuration related to OSM and the controller implementation enables OSM to convert and process these configurations.
Along with the above CRD’s an experimental Backpressure/rate limits CRD is also created, this is an experimental extension of SMI. This enables users to dynamically respond to and control inbound traffic in cases of overload situations with downstream services or performance or capacity issues with the service proxy itself. This will be removed once it becomes part of SMI.
Proxy Sidecar — Injection
A MutatingWebhook is created with required certs to interact with Kube-API, this watches for namespaces with label “openservicemesh.io/monitored-by:<mesh_name>”, mesh_name refers to the configured mesh name using ‘ — mesh-name’ or the helm release name.
OSM Controller uses informers to derive all namespaces with the label and injects a sidecar to all pods in the namespace.
Controller receiving information on the namespaces and injecting a payload to all the pods with the init container spec and a osm-init job container which configures the proxy container:
A sample container with proxy sidecar:
OSM controller — Envoy uses ADS (Aggregated Discovery Services) to discover all the proxy injected services. ADS enables discovery services for endpoints (here proxy sidecars), clusters, routes, listeners and aggregate stream for all resource types.
Demo Application
Official OSM documentation provided users with a demo application (similar to Istio’s bookinfo application). This enables users to test all the features that can be implemented using OSM. The application comprise bookstore (v1 and v2), Bookbuyer and Bookthief and all the services can be access using their respective UI.
Each application is deployed in an individual namespace and the namespaces are labeled for sidecar injection.
Sample application stack across namespaces:
All application namespaces will have an associated Service and ServiceAccount which will be used for configuring the traffic spec. For example:
All the applications are injected with proxy sidecar:
OSM Controller — ADS (Aggregated Discovery Service) discovering all individual services.
OSM Controller — SDS (Secret Discovery Service) providing required certs for all individual services which is used for mTLS.
OSM use Envoy’s go-control-plane xDS v3 API to dynamically configure the proxy config.
Egress
In OSM environment enabling egress is done via a global setting (configured using osm-config config map in osm-system namespace). The setting is toggled on or off and affects all services in the mesh. Egress is enabled by default when OSM is installed.
The configuration requires the mesh CIDR ranges to be specified. The mesh CIDR ranges are the list of CIDR ranges corresponding to the pod and service CIDRs configured in the cluster. The mesh CIDR ranges are required with egress to prevent any traffic destined within the cluster from escaping out as egress traffic and to be able to enforce mesh traffic policies for egress traffic. All the in-mesh traffic is routed based on L7 traffic policies, egress traffic is routed differently and is not subject to in-mesh traffic policies.
For example, in the sample application below the services bookbuyer and bookthief constantly performs GET requests to github.com.
With the egress feature enabled along with the mesh_cidr_ranges configured both the services can reach github.com as shown below.
Setting the egress feature to false as shown below prevents all the traffic destined outside the mesh.
Services bookbuyer and bookthief failing to reach github.com with egress turned off:
Ingress
A service can expose HTTP or HTTPS routes outside the cluster using Kubernetes Ingress along with an ingress controller. Once an ingress resource is configured to expose HTTP routes outside the cluster to a service within the cluster, OSM will configure the sidecar proxy on pods to allow ingress traffic to the service based on the ingress routing rules defined by the Kubernetes Ingress resource.
By default, OSM configures HTTP as the backend protocol for services when an ingress resource is applied with a backend service that belongs to the mesh. A mesh-wide configuration setting in OSM’s osm-config ConfigMap enables configuring ingress with the backend protocol to be HTTPS. HTTPS ingress can be enabled by updating the osm-config ConfigMap. OSM today has official documentation providing steps to integrate solutions like Nginx ingress controller, Azure Application Gateway Ingress and Gloo API Gateway.
For example an ingress implementation using Nginx Ingress controller requires the configuration below:
A Nginx ingress controller deployed in the cluster provisioned with OSM’s root certificate to be able to authenticate the certificate presented by backend servers when using HTTPS ingress (root certificate is stored in ‘osm-ca-bundle’ secret in osm-system namespace).
A sample Ingress resource (HTTPS) configuration is as below:
Once the ingress resource is created OSM injects a sidecar and will configure the sidecar proxy on pods to allow ingress traffic to the service based on the ingress routing rules.
The ingress controller works (update/sync) the same way as it does in traditional Kubernetes environments.
Traffic Access Control (Allow/Deny Traffic)
Traffic access control has three concepts, each defined through TrafficTarget using three fields:
- Source: The source of the traffic, which is reflected in the specific Pod list (selected/grouped by a selector). Currently, it is only supported by the selector using ServiceAccount. Currently, it is not supported by resource selection (such as specifying Deployment and specifying Service).
- Destination: The target of the traffic is also reflected in the specific Pod list (selected/grouped by a selector). Currently, this also only supports using a selector with ServiceAccount.
- Route: Traffic specification (HTTPRouteGroup, TCPRoute), which is used to distinguish different access modes provided by Destination.
The first scenario is to configure a TrafficTarget and HTTPRouteGroup to enable communication between Bookbuyer and Bookstore-V1.
When no access policies are defined there is no increase in the counters as all the traffic by default is dropped between services without a policy.
As there is only one version of bookstore (v1) in the scenario above, all the traffic is destined to V1, so the traffic split is configured to send all the traffic to bookstore-V1 as shown below:
In the TrafficTargets a configuration is made to enable connectivity from ServiceAccount:bookbuyer to ServiceAccount:bookstore-v1 using source and destination as shown below:
HTTPRouteGroups contain two matching paths/routes that should be served by bookstore-v1:
With the configuration above the Bookbuyer service can access Bookstore-v1service and the same can be inferred by increasing counters. The Bookthief counters are not being increased as the ServiceAccount:bookthief is not included in the sources above:
Bookthief service unable to access the bookstore-v1:
Adding bookthief service account to the sources in the above configuration enables the service to access bookstore-v1.
With the configuration above both bookbuyer and bookthief can access bookstore which results in increase of counters on all three services as shown below:
Bookthief service logs showing 200 response on requests to the configured HTTPRouteGroups:
Traffic Splitting
Traffic Split resource is used to achieve a percentage split of traffic to different versions of a same application.
Traffic split constitutes three concepts: a root-service: point of traffic origin, backend-services: subset of a root-service and weights: controlling amount of traffic flow to each backend service.
In the scenario below a new version of service bookstore — bookstore-v2 is introduced and configuration is done such that the two services bookbuyer and bookthief can access the service bookstore and the traffic in bookstore is equally (50%) served by both the versions v1 and v2.
A new deployment added to namespace bookstore with associated service and a service account
As seen below the counters are not increased as all the traffic is configured towards bookstore-v1.
Configuring weights 50% for each service bookstore-v1 and bookstore-v2:
As seen below the counters start to increase as the traffic is split between the versions.
The split can be easily verified/monitored from Zipkin, as shown below using the serviceName:bookbuyer and spanName:bookstore it is seen that the requests are being distributed across versions v1 and v2.
The dependency graph showing the source services traffic (from bookbuyer and bookthief) being distributed between destination services (bookstore-v1 and v2):
Monitoring and Tracing
OSM comes pre-installed with Prometheus, Grafana and Zipkin. Open Service Mesh (OSM) generates detailed metrics for all services communicating within the mesh. These metrics provide insights into the behavior of services in the mesh helping users to troubleshoot, maintain and analyze their applications.
Prometheus facilitates gathering of consistent traffic metrics and statistics across all services in the mesh:
Grafana for visualizing Prometheus backend timeseries database. OSM provides pre-configured dashboards (service-service, workload-workload, service-workload).
Zipkin for APM and Tracing:
Using the SMI specification offers the promise of moving more easily to another SMI-configured service mesh and also eliminates the need for a significant configuration change when moving from mesh providers. As of today SMI doesn’t fit in scenarios where advanced configuration such as circuit breaking, location based load balancing etc. are required, but all these are possible if SMI is further developed to consume advances Envoy xDS APIs.
Microsoft is planning to hand over control of Open Service Mesh to the CNCF and this will help it to score points with the wider open-source community, which has been angered by Google’s decision not to do so with Istio (handing over Istio to Open Usage Commons).
