An application network connects services across sites as if they were running together.

Skupper solves multi-cluster communication challenges through something called a Virtual Application Network or just application network. To understand the value of Skupper, it is helpful to first understand what an application network is.

An application network connects the applications and services in your hybrid cloud into a virtual network so that they can communicate with each other as if they were all running in the same site. In this diagram, an application network connects three services, each of which is running in a different cloud:

In essence, the application network connects the services in a distributed application with a microservice architecture.

Application networks are able to provide connectivity across the hybrid cloud by using Skupper routers.

Skupper routers carry application traffic between services in different sites.

Skupper routers form the backbone of an application network in the same way that conventional network routers form the backbone of a VPN. However, instead of routing IP packets between network endpoints, Skupper routers route messages between application endpoints.

Application services

An application service represents an endpoint, or destination in the application network. When an application sends a communication to an service, the Skupper routers distribute the communication to any other application in the application network that has the same service.

For example, in this diagram, Service B sends a message with an application service to its local application router. Service A and Service C are subscribed to the same service, so the application router routes copies of the message through the application network until they arrive at each destination.

In the diagram, skupper-router is a container or systemd process that acts as a software router for service communication.

Application networks provide multiple routing patterns, so communications can be distributed in anycast (balanced or closest) or multicast patterns.

Skupper is the tool that creates and manages application networks across sites.

Skupper is an open source tool for creating application networks in Kubernetes or Linux. By using Skupper, you can create a distributed application consisting of microservices running in different sites.

This diagram illustrates a Skupper network that connects three services running in three different sites:

In a Skupper network, each namespace contains a Skupper instance. When these Skupper instances connect, they continually share information about the services that each instance exposes. This means that each Skupper instance is always aware of every service that has been exposed to the Skupper network, regardless of the namespace in which each service resides.

Once a Skupper network is formed across Kubernetes namespaces, any of the services in those namespaces can be exposed (through annotation) to the Skupper network. When a service is exposed, Skupper creates proxy endpoints to make that service available on each namespace in the Skupper network.

A multi-key listener provides per-service control over load balancing and failover by binding a single endpoint to multiple routing keys (connectors). This is the recommended approach for most use cases because it offers:

For configuration details and examples, see Creating a multi-key listener using YAML.

Link cost is a configurable integer value that influences how Skupper routes traffic across all services that traverse a link between two sites. The routing algorithm favors paths with the lowest total cost from client to target server.

📌 NOTE Link cost applies to all services on a link and cannot be set differently for individual services. For per-service control, use a multi-key listener instead.

Understanding link cost behavior

Using link cost for failover

You can configure link cost so that a primary location handles all traffic until failure, then traffic fails over to a backup location. To achieve this, set the cost from the client to the backup server very high, for example 99999:

In this configuration, all connections are routed to the local server. If the local server becomes unavailable, traffic fails over to the remote server regardless of the high cost.

📌 NOTE Skupper does not provide orchestrated failover for stateful applications that require control over the order in which traffic is redirected. You must implement that orchestration separately.

For details on configuring link cost, see:

Installing the skupper command-line interface (CLI) provides a simple method to get started with Skupper.

Procedure

Use the Skupper CLI to create a site on Kubernetes from the current namespace context.

Prerequisites

Procedure

Create a highly available Skupper site on Kubernetes by enabling HA mode in the CLI.

You can create a site that is highly available by using the ha option. High availability mode is intended to maintain service continuity during router restarts or pod rescheduling, but it does not provide failover if network connectivity between sites is lost. High availability mode deploys two router pods with anti-affinity rules to ensure service continuity during node failures.

Prerequisites

Procedure

Delete a Skupper site on Kubernetes by using the CLI from the namespace where the site was created.

Prerequisites

Procedure

A token provides a secure method to link sites. By default, a token can only be used once and must be used within 15 minutes to link sites. This procedure describes how to issue a token from one site and redeem that token on another site to create a link.

Prerequisites

To link sites, you create a token on one site and redeem that token on the other site to create the link.

Procedure

There are many options to consider when linking sites using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

An alternative approach to linking sites using tokens is to create a link resource YAML file using the CLI, and to apply that resource to another site.

Prerequisites

To link sites, you create a link resource YAML file on one site and apply that resource on the other site to create the link.

Procedure

There are many options to consider when linking sites using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

If your network requires routing through an HTTP CONNECT proxy to reach remote sites, you can configure Skupper links to use a proxy. This feature is only available when using link resources, not tokens.

Prerequisites

To link sites through a proxy, you create a Secret containing the proxy configuration, generate a link resource YAML file, reference the proxy Secret in the link settings, and apply that resource to create the link.

Procedure

apiVersion: skupper.io/v2alpha1
kind: Link
metadata:
  name: link-to-remote-site
spec:
  cost: 1
  endpoints:
  - host: remote-site.example.com
    name: inter-router
    port: "55671"
  - host: remote-site.example.com
    name: edge
    port: "45671"
  tlsCredentials: link-to-remote-site
  settings:
    proxy-configuration: my-proxy-config

where my-proxy-config is the name of the Secret created in step 4. . On the linking site, apply the link resource YAML file to create the link:

+

kubectl apply -f link.yaml

All inter-site traffic is protected by mutual TLS and routed through the HTTP CONNECT proxy tunnel. You can now expose services on the application network.

There are many options to consider when linking sites using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

Link cost is a configurable integer value that influences how Skupper routes traffic across links between sites. The routing algorithm favors paths with the lowest total cost from client to target server.

📌 NOTE For most load-balancing and failover use cases, a multi-key listener provides per-service control. Link cost applies to all services that traverse a link; it is not possible to set different costs for distinct services on the same link.

Understanding link cost behavior:

The following procedure describes how to set link cost in various scenarios:

Procedure

Additional information

A common use case for link cost is automatic failover. You can configure a primary site with an effective cost of 0 (local) and a backup site with a high link cost, for example 99999:

In this configuration, all connections are routed to the local server. If the local server becomes unavailable, traffic fails over to the remote server regardless of the high cost.

📌 NOTE Skupper does not provide orchestrated failover for stateful applications that require control over the order in which traffic is redirected. You must implement that orchestration separately.

For per-service failover or weighted traffic distribution, use a multi-key listener instead.

A connector binds a local workload to listeners in remote sites. Listeners and connectors are matched using routing keys.

For more information about connectors, see Content from skupperproject.github.io is not included.Connector concept.

Procedure

There are many options to consider when creating connectors using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

Additional information

If you need to expose a service from another namespace, you must use YAML as described in Creating a connector for a different namespace using YAML.

A listener binds a local connection endpoint to connectors in remote sites. Listeners and connectors are matched using routing keys.

For more information about listeners. see Content from skupperproject.github.io is not included.Listener concept.

Procedure

There are many options to consider when creating connectors using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

You can use YAML to create and manage Skupper sites.

Prerequisites

By default, the router CPU allocation is BestEffort as described in Content from kubernetes.io is not included.Pod Quality of Service Classes, and this might affect performance under network load. Consider setting resources as described in Setting site resources.

There are many options to consider when creating sites using YAML, see the Content from skupperproject.github.io is not included.YAML Reference, including frequently used options.

Procedure

Use the ha option to create a highly available site on Kubernetes.

High availability mode is intended to maintain service continuity during router restarts or pod rescheduling, but it does not provide failover if network connectivity between sites is lost. High availability mode deploys two router pods with anti-affinity rules to ensure service continuity during node failures.

Prerequisites

Procedure

You can configure the Skupper Router and Kube Adaptor components with minimum and maximum CPU and memory resources by defining sizing models using ConfigMaps.

You can define multiple sizing configurations using separate ConfigMaps. Only one ConfigMap should be annotated as the default (skupper.io/default-site-sizing: "true").

Prerequisites

Procedure

Use AccessGrant and AccessToken resources to create a link between two Kubernetes sites using YAML.

Prerequisites

AccessGrant is a permission on a listening site that allows redemption of access tokens to create links. The component it gives permission to is the GrantServer which is a HTTPS server that ultimately sets up the link.

The GrantServer provides a URL, a secret code, and a cert that are bundled together to form an AccessToken. The number of times an AccessToken can be redeemed and how long it remains active are both configurable. On OpenShift, the GrantServer is exposed by a Route, while other systems use a LoadBalancer to make it accessible.

AccessToken is short-lived, usually single-use credential that contains the AccessGrant URL, secret code and a cert to establish a secure connection to the GrantServer. A connecting site redeems this token for a Link resource to establish a link to the listening site.

Procedure

Link cost is a configurable integer value that influences how Skupper routes traffic across links between sites. The routing algorithm favors paths with the lowest total cost from client to target server.

📌 NOTE For most load-balancing and failover use cases, a multi-key listener provides more predictable, per-service control than link cost. Link cost applies to all services that traverse a link; it is not possible to set different costs for distinct services on the same link.

Understanding link cost behavior:

Link cost is set on the Link custom resource via spec.cost. The AccessToken resource also exposes spec.linkCost, which is applied to the Link created when the token is redeemed.

Prerequisites

Procedure

Option A — Set cost when redeeming a token

Add linkCost to the AccessToken resource before applying it. When the token is redeemed, the resulting Link is created with that cost.

apiVersion: skupper.io/v2alpha1
kind: AccessToken
metadata:
  name: my-token
spec:
  url: <grant-url>
  code: <secret-code>
  ca: <ca-cert>
  linkCost: 2

Apply it:

kubectl apply -f token.yaml

Option B — Update cost on an existing link

Verifying link cost

Check the cost of a specific link:

kubectl get link west-6bfn6 -o yaml

Look for spec.cost in the output. Alternatively, for a summary:

kubectl get links

Additional information

A common use case for link cost is automatic failover. You can configure a primary site with an effective cost of 0 (local) and a backup site with a high link cost, for example 99999:

In this configuration, all connections are routed to the local server. If the local server becomes unavailable, traffic fails over to the remote server regardless of the high cost.

📌 NOTE Skupper does not provide orchestrated failover for stateful applications that require control over the order in which traffic is redirected. You must implement that orchestration separately.

For per-service failover or weighted traffic distribution, use a multi-key listener instead.

A connector binds a local workload to listeners in remote sites. Listeners and connectors are matched using routing keys.

There are many options to consider when creating connectors using YAML, see Content from skupperproject.github.io is not included.Connector resource.

Procedure

A listener binds a local connection endpoint to connectors in remote sites. Listeners and connectors are matched using routing keys.

For more information about listeners. see Content from skupperproject.github.io is not included.Listener concept.

For configuration details, see Content from skupperproject.github.io is not included.Listener resource.

Procedure

A multi-key listener binds a single local host and port to multiple routing keys in remote sites. Use a multi-key listener when you want one service endpoint to aggregate traffic from multiple connectors.

With multi-key listeners, you must choose a strategy which determines how the traffic is distributed:

📌 NOTE Multi-key listeners select between routing keys using the configured strategy. Each routing key may have multiple connectors, and link cost determines which connector is used within each routing key. The two mechanisms are independent.

For configuration details, see Content from skupperproject.github.io is not included.MultiKeyListener resource.

Prerequisites

Procedure

A connector binds a local workload to listeners in remote sites.

If you create a site in one namespace and need to expose a service in a different namespace, use this procedure to create an attached connector in the other namespace and an AttachedConnectorBinding in the site namespace.

For configuration details, see Content from skupperproject.github.io is not included.Connector resource.

Procedure

Installing the skupper command-line interface (CLI) provides a simple method to get started with Skupper.

Procedure

Use the Skupper CLI to create a site on a local system.

Prerequisites

By default, all sites are created with the namespace default. On non-Kubernetes sites, you can create multiple sites per-user by specifying a namespace, for example:

skupper site create systemd-site -p linux -n linux-ns
skupper site create docker-site -p docker -n docker-ns

Procedure

Delete a Skupper site on a local system by using the CLI.

Prerequisites

Procedure

Create a site bundle when you want to prepare a site on one system and install it on a remote host.

Sometimes, you might want to create all the configuration for a site and apply it automatically to a remote host. To support this, Skupper allows you create a .tar.gz file with all the required files and an install.sh script to start the remote site.

Prerequisites

Procedure

A token lets a local system site link securely to a Kubernetes site.

A token provides a secure method to link sites. By default, a token can only be used once and must be used within 15 minutes to link sites. This procedure describes how to issue a token from a Kubernetes site and redeem that token on a local system site to create a link.

Prerequisites

To link sites, you create a token on the Kubernetes site and redeem that token on the local system site to create the link.

There are many options to consider when linking sites using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

Procedure

An alternative approach to linking sites using tokens is to create a link resource YAML file using the CLI, and to apply that resource to another site.

Prerequisites

To link sites, you create a link resource YAML file on one site and apply that resource on the other site to create the link.

There are many options to consider when linking sites using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

Procedure

A connector binds a local workload to listeners in remote sites. Listeners and connectors are matched using routing keys.

For more information about connectors see Content from skupperproject.github.io is not included.Connector concept

Prerequisites

There are many options to consider when creating connectors using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

Procedure

A listener binds a local connection endpoint to connectors in remote sites. Listeners and connectors are matched using routing keys.

Prerequisites

There are many options to consider when creating listeners using the CLI, see Content from skupperproject.github.io is not included.CLI Reference, including frequently used options.

Procedure

You can use YAML to create and manage Skupper sites.

Prerequisites

There are many options to consider when creating sites using YAML, see Content from skupperproject.github.io is not included.YAML Reference, including frequently used options.

Procedure

Create a link resource YAML file and apply it to the local system site to establish a link.

An alternative approach to linking sites using tokens is to create a link resource YAML file using the CLI, and to apply that resource to another site.

Prerequisites

To link sites, you create a link resource YAML file on one site and apply that resource on the other site to create the link.

Procedure

A connector binds a local workload to listeners in remote sites. Listeners and connectors are matched using routing keys.

For more information about connectors see Content from skupperproject.github.io is not included.Connector concept. For configuration details, see Content from skupperproject.github.io is not included.Connector resource.

Procedure

A listener binds a local connection endpoint to connectors in remote sites. Listeners and connectors are matched using routing keys.

For more information about listeners, see Content from skupperproject.github.io is not included.Listener concept. For configuration details, see Content from skupperproject.github.io is not included.Listener resource.

Procedure

A multi-key listener binds a single local host and port to multiple routing keys in remote sites. Use a multi-key listener when you want one service endpoint to aggregate traffic from multiple connectors.

With multi-key listeners, you must choose a strategy which determines how the traffic is distributed:

Multi-key listeners provide predictable traffic distribution from the client side and typically are not influenced by link costs.

For configuration details, see Content from skupperproject.github.io is not included.Listener resource.

Prerequisites

Procedure

Currently the primary purpose of the Network Observer is to provide a console for monitoring your application network. This section describes advanced configuration.

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: network-observer-memory-alert
  namespace: west
spec:
  groups:
  - name: network-observer.rules
    rules:
    - alert: NetworkObserverHighMemory
      expr: |
        (container_memory_working_set_bytes{namespace="west", container="network-observer"}
        /
        kube_pod_container_resource_limits{namespace="west", container="network-observer", resource="memory"}) > 0.9
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: "Network Observer pod in namespace {{ $labels.namespace }} is using > 90% of its memory limit."
        description: "Pod {{ $labels.pod }} is currently using {{ $value | humanizePercentage }} of its memory limit."

<a></a>

Priority: secretName > openshiftIssued > skupperIssued

Methods:

<a></a>

Notes:

<a></a>

Available flags:

Example:

extraArgs:
  - -flow-record-ttl=1h
  - -vanflow-logging-profile=minimal

<a></a>

The Prometheus container supports custom configuration and tuning options.

scrape_configs:
  - job_name: 'network-observer'
    static_configs:
      - targets: ['localhost:8080']

prometheus:
  extraArgs:
    - --storage.tsdb.retention.time=15d
    - --storage.tsdb.retention.size=10GB

prometheus:
  extraVolumes:
    - name: extra-config
      configMap:
        name: prometheus-rules
  extraVolumeMounts:
    - name: extra-config
      mountPath: /etc/prometheus/rules

By default, Prometheus uses ephemeral storage (emptyDir). For persistent time-series data, enable a PersistentVolumeClaim.

Resource requests and limits can be configured for each container in the Pod.

containerResources:
  networkObserver:
    requests:
      cpu: 200m
      memory: 512Mi
    limits:
      cpu: 1000m
      memory: 1Gi
  prometheus:
    requests:
      cpu: 500m
      memory: 1Gi
    limits:
      cpu: 2000m
      memory: 4Gi
  proxy:
    requests:
      cpu: 50m
      memory: 64Mi
    limits:
      cpu: 200m
      memory: 256Mi

Standard labels (always present):

Example:

commonLabels:
  environment: production
  team: platform-engineering
  cost-center: "12345"

commonAnnotations:
  owner: "platform-team@company.com"
  docs: "https://wiki.company.com/network-observer"

podLabels:
  app.kubernetes.io/tier: monitoring

podAnnotations:
  prometheus.io/scrape: "false"

<a></a>

Notes:

<a></a>

The Network Observer serves Prometheus metrics on a dedicated HTTP listener, separate from the main API endpoint.

Listener Configuration:

Service:

Security: Cluster-internal only. Do not expose publicly.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: skupper-network-observer
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: network-observer
  endpoints:
    - port: metrics
      path: /metrics

scrape_configs:
  - job_name: 'network-observer'
    static_configs:
      - targets: ['skupper-network-observer-metrics.default.svc:9000']