Consul Service Discovery in Python — Core Concepts

Consul by HashiCorp is a service mesh and discovery tool that solves a fundamental problem in distributed systems: how do services find each other when instances are constantly being created, destroyed, and moved across hosts? For Python applications, Consul replaces hardcoded addresses with dynamic lookups and adds health monitoring as a bonus.

How service discovery works

The flow has three steps:

1. Registration. When a Python service starts, it tells Consul its name, address, port, and how to check its health.
2. Health checking. Consul periodically probes each registered service (HTTP endpoint, TCP connection, or script). Unhealthy services are excluded from queries.
3. Discovery. When service A needs to call service B, it queries Consul for healthy instances of B and gets back addresses.

This means no service ever needs to know the static address of another service. The directory is always current.

Using python-consul2

The python-consul2 library wraps Consul’s HTTP API:

import consul

c = consul.Consul(host="localhost", port=8500)

# Register a service with an HTTP health check
c.agent.service.register(
    name="payment-service",
    service_id="payment-1",
    address="10.0.0.5",
    port=8080,
    check=consul.Check.http("http://10.0.0.5:8080/health", interval="10s")
)

Discovery is equally straightforward:

# Get healthy instances of a service
_, services = c.health.service("payment-service", passing=True)

for svc in services:
    address = svc["Service"]["Address"]
    port = svc["Service"]["Port"]
    print(f"Available at {address}:{port}")

Health check types

Consul supports several check types:

• HTTP: Consul sends a GET request to a URL. Status 200-299 means healthy.
• TCP: Consul opens a TCP connection. Success means the port is open.
• Script/TTL: The service itself reports health periodically. Useful when you need custom logic.
• gRPC: Native gRPC health check protocol support.

You can combine multiple checks per service. A database proxy might have both a TCP check (port open?) and an HTTP check (can it actually query?).

Key-value store

Beyond service discovery, Consul includes a distributed key-value store. Python services use it for shared configuration:

# Store config
c.kv.put("config/payment/max_retry", "3")

# Read config
_, data = c.kv.get("config/payment/max_retry")
value = data["Value"].decode()

Teams use this for feature flags, database connection strings, and circuit breaker thresholds. Changes propagate to all services through Consul watches or polling.

DNS interface

Consul also provides a DNS interface. Services are available at {name}.service.consul:

$ dig payment-service.service.consul
→ 10.0.0.5
→ 10.0.0.12

Python services can use standard DNS resolution to discover other services without any Consul client library. This is useful for legacy applications that cannot be modified.

Common misconception

“Service discovery is only useful for large systems with hundreds of services.” Even two services benefit. The moment you deploy a service to a new host, change a port, or run multiple instances, hardcoded addresses become a liability. Consul eliminates that entire class of configuration bugs.

When to choose Consul

Consul fits best when you need a combined solution for service discovery, health checking, and configuration management. If you only need service discovery, lighter options like DNS-based solutions or environment variables might suffice. If you are already on Kubernetes, its built-in service discovery might be enough. Consul shines in hybrid environments (VMs plus containers) and multi-datacenter setups where Kubernetes does not span the full infrastructure.

One thing to remember: Consul gives Python services a live directory of healthy instances — register once, discover dynamically, and let health checks automatically route around failures.