Health checks

This document provides a detailed overview of the health check mechanism implemented in the IT Ticketing Service. It leverages the Spring Boot Actuator module to expose critical information about the application's operational status. Understanding these endpoints is essential for monitoring, troubleshooting, and ensuring high availability.

For a broader overview of monitoring practices, see the Monitoring documentation.

Actuator Endpoints

The application utilizes Spring Boot Actuator to provide production-ready features. The core of our health monitoring strategy revolves around the endpoints it exposes. The spring-boot-starter-actuator dependency is included in our pom.xml to enable this functionality.

<!-- pom.xml -->
<!-- Spring Boot Actuator for health checks -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

By default, Spring Boot secures most actuator endpoints. We have explicitly configured which endpoints are accessible over the web in application.properties.

# src/main/resources/application.properties

# Actuator Configuration
# Exposes only the 'health' and 'info' endpoints via HTTP
management.endpoints.web.exposure.include=health,info
# Configures the health endpoint to always show full details
management.endpoint.health.show-details=always

This configuration exposes the following key endpoints:

• /actuator/health: Provides a detailed summary of the application's health. This is the primary endpoint used by automated monitoring systems.
• /actuator/info: Displays general application information, which can be configured to include build details, git commit information, and more.

Accessing these endpoints would typically be done via http://<host>:8080/actuator/health. For local development, refer to the Quick Start Guide.

Health Indicators

Spring Boot Actuator automatically provides and configures HealthIndicator beans based on the dependencies present in the classpath. For this application, the following key indicators are active:

• Database Connectivity (db): Enabled by spring-boot-starter-data-jpa and the mysql-connector-j driver. This indicator performs a simple validation query against the MySQL database configured via spring.datasource.url to ensure a valid and active connection. A failure here is a critical issue, indicating the application cannot persist or retrieve data.

• RabbitMQ Connection (rabbit): Enabled by spring-boot-starter-amqp. This indicator checks the connection to the RabbitMQ message broker defined by spring.rabbitmq.host. It verifies that the application can communicate with the broker, which is essential for its asynchronous messaging capabilities.

• Disk Space (diskSpace): A default indicator provided by Actuator. It monitors the disk space of the partition where the application is running. It will report a DOWN status if the free space drops below a configured threshold (default is 10MB), which helps prevent failures due to a full disk.

• Ping (ping): A simple indicator that always returns UP, serving as a basic check that the application context is loaded and responsive.

These auto-configured indicators provide a comprehensive, out-of-the-box view of the application's dependencies and environment without requiring any custom code.

Health Check Responses

The /actuator/health endpoint returns a JSON object that aggregates the status of all registered HealthIndicators. The overall status is determined by an ordered lookup, with DOWN taking precedence.

Status: UP

When all components are functioning correctly, the endpoint returns an HTTP 200 OK status and a JSON body with an overall status of UP. Because we have set management.endpoint.health.show-details=always, the response includes the status of each individual component.

Example UP Response:

{
  "status": "UP",
  "components": {
    "db": {
      "status": "UP",
      "details": {
        "database": "MySQL",
        "validationQuery": "SELECT 1"
      }
    },
    "diskSpace": {
      "status": "UP",
      "details": {
        "total": 499963174912,
        "free": 361093349376,
        "threshold": 10485760
      }
    },
    "ping": {
      "status": "UP"
    },
    "rabbit": {
      "status": "UP",
      "details": {
        "version": "3.12.12"
      }
    }
  }
}

Status: DOWN

If any critical health indicator reports a failure, the overall status becomes DOWN, and the endpoint returns an HTTP 503 Service Unavailable. This immediately signals to monitoring systems and load balancers that the application instance is unhealthy.

Example DOWN Response (Database is unavailable):

{
  "status": "DOWN",
  "components": {
    "db": {
      "status": "DOWN",
      "details": {
        "error": "org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain a JDBC Connection; nested exception is com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure"
      }
    },
    "diskSpace": {
      "status": "UP",
      "details": {
        "total": 499963174912,
        "free": 361093349376,
        "threshold": 10485760
      }
    },
    "ping": {
      "status": "UP"
    },
    "rabbit": {
      "status": "UP",
      "details": {
        "version": "3.12.12"
      }
    }
  }
}

Gotcha: The management.endpoint.health.show-details property is critical for debugging. While always is useful in development and trusted environments, production environments may restrict it to when-authorized to avoid leaking internal system details to unauthenticated users.

For detailed steps on resolving DOWN states, refer to the Troubleshooting Guide.

Integration with Monitoring

The /actuator/health endpoint is designed for seamless integration with modern infrastructure and monitoring tools.

• Kubernetes Liveness and Readiness Probes: This endpoint is ideal for configuring container probes.

  • Liveness Probe: A liveness probe can target /actuator/health. If the endpoint returns a non-2xx status (e.g., 503 when DOWN), Kubernetes will automatically restart the container, attempting to recover from a failed state.
  • Readiness Probe: A readiness probe can also target /actuator/health. If the application is not ready (e.g., during startup or if a dependency is down), Kubernetes will not route traffic to it, preventing user-facing errors.

  Example Kubernetes Probe Configuration:

  livenessProbe:
    httpGet:
      path: /actuator/health
      port: 8080
    initialDelaySeconds: 30
    periodSeconds: 15
  readinessProbe:
    httpGet:
      path: /actuator/health
      port: 8080
    initialDelaySeconds: 10
    periodSeconds: 5

• Load Balancer Health Checks: Cloud providers (AWS, GCP, Azure) and hardware load balancers can be configured to poll the /actuator/health endpoint. Unhealthy instances that return a non-200 status code will be automatically removed from the load balancing pool, ensuring traffic is only sent to healthy, fully functional application instances.

• Alerting on Health Status Changes: Monitoring systems like Prometheus (with Alertmanager), Datadog, or Splunk can be configured to periodically scrape the health endpoint. They can parse the JSON response and trigger alerts if the overall status changes to DOWN or if a specific component's health degrades. This enables proactive intervention before an issue impacts a wider audience. For more on our alerting strategy, see the Monitoring documentation.