Kubernetes readiness probe endpoint returning 404

See original GitHub issue

There appears to be some change in behaviour for the Kubernetes-oriented readiness group endpoint on 2.3.2 compared to 2.3.1.

For a service that has no external dependencies (and only readinessState in the health group), the /actuator/health/readiness endpoint is returning a 404.

Configuration we are using:

management.server.port=9083
management.health.probes.enabled=true
management.endpoints.enabled-by-default=false
management.endpoint.info.enabled=true
management.endpoint.health.enabled=true
management.endpoint.health.show-details=always
management.endpoint.health.group.liveness.include=livenessState,diskSpace,refreshScope
management.endpoint.health.group.readiness.include=readinessState
management.endpoint.health.group.liveness.show-details=always
management.endpoint.health.group.readiness.show-details=always
management.endpoints.web.exposure.include=health

Expected Behaviour We expect this to just return 200 with { "status": "UP" }

Actual Behaviour

$ http http://localhost:9083/actuator/health/readiness
HTTP/1.1 404 Not Found

Full health call:

$ http http://localhost:9083/actuator/health
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: application/json
Date: Sat, 25 Jul 2020 06:27:55 GMT
Transfer-Encoding: chunked
{
    "components": {
        "discoveryComposite": {
            "components": {
                "discoveryClient": {
                    "description": "Discovery Client not initialized",
                    "status": "UNKNOWN"
                }
            },
            "description": "Discovery Client not initialized",
            "status": "UNKNOWN"
        },
        "diskSpace": {
            "details": {
                "exists": true,
                "free": 287311962112,
                "threshold": 10485760,
                "total": 499963174912
            },
            "status": "UP"
        },
        "livenessStateProbeIndicator": {
            "status": "UP"
        },
        "ping": {
            "status": "UP"
        },
        "reactiveDiscoveryClients": {
            "components": {
                "Simple Reactive Discovery Client": {
                    "description": "Discovery Client not initialized",
                    "status": "UNKNOWN"
                }
            },
            "description": "Discovery Client not initialized",
            "status": "UNKNOWN"
        },
        "readinessStateProbeIndicator": {
            "status": "UP"
        },
        "refreshScope": {
            "status": "UP"
        }
    },
    "groups": [
        "liveness",
        "readiness"
    ],
    "status": "UP"
}

This may relate to #22107.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:17
  • Comments:18 (8 by maintainers)

github_iconTop GitHub Comments

7reactions
bclozelcommented, Aug 1, 2020

This issue is now fixed in the 2.3.3 and 2.4.0 SNAPSHOTs.

I’ve carefully read the comments on this issue regarding the following surprising behavior: getting a 404 status on a configured health group, when no indicator is present. In this very case it’s arguably wrong, but we’re in a case of a regression. But some of you thought that

  1. a missing indicator in a group should fail the application at startup or
  2. that an empty group should disappear from the list of groups on the main endpoint.

The first alternative sounds nice, especially for detecting bad configurations. But it’s also likely to fail in perfectly valid cases. Your application could configure a group management.endpoint.health.group.custom.include=ping,redis and fail in a test environment where no redis instance is available. Because Spring Boot reacts to the environment, it’s expected to behave differently and adapt to the situation.

The second alternative is debatable. Right now our health groups support is auto-configured with the configuration properties and does not look into the application context to check for the existence of health indicators. We seem to all agree that a 404 response status is right in this case. Removing the group information would, in my opinion, make things less consistent as we wouldn’t know that a group has been configured. After all, a health group is just a way to wrap several indicators under the same name and customize its global health status - but health indicators are still dynamic.

After discussing that briefly with the team, we didn’t think that this needs to be changed. Note that this behavior exists since the introduction of the health groups feature. If you can make a stronger case for changing this, please create a dedicated issue and explain how this behavior is inconsistent or could lead to issues.

Thanks!

3reactions
OrangeDogcommented, Jul 29, 2020

want to be aware the groups exist, so we know we can add components to them with include ?

The API response is supposed to be for consumers of the API, not documenting configuration options for the developer. Like the rest of the actuator system, only endpoints that are currently available should be listed as available.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Spring Boot Actuator Kubernetes Probes returning 404
If deployed in a Kubernetes environment, actuator will gather the "Liveness" and "Readiness" information... Maybe this indicates that the probes ...
Read more >
Configure Liveness, Readiness and Startup Probes
This page shows how to configure liveness, readiness and startup probes for containers. The kubelet uses liveness probes to know when to ...
Read more >
Liveness and Readiness Probes in Spring Boot - Baeldung
The kubelet will use the readiness probe to determine when the application is ready to accept requests. More specifically, a pod is ready...
Read more >
liveness probe failed: http probe failed with statuscode: 404
Kubernetes makes sure the readiness probe passes before allowing a service to send traffic to the pod. If a readiness probe starts to...
Read more >
Kubernetes Liveness and Readiness Probes: How to Avoid ...
This will result in all pods being removed from the service routing. With no pods backing the service, Kubernetes will return HTTP 404,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found