I spent a week hunting a phantom memory leak just like this. Don’t trust Grafana dashboards to tell you the full story about your app’s memory usage!
Posting here to hopefully save others the headache.
Had this so many times with the JVM… The GCs are doing so much “magic” that sometimes it’s hard to wrap your head around it.
Having built a container solution from “scratch” I’m always suspicious of memory usage metrics I didn’t set up myself and verified. This is kind of thing is exactly what I’ve wanted to avoid.
I spent a week hunting a phantom memory leak just like this. Don’t trust Grafana dashboards to tell you the full story about your app’s memory usage!
Posting here to hopefully save others the headache.
Had this so many times with the JVM… The GCs are doing so much “magic” that sometimes it’s hard to wrap your head around it.
Having built a container solution from “scratch” I’m always suspicious of memory usage metrics I didn’t set up myself and verified. This is kind of thing is exactly what I’ve wanted to avoid.