Growing memory usage in go-zenon

@aliencoder are you aware of the memory usage issue on go-zenon? We were discussing w/ Kaine this week. The znnd process consumes more and more memory until it fails. The process then restarts.

Kaine asked which version we are building znnd with. The znn-controller uses 1.18. He said we need to use 1.19.4.

I tried that and it did not fix the issue. You can see the memory usage of znnd over the last 48 hours. It climbs steadily. I have about 8 different nodes that ALL exhibit this same behavior. All the community members have this issue too. Many run a cron job to restart the znnd process every day.

The image might be difficult to see. The bottom line is the znnd process usage. It rises to 85% and then restarts. I have monit monitor the process to automatically restart it. It restarts about every 48 hours.

I think we need to look into this issue.

No, I’m not aware. I’ll try to finish first with the PR stuff and after that explore the go-zenon codebase in more detail. I think first we need a cicd pipeline to be able to build and test automatically znnd :grin:

1 Like

I was experiencing the same behaviour with my old server, but now has been running 4,5 weeks straight after I replaced it with the new one.

I’m still using your initial docker image. Node commit: 7e9c440f7e5e368efbade77ca900efa1219af0d4.

Can we move discussions around go-zenon/znnd to a separate thread @CryptoFish @0x3639? Maybe @mehowbrainz can help us migrate the replies.

1 Like


what happens if you select 7 days for the duration?

Screenshot 2023-04-01 at 6.39.21 PM

ya, same behavior as mine. Looks like Monit is restarting znnd at 85% memory usage.

Let’s implement pprof tool and see exactly where the bottleneck is.

Oke, I thought it would result in a reboot of the pc. Good to know.

1 Like

Started the implementation. I’ll keep you updated with the progress.

1 Like

You can find the implementation here.

Download or build znnd from source and start it (default address is http://localhost:6060):

./znnd --pprof

Download and install the pprof tool:

go install

Here is a step-by-step tutorial.

1 Like