First of all, what is Profiling?
Sometimes applications face unexpected runtime problems related to memory leaks or CPU / memory allocation performance. To analyze an application and see where the bottlenecks are, optimization opportunities or fix runtime problems we use a tool that exports metrics called Profiler. On large scale production level software, profiling is a necessary step.
Profiling in Go
Go has a standard tool for profiling called pprof, it’s fairly simple to use and even generates a web page with a tree view on the code runtime metrics. It’s great to use while developing to find problems and bottlenecks early, but sometimes it’s only after being deployed to production that those problems really show up.
Although it’s easy to use, there are some easier and more helpful ways of using profiling with “plug and play tools”. Google’s Cloud Profiler and Datadog’s Continuous Profiler are some really interesting and robust tools, but add a bigger cost, they have great UIs and visualization tools that really help you understand your application resource consumption on a method level. If budget is not your problem, they really are the way to go!
Prometheus, on the other hand, is an open source monitoring tool that was developed by SoundCloud and is being used by large companies like Digital Ocean. It has clients for various languages, including Go, and the best part is it makes it really easy to monitor applications and it’s very lightweight.
Observability
With all of the metrics on Prometheus, now I want to visualize them in a way that helps me really understand and see insights on performance and bottlenecks, that is where Grafana comes in play. It also has an open source version and helps create dashboards unifying data from various data sources, including Prometheus. To make this even simpler, there’s a dashboard that can be imported into Grafana that was made to read Prometheus metrics exported by the Go client
Code
I wrote a simple app to export profiling and custom metrics to Prometheus and made a docker-compose so you can run and check it out for yourself!
https://github.com/rafaelrubbioli/prometheus-go