Izek Chen

Jun 15, 2021

2 min read

Analyze mongodb3.x slow query log with filebeat and elasticsearch

The log format for MongoDB v.3.x compare with later version are very different. And it is not working with default filebeat pipeline. However, there is a way we can parse it

Default filebeat mongodb pipeline

The grok part below are not working in the old version of mongodb logs

log example:

pipeline:

Custom pipeline

We can copy the official pipeline and create a custom one with extra grok to easy parsing the data.

Added one grok and one convert to change the time_taken to interger

Modify filebeat module

add the custom pipeline to the mongodb module

Grafana + Elasticsearch graph

At the end, you can use the graph to quickly find out the slow query and monitor the trend

MongoDB Slow query determination

TBC