The memory_limiter processor limits the memory consumption of pgpro-otel-collector.

As an example, let's limit collector memory consumption to 2 GiB with possible consumption spikes up to 410 MiB. The configuration in this case looks as follows:

processors:
  memory_limiter/metrics:
    check_interval: 1s
    limit_mib: 2048
service:
  pipelines:
    metrics:
      processors:
        - memory_limiter/metrics
        - batch/metrics
      ...

Note that memory_limiter is added before batch.

With this configuration, the collector may consume up to 20% (default) more memory due to the spike_limit_mib parameter.

Additionally, you can use the GOMEMLIMIT environment variable:

Using both these tools can limit the collector's memory consumption in critical situations — for example, when too many new objects appear in the database.

Now let's consider the limitations of the memory_limiter processor.

When the memory limit is reached, all new metric records are dropped, as indicated by the following collector message:

{
    "level": "warn",
    "ts": "2025-09-22T14:29:46.994+0300",
    "msg": "Memory usage is above soft limit. Refusing data.",
    "resource": {
        "service.instance.id": "f3abd937-782d-4e3d-89fe-8950d54b925e",
        "service.name": "pgpro-otel-collector",
        "service.version": "v0.4.0"
    },
    "otelcol.component.kind": "processor",
    "cur_mem_mib": 908
}

When the hard limit is reached, the following message appears in the collector logs:

{
    "level": "warn",
    "ts": "2025-09-22T14:29:49.970+0300",
    "msg": "Memory usage is above hard limit. Forcing a GC.",
    "resource": {
        "service.instance.id": "f3abd937-782d-4e3d-89fe-8950d54b925e",
        "service.name": "pgpro-otel-collector",
        "service.version": "v0.4.0"
    },
    "otelcol.component.kind": "processor",
    "cur_mem_mib": 1026
}

After this message, the collector triggers garbage collection. Note that additional GC calls create extra CPU load.

The memory_limiter processor does not provide absolute memory consumption guarantees. In some cases, the garbage collector may not keep up with memory cleaning due to high CPU load.

It is recommended to set the limit to 50% of the server's resources. For example, for 8 GiB it would be 4 GiB.