Repository navigation
ValueError when getting open file descriptors limit #80
Description
Code in process_collectors.py relies on parsing /proc/PID/limits when gettings system limits. However, on some configurations, there could be "unlimited" string instead of a numeric value in that file. This lead to ValueError when this string is passed into float method.
Full example exception is as follows:
Traceback (most recent call last):
File "/venv/lib/python3.5/site-packages/tornado/web.py", line 1443, in _execute
result = method(*self.path_args, **self.path_kwargs)
File "/venv/lib/python3.5/site-packages/video/stats/events/endpoint/endpoint.py", line 194, in get
resp = generate_latest(core.REGISTRY)
File "/venv/lib/python3.5/site-packages/prometheus_client/exposition.py", line 33, in generate_latest
for metric in registry.collect():
File "/venv/lib/python3.5/site-packages/prometheus_client/core.py", line 54, in collect
for metric in collector.collect():
File "/venv/lib/python3.5/site-packages/prometheus_client/process_collector.py", line 83, in collect
'Maximum number of open file descriptors.', value=float(line.split()[3]))
ValueError: could not convert string to float: 'unlimited'
This happens in a Docker container with ubuntu, for instance, which has:
root@87220b5caf91:/app# cat /proc/76888/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited 9223372036854775807 bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 35184372088832 bytes
Max core file size unlimited 9223372036854775807 bytes
Max resident set unlimited 1073741824 bytes
Max processes unlimited 4000 processes
Max open files unlimited 65536 files
Max locked memory unlimited 1073741824 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 128 512 signals
Max msgqueue size unlimited 8192 bytes
Activity
brian-brazil commentedon May 17, 2016
Hmm, in that case we need to use the hard limit. I wonder if other languages have the same issue.
brian-brazil commentedon May 17, 2016
Also how is it possible for the soft limit to be higher than the hard limit? That sounds like a kernel bug.
anti1869 commentedon May 17, 2016
The system is Ubuntu 15.10 in Docker container on top of the Joyent's Smart Data Center. May be this behaviour is caused by their docker implementation.
brian-brazil commentedon May 17, 2016
I think you're going to need to ask them what this is meant to mean. This doesn't make sense to me from a Unix standpoint.
brian-brazil commentedon Nov 3, 2016
Without further information about what this is meant to mean, we can't do much here.
Ignore ValueErrors (prometheus#80).
jgysland commentedon Sep 21, 2017
I'm having the same issue, also on Joyent's implementation of docker.
address prometheus#80