LOG-9336: prevent "Too many open files" by raising FD limits

[vparfonov]

Adds a three-layer defense against file descriptor exhaustion:

1. Auto-raise RLIMIT_NOFILE soft limit to hard limit at startup, with macOS kern.maxfilesperproc fallback for RLIM_INFINITY hard limits.
2. New max_open_files config for the file source with LRU eviction — when the limit is reached, the least recently read file is closed (checkpoint preserved, no data loss).
3. Auto-derive a sensible default (80% of OS soft limit) so the feature works out of the box without manual configuration.

[openshift-ci-robot]

@vparfonov: This pull request references LOG-9336 which is a valid jira issue.

Details

In response to this:

Adds a three-layer defense against file descriptor exhaustion:

1. Auto-raise RLIMIT_NOFILE soft limit to hard limit at startup, with macOS kern.maxfilesperproc fallback for RLIM_INFINITY hard limits.
2. New max_open_files config for the file source with LRU eviction — when the limit is reached, the least recently read file is closed (checkpoint preserved, no data loss).
3. Auto-derive a sensible default (80% of OS soft limit) so the feature works out of the box without manual configuration.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[jcantrill]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jcantrill, vparfonov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [jcantrill]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@vparfonov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/cargo-clippy-check	21c1f21	link	false	/test cargo-clippy-check
ci/prow/unit	21c1f21	link	true	/test unit

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.