Skip to content

HADOOP-19846: Increase ulimit nproc for Yetus.#8358

Open
ajfabbri wants to merge 2 commits intoapache:trunkfrom
ajfabbri:af/HADOOP-19846
Open

HADOOP-19846: Increase ulimit nproc for Yetus.#8358
ajfabbri wants to merge 2 commits intoapache:trunkfrom
ajfabbri:af/HADOOP-19846

Conversation

@ajfabbri
Copy link
Contributor

@ajfabbri ajfabbri commented Mar 19, 2026

Description of PR

How was this patch tested?

We will monitor CI and see if this improves the flakiness.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • [ na] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [ na] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [ na] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8358/1/console in case of problems.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 2s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 2m 10s Maven dependency ordering for branch
+1 💚 mvninstall 54m 9s trunk passed
+1 💚 mvnsite 0m 0s trunk passed
+1 💚 shadedclient 34m 43s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 32s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
-1 ❌ shellcheck 0m 2s /results-shellcheck.txt The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3)
+1 💚 shadedclient 33m 46s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
130m 11s
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8358/1/artifact/out/Dockerfile
GITHUB PR #8358
Optional Tests dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname Linux 8955621b8ecb 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b4ced42
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8358/1/testReport/
Max. process+thread count 590 (vs. ulimit of 10000)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8358/1/console
versions git=2.43.0 maven=3.9.11 shellcheck=0.9.0
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8358/2/console in case of problems.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 55s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 2m 3s Maven dependency ordering for branch
+1 💚 mvninstall 50m 49s trunk passed
+1 💚 mvnsite 0m 0s trunk passed
+1 💚 shadedclient 33m 27s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 47s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
+1 💚 shellcheck 0m 2s No new issues.
+1 💚 shadedclient 32m 58s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
124m 32s
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8358/2/artifact/out/Dockerfile
GITHUB PR #8358
Optional Tests dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname Linux 64f0838beb3f 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 72a2865
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8358/2/testReport/
Max. process+thread count 586 (vs. ulimit of 10000)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8358/2/console
versions git=2.43.0 maven=3.9.11 shellcheck=0.9.0
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

HADOOP_HOMEBREW_DIR=/usr/local
fi
#shellcheck disable=SC2034
PROC_LIMIT=10000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the default value if we don't set it explicitly?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, I figured it out by looking at Yetus reports in other PRs, it's 5500 by default. double it makes sense

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. Thanks for the review!

@ajfabbri ajfabbri marked this pull request as ready for review March 20, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants