Skip to content

fix forced-shutdown (double SIGINT) behaviour for JRuby 10#19017

Merged
yaauie merged 1 commit into
elastic:mainfrom
yaauie:fix-forced-shutdown-jruby10
Apr 21, 2026
Merged

fix forced-shutdown (double SIGINT) behaviour for JRuby 10#19017
yaauie merged 1 commit into
elastic:mainfrom
yaauie:fix-forced-shutdown-jruby10

Conversation

@yaauie
Copy link
Copy Markdown
Member

@yaauie yaauie commented Apr 17, 2026

Release notes

[rn:skip]

What does this PR do?

Fixes an issue introduced with the upgrade to JRuby 10 in which a doubled-SIGINT does not force an immediate shutdown because the SystemExit exception is caught and squashed by the default thread exception handler.

Why is it important/What is the impact to the user?

Sometimes our users need to force Logstash to shut down before it has finished processing the events in-flight. The upgrade to JRuby10 inadvertently removed this functionality, and we restore it here.

Checklist

  • My code follows the style guidelines of this project
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
  • I have added tests that prove my fix is effective or that my feature works

How to test this PR locally

  1. Invoke Logstash with intentional back-pressure:
    bin/logstash -e 'input { generator {} } filter { ruby { code => "sleep 1" } } output { sink {} }'
    
  2. Wait until processing, then send TWO SIGINT signals (CTRL+C in most shells)
  3. Observe that the process correctly bails immediately after the second SIGINT.

(without this patch, Logstash keeps processing until the in-flight events are done)

Related issues

Caused by: #18755

Logs

Without (note how it keeps executing after logging "Terminating Immediately"):

[2026-04-17T15:29:58,450][INFO ][logstash.agent           ] Pipelines running {count: 1, running_pipelines: [:main], non_running_pipelines: []}
^C[2026-04-17T15:30:02,168][WARN ][logstash.runner          ] SIGINT received. Shutting down.
^C[2026-04-17T15:30:02,946][FATAL][logstash.runner          ] SIGINT received. Terminating immediately..
[2026-04-17T15:30:02,947][ERROR][org.logstash.Logstash    ] uncaught exception (in thread SIGINT handler)
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:974)
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:933)
        at org.jruby.RubyThread.kill(org/jruby/RubyThread.java:1955)
[2026-04-17T15:30:07,184][WARN ][logstash.runner          ] Received shutdown signal, but pipeline is still waiting for in-flight events
to be processed. Sending another ^C will force quit Logstash, but this may cause
data loss.
[2026-04-17T15:30:07,283][ERROR][org.logstash.execution.ShutdownWatcherExt] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
[2026-04-17T15:30:07,283][INFO ][org.logstash.execution.ShutdownWatcherExt] The queue for pipeline main is draining before shutdown.
^C[2026-04-17T15:30:20,063][FATAL][logstash.runner          ] SIGINT received. Terminating immediately..
[2026-04-17T15:30:20,064][ERROR][org.logstash.Logstash    ] uncaught exception (in thread SIGINT handler)
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:974)
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:933)
        at org.jruby.RubyThread.kill(org/jruby/RubyThread.java:1955)

With (note how it actually terminates immediately after logging "Terminating Immediately"):

[2026-04-17T16:06:52,247][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id" => "main"}
[2026-04-17T16:06:52,256][INFO ][logstash.agent           ] Pipelines running {count: 1, running_pipelines: [:main], non_running_pipelines: []}
^C[2026-04-17T16:06:57,660][WARN ][logstash.runner          ] SIGINT received. Shutting down.
[2026-04-17T16:07:02,674][WARN ][logstash.runner          ] Received shutdown signal, but pipeline is still waiting for in-flight events
to be processed. Sending another ^C will force quit Logstash, but this may cause
data loss.
[2026-04-17T16:07:02,791][ERROR][org.logstash.execution.ShutdownWatcherExt] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
[2026-04-17T16:07:02,792][INFO ][org.logstash.execution.ShutdownWatcherExt] The queue for pipeline main is draining before shutdown.
^C[2026-04-17T16:07:04,711][FATAL][logstash.runner          ] SIGINT received. Terminating immediately..
[error: 1 (00:00:19)] 

@github-actions
Copy link
Copy Markdown
Contributor

🤖 GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
  • run exhaustive tests : Run the exhaustive tests Buildkite pipeline.

@elasticmachine
Copy link
Copy Markdown

💛 Build succeeded, but was flaky

Failed CI Steps

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 17, 2026

This pull request does not have a backport label. Could you fix it @yaauie? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
  • If no backport is necessary, please add the backport-skip label

Copy link
Copy Markdown
Contributor

@mashhurs mashhurs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed the behavior.
Nice catch.

private static void installGlobalUncaughtExceptionHandler() {
Thread.setDefaultUncaughtExceptionHandler((thread, e) -> {
if (e instanceof Error) {
if (e instanceof org.jruby.exceptions.SystemExit) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just nit: we can import org.jruby.exceptions.SystemExit on the top to keep the standard importing import org.jruby.exceptions.*

@yaauie yaauie merged commit 296f936 into elastic:main Apr 21, 2026
14 checks passed
@yaauie yaauie deleted the fix-forced-shutdown-jruby10 branch April 21, 2026 20:48
@yaauie
Copy link
Copy Markdown
Member Author

yaauie commented Apr 21, 2026

@Mergifyio backport 9.4

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 21, 2026

backport 9.4

✅ Backports have been created

Details

yaauie added a commit that referenced this pull request Apr 21, 2026
…19034)

(cherry picked from commit 296f936)

Co-authored-by: Rye Biesemeyer <yaauie@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants