Race condition in Py_FinalizeEx: _PyObject_Dump and "lost sys.stderr" when SIGINT arrives during _PyImport_Cleanup

[raminfp]

Bug description:

When SIGINT is delivered during interpreter shutdown (specifically during _PyImport_Cleanup in Py_FinalizeEx), CPython falls back to _PyObject_Dump and prints "lost sys.stderr". This happens because sys.stderr is set to None during module cleanup, but signals are not blocked during this critical phase.

The _PyObject_Dump fallback leaks raw heap and type object addresses to the process output:

object address  : 0x72fed827c940
object refcount : 6
object type     : 0xa3e620
object type name: KeyboardInterrupt
object repr     : KeyboardInterrupt()
lost sys.stderr

Users should never see _PyObject_Dump output under normal operation. This is a C-level internal debug function (Objects/object.c) that bypasses Python's I/O system entirely.

Python version

Python 3.12.7 (main, Jun 18 2025, 13:16:51) [GCC 14.2.0]
Linux 6.11.0-29-generic x86_64

How to reproduce

The reproducer is a single self-contained Python file. It only requires curl (available on all Linux systems).

Quick start

python3 reproduce_bug_curl.py

The bug typically reproduces on the first attempt.

Self-contained reproducer (reproduce_bug_curl.py)

Click to expand full reproducer code

#!/usr/bin/env python3
"""
Self-contained reproducer for CPython _PyObject_Dump bug during Py_FinalizeEx.

Bug: When SIGINT arrives during Py_FinalizeEx while sys.stderr is being cleared
in _PyImport_Cleanup, CPython falls back to _PyObject_Dump because
PyErr_Display sees sys.stderr=None.

The Py_FinalizeEx shutdown sequence:
  1. wait_for_thread_shutdown -> threading._shutdown -> _python_exit -> t.join()
  2. call_py_exitfuncs -> atexit handlers
  3. _PyImport_Cleanup -> sys.stderr becomes None
  4. If SIGINT arrives NOW -> PyErr_Display sees sys.stderr=None -> _PyObject_Dump

This reproducer uses only curl (no external tools needed).

Usage:
  python reproduce_bug_curl.py              # Run bug reproducer (default)
  python reproduce_bug_curl.py --worker     # Run in worker mode (called internally)
"""

import subprocess
import time
import os
import sys
import signal
import tempfile
import concurrent.futures


# ============================================================================
# IP ranges — generate enough concurrent work to create the thread-shutdown
# timing window needed for the bug
# ============================================================================

IP_RANGES = [
    "2.58.104.0/24",
]


# ============================================================================
# Worker code: curl-based HTTP scanner
# ============================================================================

def generate_ips(ip_ranges):
    """Generate list of IPs from CIDR ranges"""
    import ipaddress
    ips = []
    for cidr in ip_ranges:
        network = ipaddress.IPv4Network(cidr, strict=False)
        for ip in network.hosts():
            ips.append(str(ip))
    return ips


def scan_ip_curl(ip, timeout=5):
    """
    Scan a single IP using curl.
    Attempts HTTPS connection to the IP with a short timeout.
    The actual connection result doesn't matter — we just need blocking I/O
    in many threads to create the shutdown timing window.
    """
    try:
        result = subprocess.run(
            [
                "curl", "-s", "-o", "/dev/null",
                "-w", "%{http_code} %{time_total}",
                "--connect-timeout", str(timeout),
                "--max-time", str(timeout),
                "-k",  # allow insecure (self-signed certs)
                f"https://{ip}/"
            ],
            capture_output=True,
            text=True,
            timeout=timeout + 2,
        )
        output = result.stdout.strip()
        parts = output.split()
        if len(parts) >= 2:
            http_code = parts[0]
            time_total = float(parts[1])
            return {
                "ip": ip,
                "success": http_code not in ("000", ""),
                "time_ms": time_total * 1000,
                "message": f"HTTP {http_code} in {time_total*1000:.0f}ms",
            }
        return {"ip": ip, "success": False, "time_ms": -1, "message": f"curl: {output}"}
    except subprocess.TimeoutExpired:
        return {"ip": ip, "success": False, "time_ms": -1, "message": "Timeout"}
    except Exception as e:
        return {"ip": ip, "success": False, "time_ms": -1, "message": str(e)[:50]}


def worker_main():
    """
    Worker mode: scan IPs using curl with many threads.
    This creates the multi-threaded shutdown scenario needed for the bug.
    """
    ips = generate_ips(IP_RANGES)

    print(f"Worker started: {len(ips)} IPs, 100 workers", flush=True)

    with concurrent.futures.ThreadPoolExecutor(max_workers=100) as executor:
        future_to_ip = {
            executor.submit(scan_ip_curl, ip, 3): ip
            for ip in ips
        }

        for i, future in enumerate(concurrent.futures.as_completed(future_to_ip)):
            ip = future_to_ip[future]
            try:
                result = future.result()
                status = "+" if result["success"] else "x"
                print(f"[{i+1}/{len(ips)}] {status} {ip}: {result['message']}", flush=True)
            except Exception as e:
                print(f"[{i+1}/{len(ips)}] x {ip}: Error - {e}", flush=True)


# ============================================================================
# Bug reproducer: launch worker and send SIGINT burst during shutdown
# ============================================================================

def send_sigint(pid):
    try:
        os.kill(pid, signal.SIGINT)
        return True
    except ProcessLookupError:
        return False


def is_alive(pid):
    try:
        os.kill(pid, 0)
        return True
    except ProcessLookupError:
        return False


def read_output_lines(proc, count=10, timeout=120):
    """Read worker output lines until we have enough results"""
    lines = []
    deadline = time.time() + timeout
    while len(lines) < count and time.time() < deadline:
        line = proc.stdout.readline()
        if not line:
            break
        decoded = line.decode(errors='replace').strip()
        if decoded:
            lines.append(decoded)
            print(f"  {decoded}")
    return lines


def run_attempt(attempt, script_path):
    """Run worker subprocess and send continuous burst of SIGINT"""
    print(f"\n{'='*60}")
    print(f"Attempt {attempt}")
    print(f"{'='*60}")

    cmd = [sys.executable, '-u', script_path, '--worker']

    proc = subprocess.Popen(
        cmd,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
    )

    pid = proc.pid
    print(f"  PID: {pid}")
    print(f"  Waiting for scan output...\n")

    output_lines = read_output_lines(proc, count=20, timeout=180)

    if not output_lines:
        print("  No output, aborting")
        proc.kill()
        proc.communicate()
        return None

    if not is_alive(pid):
        _, stderr = proc.communicate()
        return stderr.decode(errors='replace')

    print(f"\n  Got {len(output_lines)} results. Sending SIGINT burst...\n")

    # Send continuous burst of SIGINT signals for ~5 seconds.
    # This covers the entire shutdown sequence:
    #   as_completed -> __exit__ -> shutdown -> atexit -> _PyImport_Cleanup
    # One of these signals MUST land during _PyImport_Cleanup when
    # sys.stderr is being cleared -> triggers _PyObject_Dump
    signal_count = 0
    start = time.time()
    duration = 5.0  # seconds of continuous SIGINT bombardment

    while time.time() - start < duration:
        if not is_alive(pid):
            break
        send_sigint(pid)
        signal_count += 1
        time.sleep(0.005)  # 200 signals/second

    elapsed = time.time() - start
    print(f"  Sent {signal_count} SIGINT signals over {elapsed:.1f}s")

    try:
        stdout, stderr = proc.communicate(timeout=15)
    except subprocess.TimeoutExpired:
        proc.kill()
        stdout, stderr = proc.communicate()

    return stderr.decode(errors='replace')


def reproducer_main():
    """Entry point for bug reproducer mode"""
    script_path = os.path.abspath(__file__)

    print(f"Python: {sys.version}")
    print(f"Platform: {sys.platform}")
    print(f"Worker: {script_path} --worker")
    print(f"Method: curl (HTTPS connect to IP ranges)")
    print(f"Workers: 100 threads")
    print(f"Target: _PyObject_Dump + 'lost sys.stderr'")
    print(f"")
    print(f"This reproducer launches a multi-threaded curl-based scanner,")
    print(f"then sends rapid SIGINT signals during Python's shutdown sequence")
    print(f"to trigger the _PyObject_Dump bug when sys.stderr becomes None.")

    max_attempts = 10

    for attempt in range(1, max_attempts + 1):
        stderr_text = run_attempt(attempt, script_path)

        if stderr_text is None:
            continue

        has_object_dump = 'object address' in stderr_text
        has_lost_stderr = 'lost sys.stderr' in stderr_text
        has_keyboard_interrupt = 'KeyboardInterrupt' in stderr_text

        print(f"\n  --- STDERR ---")
        print(stderr_text)
        print(f"  --- END ---")

        if has_object_dump:
            print(f"\n{'='*60}")
            print(f"BUG REPRODUCED on attempt {attempt}!")
            print(f"{'='*60}")
            if has_lost_stderr:
                print("Full bug with 'lost sys.stderr'!")
            return 0

        if has_keyboard_interrupt:
            print(f"  Got KeyboardInterrupt but not _PyObject_Dump. Retrying...")
        else:
            print(f"  No relevant output. Retrying...")

    print(f"\nFailed after {max_attempts} attempts.")
    return 1


# ============================================================================
# Main entry point
# ============================================================================

def main():
    if "--worker" in sys.argv:
        worker_main()
    else:
        reproducer_main()


if __name__ == "__main__":
    sys.exit(main())

How the reproducer works

The script operates in two modes:

1. Reproducer mode (default): Launches the worker as a subprocess, waits for it to start processing, then sends ~960 SIGINT signals over 5 seconds (~200 signals/second) to cover the entire shutdown sequence window.

2. Worker mode (--worker): Uses concurrent.futures.ThreadPoolExecutor with 100 workers to run curl HTTPS requests against IP ranges. This creates many active threads, which is essential for the shutdown timing window.

The key parameters:

• 100 concurrent threads in ThreadPoolExecutor
• ~7,800 curl tasks submitted
• ~960 SIGINT signals over 5 seconds during shutdown
• Signal interval: 5ms (~200 signals/second)

Requirements

• Linux
• Python 3.12 (confirmed on 3.12.7)
• curl (pre-installed on virtually all Linux distributions)

No third-party Python packages or external tools are required.

Observed output

Bug reproduces on the first attempt:

$ python3 reproduce_bug_curl.py

Python: 3.12.7 (main, Jun 18 2025, 13:16:51) [GCC 14.2.0]
Platform: linux
Worker: /home/user/reproduce_bug_curl.py --worker
Method: curl (HTTPS connect to IP ranges)
Workers: 100 threads
Target: _PyObject_Dump + 'lost sys.stderr'
...

  Got 20 results. Sending SIGINT burst...

  Sent 956 SIGINT signals over 5.0s

  --- STDERR ---
Traceback (most recent call last):
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 243, in as_completed
  File "/usr/lib/python3.12/threading.py", line 655, in wait
  File "/usr/lib/python3.12/threading.py", line 355, in wait
    waiter.acquire()
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/reproduce_bug_curl.py", line 139, in worker_main
    for i, future in enumerate(concurrent.futures.as_completed(future_to_ip)):
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 258, in as_completed
    with f._condition:
         ^^^^^^^^^^^^
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/reproduce_bug_curl.py", line 304, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/reproduce_bug_curl.py", line 298, in main
  File "/home/user/reproduce_bug_curl.py", line 133, in worker_main
    with concurrent.futures.ThreadPoolExecutor(max_workers=100) as executor:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 647, in __exit__
    self.shutdown(wait=True)
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 238, in shutdown
object address  : 0x72fed827c940
object refcount : 6
object type     : 0xa3e620
object type name: KeyboardInterrupt
object repr     : KeyboardInterrupt()
lost sys.stderr
Exception ignored in: <module 'threading' from '/usr/lib/python3.12/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1594, in _shutdown
    atexit_call()
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 31, in _python_exit
    t.join()
  File "/usr/lib/python3.12/threading.py", line 1149, in join
  File "/usr/lib/python3.12/threading.py", line 1169, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt:
  --- END ---

============================================================
BUG REPRODUCED on attempt 1!
============================================================
Full bug with 'lost sys.stderr'!

Key observations

1. _PyObject_Dump was invoked — This is a C-level internal function (Objects/object.c) that writes raw object information directly to fd 2, bypassing Python's I/O system. Users should never see this output.

2. Memory addresses leaked — object address: 0x72fed827c940 (heap address) and object type: 0xa3e620 (type object address) are exposed.

3. "lost sys.stderr" — Originates from Python/pythonrun.c in the PyErr_Display / _PyErr_WriteUnraisableMsg path. Emitted when sys.stderr is NULL or destroyed during module cleanup.

4. Traceback truncated mid-line — The normal traceback is cut after thread.py", line 238, in shutdown and immediately followed by _PyObject_Dump output, indicating the normal exception handling path was interrupted.

Expected behavior

• The interpreter should handle SIGINT gracefully during shutdown, or block signals during critical cleanup phases
• _PyObject_Dump (a C-level debug fallback) should never be visible to end users
• Internal memory addresses should not be leaked to process output

Root cause analysis

The race condition is in Py_FinalizeEx (Python/pylifecycle.c):

1. wait_for_thread_shutdown() -> threading._shutdown() -> t.join()
2. call_py_exitfuncs()        -> atexit handlers
3. _PyImport_Cleanup()        -> sys.stderr = None    <-- VULNERABLE WINDOW
4. SIGINT arrives here        -> KeyboardInterrupt raised
                              -> PyErr_Display() finds sys.stderr = NULL
                              -> falls back to _PyObject_Dump()
                              -> prints "lost sys.stderr"

Signals are not blocked during _PyImport_Cleanup, so SIGINT can trigger exception handling after sys.stderr has been cleared.

CPython versions tested on:

3.12

Operating systems tested on:

Linux

[ZeroIntensity]

This is a C-level internal debug function (Objects/object.c)

Not anymore!

I think it's somewhat expected to see some nifty output if you CTRL+C during finalization. You're basically telling Python, "hey, stop finalizing and just shut down, I don't care what's happening." Blocking signals can be especially annoying/dangerous because if something were to get stuck during cleanup, you can't kill it with CTRL+C.

For what it's worth, I couldn't reproduce this on 3.14+. 3.12 is security-only, so we can't really fix it there, and I think blocking signals during finalization is too invasive a change for 3.13.

_PyObject_Dump (a C-level debug fallback) should never be visible to end users

How come? It's designed as a fallback exactly for cases like this.

More generally, I'm curious as to what prompted this. Is there an actual issue this caused, or were you simply annoyed with the output? If it's the latter, then I'm sorry, but I don't think there's anything worth fixing here -- if you throw a SIGINT at a Python process at some arbitrary time, then you just have to deal with the reality that you might get some garbage output.

[raminfp]

Hey @ZeroIntensity

You mentioned that 3.12 is security-only and the bug couldn't be reproduced on 3.14+. However, the bug reproduces on Python 3.13.0, which is the current stable release receiving bugfix updates:

$ python3.13 reproduce_bug_curl.py

Python: 3.13.0 (main, Jun 17 2025, 17:03:05) [GCC 14.2.0]
Platform: linux
...

BUG REPRODUCED on attempt 2!

object address  : 0x7fc904720c40
object refcount : 6
object type     : 0x9a7a40
object type name: KeyboardInterrupt
object repr     : KeyboardInterrupt()
lost sys.stderr

Python 3.13 is not security-only — it is the current active release branch that accepts bugfix commits. This means the fix is eligible for 3.13.x.

Addressing "Is there an actual issue this caused?"

Yes. The reproducer is derived from a real-world tool — a network scanner using concurrent.futures.ThreadPoolExecutor with many workers. In production:

• Users press Ctrl+C once to stop the scan
• If the timing is right (which happens in practice with 100+ threads), a single Ctrl+C can land during _PyImport_Cleanup
• The burst approach in the reproducer simply increases the probability to make it reliably reproducible for testing

The key point: this is not only an artificial scenario. Any multi-threaded Python program using ThreadPoolExecutor that gets Ctrl+C during shutdown has a window where this can occur. The more threads, the wider the window.

Addressing "blocking signals can be dangerous"

I agree that permanently blocking signals is problematic. However, the vulnerable window in _PyImport_Cleanup is very short. A more targeted approach could work:

• Option A: Temporarily block SIGINT only during the sys.stderr = None assignment in _PyImport_Cleanup, then restore. This is microseconds, not seconds.
• Option B: Instead of setting sys.stderr to None, set it to a minimal fallback writer that doesn't crash PyErr_Display. This avoids the signal-blocking question entirely.
• Option C: Make PyErr_Display handle sys.stderr = None more gracefully — suppress output instead of falling through to _PyObject_Dump.

None of these require blanket signal blocking.

Addressing "_PyObject_Dump is designed as a fallback"

I understand _PyObject_Dump is intentionally a fallback, and I see it has been promoted to public API (PyObject_Dump) in 3.15 for debugging purposes.

However, the concern is not about the function's existence — it's about when it gets triggered. A user pressing Ctrl+C on a multi-threaded program is normal operation, not a debugging scenario. The fallback to _PyObject_Dump here means the normal error-reporting path is broken during a window that real users can hit.

[raminfp]

Update: Bug also reproduces on Python 3.14.0

@ZeroIntensity You mentioned:

For what it's worth, I couldn't reproduce this on 3.14+.

I built Python 3.14.0 from source and tested it. The bug reproduces on 3.14.0 as well (attempt 2):

$ /tmp/python314/bin/python3.14 reproduce_bug_curl.py

Python: 3.14.0 (main, Feb  9 2026, 10:08:32) [GCC 14.2.0]
Platform: linux

BUG REPRODUCED on attempt 2!

object address  : 0x7f18b661b880
object refcount : 3
object type     : 0x6390228baae0
object type name: KeyboardInterrupt
object repr     : KeyboardInterrupt()
lost sys.stderr

Summary of all tested versions

Version	Result	Attempt
Python 3.12.7	Bug reproduced	attempt 1
Python 3.13.0	Bug reproduced	attempt 2
Python 3.14.0	Bug reproduced	attempt 2

The race condition in _PyImport_Cleanup exists across all current Python versions (3.12, 3.13, 3.14). This is not a version-specific issue — it's a fundamental signal safety gap in Py_FinalizeEx.

[ZeroIntensity]

Python 3.13 is not security-only — it is the current active release branch that accepts bugfix commits. This means the fix is eligible for 3.13.x.

I don't appreciate the use of ChatGPT because it makes false assertions like this -- just because a fix could be backported doesn't mean it will. Again, changing handling of sys.stderr or signals is invasive and not something to be taken lightly; there is a very real chance that changing this will break code that already relies on the _PyObject_Dump output. If this gets fixed, it will likely only be for the 3.15 release in October.

The key point: this is not only an artificial scenario. Any multi-threaded Python program using ThreadPoolExecutor that gets Ctrl+C during shutdown has a window where this can occur. The more threads, the wider the window.

Okay, but why does the network scanner care? What is it doing that this behavior breaks?

Option A: Temporarily block SIGINT only during the sys.stderr = None assignment in _PyImport_Cleanup, then restore. This is microseconds, not seconds.

I meant that blocking signals, even during this small window, is dangerous. SIGINT is a debugging tool on its own, so if something gets stuck here, we're removing the ability to debug it with CTRL+C.

Option B: Instead of setting sys.stderr to None, set it to a minimal fallback writer that doesn't crash PyErr_Display. This avoids the signal-blocking question entirely.

Again, ChatGPT is wrong. call_py_exitfuncs and _PyImport_Cleanup no longer exist and have not existed since 3.9. sys.stderr is now simply cleared when clearing sys.__dict__. So, if we put a new object in there, it still has to be deallocated. There will always have to be a point where sys.stderr is None during finalization, or otherwise we're leaking memory. (We could make an immortal static object that prints to the C stderr stream, but is that really better than just using _PyObject_Dump?)

Option C: Make PyErr_Display handle sys.stderr = None more gracefully — suppress output instead of falling through to _PyObject_Dump.

See the Zen of Python: "Errors should never pass silently."

The race condition in _PyImport_Cleanup exists across all current Python versions (3.12, 3.13, 3.14). This is not a version-specific issue — it's a fundamental signal safety gap in Py_FinalizeEx.

Python is generally not signal-safe, especially during finalization.

[raminfp]

@ZeroIntensity Fair point about the function names, I used AI to help with the report writing and didn't verify those against current source. My bad. The testing across 3.12, 3.13 and 3.14 is mine though.

About your question: why does the scanner care:

I found this while working on a real tool, not through any artificial test. I have a scanner that uses ThreadPoolExecutor with 100 workers to scan Cloudflare IP ranges. I hit Ctrl+C to stop it, and got this in my terminal:

^C^CTraceback (most recent call last):
  File "/home/raminfp/PycharmProjects/ping_scanner/scanner_v2.py", line 346, in scan_ip_range
    for i, future in enumerate(concurrent.futures.as_completed(future_to_ip)):
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 243, in as_completed
    waiter.event.wait(wait_timeout)
  File "/usr/lib/python3.12/threading.py", line 655, in wait
    signaled = self._cond.wait(timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/threading.py", line 355, in wait
    waiter.acquire()
KeyboardInterrupt
^Cobject address  : 0x7698080f9de0
object refcount : 6
object type     : 0xa3e620
object type name: KeyboardInterrupt
object repr     : KeyboardInterrupt()
lost sys.stderr
^C^CException ignored in: <module 'threading' from '/usr/lib/python3.12/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1594, in _shutdown
    atexit_call()
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 31, in _python_exit
    t.join()
  File "/usr/lib/python3.12/threading.py", line 1149, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.12/threading.py", line 1169, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt:
^C

You can see the ^C in the log. just normal Ctrl+C, nothing artificial. When I saw object address : 0x7698080f9de0 and lost sys.stderr I honestly thought something had gone wrong with my program. Took me a while to figure out this was coming from CPython itself. The burst reproducer I wrote later just makes it easier to trigger reliably for testing.

About the immortal static object you mentioned . I think that would actually be a good improvement. Right now when _PyObject_Dump kicks in, users see refcounts and type pointers that mean nothing to them. If there was a static writer that just forwarded to C stderr, PyErr_Display could still print a normal KeyboardInterrupt traceback.

[ZeroIntensity]

Okay, we do have an internal printer API for directly writing to C file streams during interpreter initialization (meaning io hasn't been loaded yet). Currently, there's no statically allocated variation, but I suppose it would be relatively easy to set that up.

That said, I'm almost certain you'll find other problems with sending a SIGINT during finalization, especially in multithreaded programs. It'd be good if you could refactor your code to do less things during finalization (such as joining your threads explicitly).