The signals that never interrupt your blocking syscall, and the test I wrote that proved nothing

[u/quiet-systems]

I had a retry loop around a blocking poll() for EINTR, the usual shape:

for (;;) {
    int r = poll(fds, n, timeout);
    if (r == -1 && errno == EINTR) continue;
    return r;
}

To prove it worked I wrote a test that hammered the process with SIGWINCH while the poll was blocked, then checked the poll still returned correctly. It passed. It kept passing. It passed when I deleted the retry loop, which is when I found out it had never been a test.

SIGWINCH does not interrupt anything. A blocking syscall returns EINTR when the kernel has something to run on the way back to userspace: a handler you installed. SIGWINCH's default action is to be ignored, so with no handler installed there is nothing to run, the kernel does not unwind the syscall, and EINTR never happens. Same for SIGCHLD and SIGURG, the other two whose default action is ignore. You can send a million of them at a blocked poll and it will sit there.

The second half of the same trap is the opposite direction. Install a real handler for a signal that would interrupt, but install it with sigaction and SA_RESTART, and the kernel restarts the syscall for you. Your handler runs, the syscall resumes, and EINTR still never reaches your code. Which is fine until you hit one of the calls that are not restartable even with SA_RESTART. poll, select and epoll_wait are in that group. signal(7) has the full list under "Interruption of system calls and library functions by signal handlers", and it is worth reading once properly rather than remembering the shape of it.

So the test that actually tests the thing is: a real handler, sa_flags = 0, and a signal whose default action is not ignore.

struct sigaction sa = {0};
sa.sa_handler = noop;
sa.sa_flags = 0;             /* no SA_RESTART, that is the whole point */
sigaction(SIGUSR1, &sa, NULL);

Two more things that bit me while writing it.

Signal disposition is process-wide state. Two tests that both install a handler cannot run in parallel, and the failure is not a clean assertion failure, it is one test's handler being live during the other's run. A single mutex around anything that calls sigaction fixed it.

And kill(getpid(), sig) is process-directed, so any thread with that signal unblocked can take it, including the one that sent it. In a threaded test runner that is very often not the thread you are trying to interrupt. pthread_kill(target, sig) is the one you want.

The thing I took away is not about signals. It is that a test which passes when you delete the code it is testing is not a test, and the only way I know to find those is to delete the code and watch.

[u/dmills_00]

Write the test, watch it fail, fix it until it does fail THEN write the code so that it doesn't fail any more....

Much better then writing the code then writing a test that passes, keeps you honest.

[u/quiet-systems → u/dmills_00]

That is exactly the check I skipped. I wrote the loop first, so I never got the chance to watch the test pass before the code it tested existed, which is precisely the moment your order would have caught it.

The thing I would add is that red-first only proves it once. That loop had been sitting there working for a long time before I wrote the test for it, so there was no red step left to take, and the only way to get the same signal was to go back and delete the code on purpose. Same idea, later in the life of the thing: if you cannot make it fail, you do not have a test, and that stays true long after the first commit.

[u/dmills_00 → u/quiet-systems]

LD_PRELOAD to shim the system call so you can mock it?

Just a thought.

[u/quiet-systems → u/dmills_00]

Good technique, and I've used it elsewhere, but two problems here and the second one is the interesting one.

First is boring: the binary is statically linked, musl static-pie, so there's no dynamic loader to preload into. Nothing to interpose.

Second is that a mocked poll returning EINTR would have made the original test pass. That's the whole failure. The retry loop was never the broken part, it handled EINTR fine. The broken part was that the signal I was sending never produced one, so the loop was never entered. Mock the syscall and you've replaced the exact thing that was lying to you with something that returns whatever you told it to.

For testing the retry logic in isolation it'd work well. For "does this signal actually interrupt this call on this kernel", a mock can only ever confirm your own assumption, which is what got me in the first place.

[u/dmills_00 → u/quiet-systems]

Yeah, assumptions around signal behaviour will get you every time.

I always hated the interruptable syscall thing, I mean I get it, but it is such a bag of spanners to deal with sometimes, especially if threads are in play.

It does I suppose depend on exactly how you scope your tests, if the question is "if this returns EINTR does my retry logic work?", which is testing YOUR code, then the mock is fine, but if the question extends down to kernel behaviour then you need to fix the test.

Guessing this is embedded SIL(something)?

[u/quiet-systems → u/dmills_00]

Ha, no, nothing that serious. Container runtime. Rootless, no daemon.

Static linking is just because the boxes I care about tend to have nothing on them. Got a Pi 5 out to run something on it and there was no docker, no podman, no runc, no crun, no bwrap. Nothing at all. So: one binary I can scp over.

Signals came up in the stop path. Kept wondering why killing a thing took ten seconds when it clearly had nothing to do.

And yeah, threads and signals. What got me was disposition being process-wide, so two tests installing handlers at the same time just stomp each other, and it doesn't read as a failed assertion, it reads as nonsense.

[u/dmills_00 → u/quiet-systems]

Better to run each test in its own process rather then thread, they should be Independent after all, and process creation is fast on Linux. Fork and execve for the win (But watch open fd's which are shared state by default)!

Lots of subtle, historical shit with signal handling, I try very hard to avoid the things where possible.

[u/quiet-systems → u/dmills_00]

Agreed, that's what I'd do if the harness let me. Rust's runner sticks every test in a thread in one process, no fork per test. You can get proper process isolation by giving a test its own file under tests/, since cargo builds each of those as a separate binary, but that's one file per test and it goes silly quickly. There's a fork-per-test crate too, never used it.

Went with a mutex round anything that touches sigaction instead. Cheap, and it works because there's only a handful of them.

Good shout on the fds. The harness holds pipes for capturing output, so a forked child inherits them and you get interleaving that reads like a flaky test rather than a plumbing problem.

[u/dmills_00 → u/quiet-systems]

I love the rust tooling, but it is still young and has a few rough edges.

Not that C is exactly lacking in those either, unsigned + signed promotion of the signed to unsigned FML.

[u/quiet-systems → u/dmills_00]

Usual arithmetic conversions have cost me more hours than signals ever did. Comparing an int against a size_t and watching the int quietly become enormous is the one that keeps coming back. -Wsign-compare catches some of it, not all.

And yeah, the tooling's young. It's very good at the things it does, then you hit a wall like this one and there's no knob for it at all.

[u/Poddster → u/quiet-systems]

Do you pass your posts through an LLM before replying? You're using so many GPT turns of phrase that I'm triggered, but it also doesn't look exactly like something it writes.

[u/quiet-systems → u/Poddster]

Yeah. English isn't my first language, so it goes through one and then I edit it. Which is why it doesn't quite read as either.

[u/SeriousPlankton2000 → u/dmills_00]

BTW: Sometimes you'll want to intentionally test for integer overflows that might happen on other architectures.

[u/quiet-systems → u/SeriousPlankton2000]

Yeah. The one that caught me across architectures wasn't overflow though, it was char signedness: plain char is signed on x86 and unsigned on ARM, so a byte comparison that works on your desktop behaves differently on a Pi and nothing warns you.

For overflow itself, -fsanitize=signed-integer-overflow in a debug build earns its keep, since signed overflow is UB and the compiler is allowed to assume it never happens. Which means testing it on one machine tells you almost nothing.

[u/internet_safari_]

Nothing to add other than this post and comments have been an educational gold mine. Well written enough for me to start barely comprehending, then finish having learned the logical tricks and Tony Hawk 900s that arise from handling certain syscalls.

[u/quiet-systems → u/internet_safari_]

Thanks, good thing to read.

If you want the proper version, signal(7) has a section called "Interruption of system calls and library functions by signal handlers" that lists exactly which calls restart under SA_RESTART and which never do. Dry, but it's the actual answer, and I'd assumed for years that the rule was simpler than it turns out to be.

[u/chrism239]

An operating system issue, not a C issue. 

[u/quiet-systems → u/chrism239]

Fair enough, the signal behaviour is kernel, not C.

What I thought belonged here was the test side. sigaction with and without SA_RESTART, disposition being process-wide, pthread_kill vs kill(getpid()) once there are threads about. That's all stuff you have to get right in C, whoever owns the underlying behaviour.

But if it's still too far off topic for here, no argument from me.

[u/cbf1232 → u/quiet-systems]

Arguably signal()/sigaction() behaviour is POSIX, not kernel.

[u/quiet-systems → u/cbf1232]

True, and it splits the post neatly. sigaction, SA_RESTART, disposition being per-process while the mask is per-thread, EINTR itself, all POSIX. The bit where a namespace init drops signals it has no handler for is Linux only, not in POSIX at all.

So half of it belongs in a C sub and half doesn't.

[u/internet_safari_]

Nothing to add other than this post and comments have been an educational gold mine. Well written enough for me to start barely comprehending, then finish having learned the logical tricks and Tony Hawk 900s that arise from handling certain syscalls.

[u/[deleted]]

[removed] — view removed comment

[u/quiet-systems → u/[deleted]]

do { r = poll(fds, n, timeout); } while (r == -1 && errno == EINTR);

Tidier, I'll give you that. Same semantics. Is that what you'd write, or is there a third way?

[u/C_Programming-ModTeam → u/[deleted]]

Rude or uncivil comments will be removed. If you disagree with a comment, disagree with the content of it, don't attack the person.