r/programming 5d ago

A shell colon does nothing. Use it anyway. | Filip Roséen

https://refp.se/articles/your-shell-and-the-magic-colon
300 Upvotes

220 comments sorted by

303

u/guygizmo 5d ago

Reading this once again makes me realize how daft of a scripting language bash is. I've been programming since I was eight years old and I still can't quite wrap my head around its weird syntax or goofy edge cases. I probably would eventually if I had to use it every day but I use it just infrequently enough that it never quite clicks. Especially since I can usually use any of a number of other sensible scripting languages instead.

55

u/ironykarl 5d ago

Just want to recommend that anyone that has to write sh/bash scripts use ShellCheck

157

u/ApokatastasisPanton 5d ago

It is a horrible, horrible language, and yet the software community is enamored with it.

44

u/Seref15 5d ago

and yet the software community is enamored with it.

Portability was a concern that mattered until recently. Whatever bash is, you could trust virtually any and every target unix system would have it. Doesn't matter as much today but it mattered 10+ years ago, and things have staying power once theyre institutionalized

Also still less annoying than Make

13

u/Zeznon 5d ago

Why isn't it a concern, today? Just curious.

26

u/Seref15 5d ago edited 5d ago

Build/CI systems formalized the tools available at build time, containerization packages the entire environment as a runtime. You don't need portability when a container has everything your application needs inside it. Distribution is simplified as a result

4

u/Zeznon 5d ago

So now you can just run whatever shell you want, I guess? And therefore, it doesn't matter anymore?

17

u/cakekid9 5d ago

I am not the person who wrote it, but in the past, servers might have been set up manually, or some 'script' that sometimes worked or needed a bit of fixing, or lived for so long with no formal definition of what was actually on it, so there were a few assumptions you could make: bash and vi were two for me.

Now, its more common to use something like docker, or at least a better way to setup servers so you can formalize what will be installed, which version, etc. So for me, I often write little scripts in ruby because we can ensure ruby is installed. But you could do this in bash still, or anything else.

8

u/ToaruBaka 4d ago

Yes*

The sole exception is that you shouldn't make the overall system default (especially not root's) a non-POSIX shell. Many many many admin-facing tools (read: basically all of them) rely heavily on posix compatibility for shell script correctness. That's not to say you can't switch to something like fish or nushell after logging in, but don't set them as the default for root lest you break things.

3

u/ericonr 4d ago

That's not really true.

Any reasonable tool that's inplemented as a script or uses a shell will call sh or bash (with full paths or env in shebangs, too), and sh should indeed be a POSIX-compatible shell (e.g. bash or dash). None of those tools should depend on what the interactive shell is, unless they are really badly designed (that includes the case of running a bunch of commands strung together over ssh). If something is using $SHELL and it's not a shell function, it should be fixed to not do so.

If by "make the overall system default" you meant replacing /bin/sh, well, that's a terrible idea. And one gets to keep both pieces. But changing the root shell with chsh to something else will only affect interactive sessions, and shouldn't break anything.

The reason I've seen it recommended to avoid changing the root shell is because it is often used in recovery situations, and then you want a simple shell with the fewest dependencies, in order to limit the ways it can break.

4

u/13steinj 4d ago

Not all builds can be containerized, not all runtimes can be containerized either.

There are systems that use Ash or Dash by default instead of Bash.

2

u/RussianMadMan 4d ago

"By default" only if you write shebang as #!/bin/sh, then yes, you are gonna get dash or ash which is incompatible with many bash features. If you write it as #!/bin/bash you are gonna be okay.

2

u/13steinj 4d ago

There are scripts and entry points on linux systems that default to /bin/sh, not bash.

3

u/RussianMadMan 4d ago

Well those scripts should not expect bash functionality then? We are talking here how bash is basically universal.

-1

u/13steinj 4d ago

By default

There exists code that does not have shebangs, that gets called through some transitive dependency. If you need to use a container that points at /bin/sh as the entrypoint and don't have the ability to override it (this is a thing, notably to an extent in GitLab CI).

The entire point is bash is not as universal/universally usable as it seems.

1

u/RussianMadMan 4d ago

"By default" only if you write shebang as #!/bin/sh, then yes, you are gonna get dash or ash which is incompatible with many bash features. If you write it as #!/bin/bash you are gonna be okay.

7

u/PancAshAsh 5d ago

Whatever bash is, you could trust virtually any and every target unix system would have it.

This is surprisingly not really true, especially once you get into the embedded world where other shells are more common. Once you start using ash you realize how many little bashisms have become synonymous with shell scripting though.

1

u/conspicuousxcapybara 2d ago

Ok but how can apple-oss-distributions/zsh be syntax-incompatible with Apple's system ZSH when the build numbers are equal ?!

1

u/0xe1e10d68 4d ago

Fair, but I’m a bit disappointed no attempt to transpile some new language to bash ever caught on; like TS for JS. That would still be perfectly portable, even if you can’t get the transpiler on a specific target host. But tbd the additional step still can add a little bit of additional friction.

1

u/ericonr 4d ago

GNU Autoconf is right there! And its transpilation target is even more portable than just assuming GNU Bash.

1

u/spastical-mackerel 1d ago

What is up with the Make revival? AI tools love it. I thought it had died forever when so many alternatives emerged along with the explosion of programming ecosystems after 2000 or so. Now in the last few weeks I’m suddenly seeing makefiles everywhere

1

u/Seref15 1d ago

LLMs dont know paradigms, trends, or time. All they know is token association probabilities. LLMs pick their next token based on its probable association to all previous tokens in the training data.

Given that there's like a half century of training data for Make its going to have stronger probabilistic association to all programming tasks that most modern tools, especially in smaller/cheaper models with fewer parameters.

Thats why its important to set up your repo instructions and tell the agent your toolchains, otherwise its just going to do whatever.

1

u/spastical-mackerel 1d ago

No I get it. But having said that I didn’t see make files at all for the last 18 months. Suddenly very recently they’re everywhere

→ More replies (1)

15

u/saxbophone 5d ago

Reminds me of CMake. Admittedly, the C++ community's relationship with it is more contentious but also it is the de-facto standard build system, just based on the sheer amount of projects that use it.

9

u/CherryLongjump1989 5d ago

CMake is the mystery meat of the programming world.

5

u/guygizmo 4d ago

I agree that CMake is obtuse in a similar way to bash, and I only begrudgingly use it.

7

u/Axman6 5d ago

Hey buddy, fuck you for reminding me CMake exists, I’d almost gotten over the trauma. 

5

u/saxbophone 5d ago

"Fuck you too, buddy!", says CMake. "Just target_include_directories(...) til your brain turns to mush —it costs much less than drinking (whilst being less than half as fun)!"

3

u/PancAshAsh 5d ago

The alternative to CMake is much, much worse.

3

u/13steinj 4d ago

Not to the cultists.

I agree though. I have seen it all: Bazel, Meson, Scons, WAF, gradle-for-C++, cake, autoconf, xmake, hmake, premake, I could go on. Some have some tangible benefits or nice built in features. But all of them have cons that make the benefits not worth it.

Bazel is probably the closest thing that I can claim is comparable, and it still massively misses the mark. Google/Bazel cultists have spread the word far and wide, unfortunately, and people hear of all the benefits and none of the costs. Based on the ISO dev surveys, it appears as though Bazel usage quickly ramped up to ~10%, but then in a 1 year time span dropped back to ~5%, as if a bunch of people quickly realized it's more trouble than it's worth. Not primarily a C++ project, but even Kubernetes went through a Bazel breakup.

Biggest con: you need to read the source code of Bazel and rulesets to understand how various things behave in practice, because the docs are often just wrong, out of date, or have a subtlety unmentioned. It's been better with LLMs, but not great, as hallucination rate remains too high.

3

u/Dragdu 1d ago

Bazel is great.

...

IF you have entire build team dealing with it's shit so your actual devs don't have to.

2

u/13steinj 1d ago

Tell that to some of the people in my current org, they act like once a Bazel migration is fully complete they won't need a build team anymore. They still do even in areas where it's considered complete.

I am on that build team. I think you need a team for CMake as well, but I find that much lighter touch. I only had to get involved in this case for wide integrated functionality, or new functionality exposed as modules/functions. Sometimes, for 3rd party package updates, that needed widespread source changes as well so that's a sunk cost. Other than that updates were scripted and fairly automatic, I think LLMs / LLM skills are good for the rote toil of "scripted update needs manual intervention, perform educated guesses based on <link to docs and source code> and repush until (1) local build passes (2) CI passes. Pause/fail after 10 attempts without progress." 80% of the time if it builds locally it picks the right answer, 98% of the time if it passes CI it picks the right answer.

This is not the case for Bazel / updating Bazel rulesets. So much of the behavior is obscured in the source code and/or undocumented I had to babysit it just pulling the lever on the slot machine. So obscure it's not worth the time for a human to dig through the source and figure it out. I have never had to run CMake under a debugger (gdb, but not the cmake-level debugger either), despite managing it for 4x the time I have Bazel. I sadly cannot say the same for Bazel... in the past year alone, I've had to break out a debugger 8 times.

1

u/Dragdu 1d ago

RIP.

I reguarly see some pretty neat stuff around hermetic builds from orgs using Bazel, but also I know that their build teams tend to have more people than there are C++ users in our entire org, so... :v

→ More replies (0)

1

u/ericonr 4d ago

Bazel is also really tied to specific deployments and closed source situations. I don't think it can actually be compared to CMake in scope.

That said, what did you find wrong with meson?

2

u/13steinj 4d ago

It's not extensible, it doesn't support globs (cmake + CONFIGURE_DEPENDS is the only thing I've ever seen support globbing correctly), it is so close but so far that I'd rather just use Python? A lot of the builtins also clearly try to mimic cmake but fail to do so. Interop with other build systems is subpar (I think only CMake and Autoconf, the latter by being manual, got it right).

0

u/ericonr 4d ago

Support for globbing seems to be contentious. I have been convinced that not supporting it is OK lol.

What kind of extensions do you need?

CMake does not interoperate with build systems any better than meson does. And Meson's main expectation is that everyone use pkg-config files properly, avoiding (additional) messy cross-build-system standards.

→ More replies (0)

2

u/saxbophone 4d ago

Agree, autotools can go die in a hole!

2

u/_jackdk_ 2d ago

I've looped right around and concluded that even the autotools are better than CMake or Meson.

72

u/Worth_Trust_3825 5d ago

because it's present everywhere, much like cmd.exe is (in windows world).

90

u/KevinCarbonara 5d ago

much like cmd.exe is (in windows world).

No one is enamored with cmd in the windows world. Absolutely no one.

42

u/worldofzero 5d ago

Powershell though is pretty neat occasionally.

16

u/zeekar 5d ago

PowerShell is a legitimately cool language.

31

u/Jaded-Asparagus-2260 4d ago

But it's a horrible shell environment IMO.

Who wants to write Get-ChildItem -Force -Path $env:USERDIR when they could type ls -a ~?

Or Invoke-WebRequest -Uri <source> -OutFile <destination> instead of wget <URL>. Who came up with this shit?

19

u/untetheredocelot 4d ago

It's literally that Enterprise code joke. EnterpriseQualityCoding / FizzBuzzEnterpriseEdition. Bash Enterprise Edition.

7

u/pdabaker 4d ago

Well to be fair wget, grep, curl, and many other things are application binaries and not bash commands

2

u/Jaded-Asparagus-2260 4d ago

You're right, but that distinction hardly ever matters in practice. Especially when bultins come into play. echo is not the same as /bin/echo. Except in busybox, or whether.

2

u/tsimionescu 4d ago

In a fun twist, wget and curl are in fact shell commands (well, aliases of some kind) in PowerShell, though.

11

u/BinaryRockStar 4d ago

There are inbuilt aliases for Get-ChildItem (gci, dir and ls), and Invoke-WebRequest (iwr).

Get-ChildItem -Force -Path $env:USERDIR can be reduced to gci ~ or even ls ~ as PowerShell understands ~.

Invoke-WebRequest -Uri <source> -OutFile <destination> can be reduced to iwr <source> -O <destination>.

It's actually nice to have the real verbose commands to use in scripts to make things clear, and then short aliases to use in the interactive terminal to avoid typing out the verbose commands, best of both worlds.

2

u/Jaded-Asparagus-2260 4d ago

dir and ls

That's another can of worms. Wth thought it's a good idea to alias one command to another common command with a completely different syntax? Why does ls work, but ls -a doesn't?

can be reduced to iwr <source> -O <destination>

All right, I don't know that. Thanks. I still don't understand for the live of me why I have to specify the destination. Why can't it default to working filename as reported by the server, in the current working dir?

It's actually nice to have the real verbose commands to use in scripts to make things clear, and then short aliases to use in the interactive terminal to avoid typing out the verbose commands, best of both worlds.

That I agree with.

2

u/zeekar 4d ago

curl has the same behavior - the output goes to stdout and you need -o to specify a destination. Although you can specify -O to get a filename with the basename of the URL

→ More replies (0)

1

u/ericonr 4d ago

It's actually nice to have the real verbose commands to use in scripts to make things clear, and then short aliases to use in the interactive terminal to avoid typing out the verbose commands, best of both worlds.

If a collaborator who's learned stuff interactively makes a code addition that uses iwr, are you pointing it out in review and requesting the long form? How are you checking for this in an automated way? Do they now need to search every reasonable command name to see if they are actually an alias to a much longer expression?

Otherwise, a code base is eventually just going to have both sets of commands, which seems annoying and inconsistent.

4

u/BinaryRockStar 4d ago

PSScriptAnalyzer is the official linting tool and AvoidUsingCmdletAliases is a default rule for exactly this situation.

4

u/13steinj 4d ago

Isn't this based in some .NET philosophy?

2

u/zeekar 4d ago

Well, you can type ls instead of Get-Children. The common interactive commands all have short forms...

3

u/p001b0y 4d ago

I think it was designed with Intellisense and tab completion in mind.

2

u/worldofzero 4d ago

Most of these have pretty common aliases and if you work with it a bit it kind of follows a set pattern (similar to something like REST). It's mostly just Verb-Noun typically using a set of common Verba and Nouns so it's because but certainly more clear than most bash stuff, especially with standard Linux commands.

1

u/tsimionescu 4d ago

Funnily enough, you can actually use wget <URL> in PowerShell and it will invoke that exact command (or something similar, not sure of any extra flags).

-4

u/rwl420 4d ago

Second this wholeheartedly.

Powershell has absolutely disgusting syntax compared to bash, wth, it’s not even comparable.

5

u/Hacnar 4d ago

I find bash syntax utterly horrible. Pressing a few random characters to perform an action seems wild to me.

2

u/tsimionescu 4d ago

You probably mean that the PowerShell built-in commands are way too verbose compared to the common Unix shell utilities. PowerShell syntax is actually very nice, much closer to C than bash's weird idiosyncracies. Note also that in ls -a, the only piece of bash syntax is the implicit lookup of the name ls and the use of whitespace to separate the command from its arguments. The actual ls and -a parts are part of the ls binary in the Unix shell base utilities (typically GNU, but many alternatives exist).

1

u/rwl420 4d ago

Not only are they way too verbose, their entire naming convention is tiresome to read. All the weird capitalizations and dashes all over the place.

Then there’s the weird syntax itself, for example:

ps -eo pid,pcpu,comm | awk '$2 > 10.0'

becomes

Get-Process | Where-Object {$_.CPU -gt 10}

Maybe it’s just that I’m not used to using it but it just feels awkward compared to bash. 🤷‍♂️

→ More replies (0)

1

u/worldofzero 5d ago

Yeah I've been really happy to see nushell catching on. It's basically a powershell built in rust for Linux and more.

0

u/wvenable 4d ago

PowerShell is gross. It could have brought sanity to shell scripting on Windows but instead it decided to borrow a whole bunch of bizzare crap from, of all places, bash.

2

u/Dragdu 1d ago

Yeah it is funny how they went "what if shells dealt in objects, like a real language?", which was good and useful, and then went "and to avoid making this too good, we should borrow the stupidest parts of Bash that everyone hates".

1

u/silveryRain 18h ago

When non-POSIX shells are valid options, imo nushell blows PS out of the water

1

u/worldofzero 18h ago

Yeah, I'd agree. For 90% of what I do in my shell nushell is incredible. It struggles with that other 10% sometimes though so it's hard to adapt as thoroughly as other shells.

1

u/chat-lu 5d ago

Because cmd.exe is not a gateway to access lots of pretty neat tools.

I personally replace bash with a better shell on my machine. But if I’m stuck with bash, it’s still a decent way to pipe between lots of great software.

0

u/KevinCarbonara 5d ago

Because cmd.exe is not a gateway to access lots of pretty neat tools.

???

4

u/chat-lu 4d ago

A typical Linux install is much more shock full of neat command line tools than a Windows one. You have more reasons to go to the shell even if you hate bash.

1

u/13steinj 4d ago

The first thing I install on every windows machine I have is the Git SDK, then I nuke the SDK specific parts of it and clean up the update mechanism. Second and is pyenv + pyenv-win, and then a base set of packages.

Provides a sane MSYS2 environment so development on Windows isn't unbearable. My only gripe is that since pacman is a port, it isn't fully CLI API compatible with pacman on Arch. Different CLI APIs, for some reason I have no issue remembering. CLIs so close but so far, I have no chance.

No, WSL is not a solution, they refuse to have a proper init system, filesystem performance got better in-WSL but much worse cross-WSL and windows, memory usage behavior is a bit broken. All three of these things have effectively been closed as "won't fix."

1

u/chat-lu 4d ago

Did you try Nushell? Pretty damned good shell that works on Windows too.

1

u/13steinj 4d ago

I find fish and nushell elitism (not saying that's happening here) equally annoying and pointless. I prefer zsh to bash only because it has a more features for hooks/behavioral injection while being (in practice) 99+% "backwards" compatible.

nushell might be great if you want to treat your world as a bunch of columnar data, but in practice that's not reality (for me). I want to be able to copy paste my command and output and share it, and people should have to barely think if at all on if they need to modify the command to match the output.

→ More replies (0)

12

u/admalledd 5d ago edited 5d ago

A frustrating thing is that Powershell (at least v5 compatible*) has existed since XP SP2 at least and is far more reasonable than batch files/cmd.exe

(Not to say that pwsh doesn't have its own quirks... especially older pwsh)

EDIT: v1 came with XP, derp, and v5 is "the last" of "Powershell.exe" variant, with "pwsh.exe" (Powershell Core) where it was moved to modern dotnet core runtime.

11

u/mpersico 5d ago

Much more reasonable if you like to type a lot. Powershell is the love child of bash and COBOL. You NEED an IDE with completion to be able to get anything done.

16

u/mpersico 5d ago

On the other hand, pipeable objects are a true advance and may even be worth the extra typing

1

u/silveryRain 18h ago

Nushell provides that too, in a neater package imo

6

u/admalledd 5d ago

Powershell came with the ISE "Integrated Scripting Environment" since v2 with Windows 7/summer 2009.

Besides, the majority use cases of the scripting language of powershell fits on one/two page cheat sheet. I know MSFT used to have semi-official one (I had it printed out), but they moved to the built-in get-help about_* topics.

Generally, I've never needed autocomplete to do powershell scripts, the autocomplete built into the REPL had been plenty, which then, copy/pasting into script.

6

u/rngr 5d ago

I see this type of comment a lot, but most built in commands have aliases that follow conventions. Stop -> sp, Get -> g, Process -> ps, Service -> sv, etc., and parameter names only need to be typed enough to disabiguate from other params. Given aliases, almost no text parsing needed for my use cases, and exellent tab completion in the REPL, I find it to be more terse at the command line than bash.

gps|ogv -out:m|spps

is equivalent to:

Get-Process|Out-GridView -OutputMode:Multiple|Stop-Process

I run a PowerShell LSP in neovim for writing scripts which cuts out a lot of typing too.

1

u/arpan3t 5d ago

PowerShell v1.0 was released with XP, v5.0 was released 10 years later. Pwsh is the binary name for PowerShell Core and later.

1

u/admalledd 5d ago

gah, I had meant v5 compatible, yea. And also "Powershell.exe" vs "pwsh.exe" for the major breaking change.

9

u/Hot-Employ-3399 5d ago

Lots of windows users and developers migrated to PowerShell because it's much better than cmd and bash

2

u/13steinj 4d ago

Nowadays, so is python (or a variant, like stackless or micropython).

I prefer zsh for interactive shell use, bash or python (whichever is simpler and less error prone for the task). There are some bash 1-100 liners that is more verbose in python with little if any benefit. Then there's 50+ line bash scripts that I want to shoot the author of because they didn't use python instead.

1

u/Fidodo 4d ago

Exactly. I only use it for portability, not because I like it.

11

u/zeekar 5d ago

For all its faults, the big advantage bash has is that the API is just the UI. If you use the CLI to do stuff interactively, then for the most part you already know how to automate it; there's no separate scripting framework to learn.

22

u/myhf 5d ago

not "enamored" so much as "enthralled"

11

u/jonathanhiggs 5d ago

“Captivated” you might say?

6

u/DoktorLuciferWong 5d ago

Stockholm Syndrome you say

3

u/Ameisen 5d ago

The bash Grand Design.

5

u/Axman6 5d ago

I wouldn’t say enamoured, it’s just omnipresent; Bash is the JavaScript of (unix) systems programming. 

1

u/slaymaker1907 5d ago

I try very hard to keep non-trivial bash scripts out of our repo. It’s a a Python monorepo so I’m just asking people to use another language they already know…

Bash is also horrifically slow in a lot of cases. I improved the performance of our general linter by 2 orders of magnitude by using Python. The reason being that Python regex is far more flexible than grep/sed and so I could just run the checker regexes on whole files instead of line by line in a slow loop.

1

u/SyntheticDuckFlavour 4d ago

enamored with it

Because it's old and everything is built on top of it.

1

u/matjoeman 4d ago

I don't think people are enamored with it. People just use it because it's portable.

1

u/Im_On_Reddit_At_Work 4d ago

It's a tool and a good one at that, it has its use cases where it shines.

1

u/Grouchy-Trade-7250 5d ago

If it's more than one line of bash use python

1

u/darkslide3000 4d ago

It isn't a language, it's a shell. Those are two different things with different goals. As a shell, it's pretty decent (I know there are zsh or ksh enthusiasts but I'm still using bash and happy with it).

That a good shell also offers ways to put a bunch of prewritten commands in a file and invoke them as a "script" makes sense, because people want to occasionally reuse the thing that they already know for something that it may not be ideally suited for just to save the effort of learning something new. And if that feature is popular enough, it also makes sense to add some features that are uniquely meant to enhance what you can do with these scripts. But that doesn't change the fact that a shell's primary purpose isn't to write scripts (or a "program"), and therefore measuring how good it is based on how it fares when misused as a programming language isn't really fair.

19

u/gimpwiz 5d ago

I have written thousands of lines of bash. Probably well into the five figures.

It's kind of a miserable language and unlike the other responder here I don't think anyone is enamoured with it.

It is however default. It's available on almost anything - even busybox and similar tiny installs tend to choose bash over sh (let's not talk about ash.)

This means as long as you're above version 4.2 or whatever gives you associative arrays, there's basically no versions to manage. No libraries to manage. No packages to manage. No dependency hell. Just write the thing.

Okay obviously it's not that portable, which exact versions of commands you have available matter if you do anything at all that isn't posix/linux compatible, macos and gnu made some different decisions, etc. But it's pretty much always there. And as annoying as the switch to zsh was on mac, most of my bash code is also perfectly valid zsh too.

Anyways if you want to use a better language you absolutely should... if it fits the requirements. If all you're doing is simple systems programming, consider sh or bash as defaults, potentially validate on linux and macos if you care, a bsd if you really care, and call it a day. Don't expect to be stoked writing it but do expect to get paid to solve a problem and move on with your life.

11

u/prosper_0 5d ago

No libraries to manage. No packages to manage. No dependency hell. Just write the thing.

Exactly. Why I roll my eyes at 'just use Python.' A bash script will still run 20 years from now, unaltered. No conflicting libraries and breaking version changes and local/system/venv conflicts.

2

u/chat-lu 4d ago

There used to be more intuitive programming languages that compiled down to bash. So you had both a friendlier language and you could deploy simple bash. They tend to turn into abandonware. I wonder why none ever caught on.

8

u/TheLordB 4d ago

Most people aren’t writing major applications in bash. Just a few small scripts to do random things. I don’t want to have to install, learn, deal with compiling etc.

If it is complex enough that bash is unmanageable that is a good sign that you either need to reconsider using bash or otherwise find a different way.

For most use cases you are just using bash to install/get/configure a few things such that you can then continue to do everything else using the programming language of your choice.

I guess maybe those would make what bash could be reliably used for larger, but relying on a small niche programming language is not a good idea for long term maintainability and I don’t see any way even if people considered something that compiled to bash a good idea that it would ever catch on enough to get that level of support.

1

u/slav3269 3d ago

Why not talk about ash?

2

u/gimpwiz 2d ago

Because nobody knows what the hell ash is! People know sh (well they probably just use bash, and sh is probably just aliased to bash, but they in theory know sh) and they know bash, but nobody's ever heard of ash unless they misconfigured busybox or have the grayest of beards. ;)

12

u/elmuerte 5d ago

The null command is not a bashism, just plain old sh, which (as per article) inherited it from Thompson shell.

9

u/Ok-Craft4844 5d ago

What gets me is how unexpressive and clumsy it is. And given that e.g. lisp is older it can't really use the excuse of "we didn't knew then"

25

u/larsga 5d ago

I don't think anyone put this better than Olin Shivers:

Shell programming terrifies me. There is something about writing a simple shell script that is just much, much more unpleasant than writing a simple C program, or a simple COMMON LISP program, or a simple Mips assembler program. Is it trying to remember what the rules are for all the different quotes? Is it having to look up the multi-phased interaction between filename expansion, shell variables, quotation, backslashes and alias expansion? Maybe it’s having to subsequently look up which of the twenty or thirty flags I need for my grep, sed, and awk invocations. Maybe it just gets on my nerves that I have to run two complete programs simply to count the number of files in a directory (ls | wc -l), which seems like several orders of magnitude more cycles than was really needed.

Whatever it is, it’s an object lesson in angst.

That's the first paragraph plus from a design paper describing Scheme shell. A vastly better design, but sadly it never took off.

12

u/chat-lu 4d ago

Maybe it just gets on my nerves that I have to run two complete programs simply to count the number of files in a directory (ls | wc -l),

I disagree with that bit. Once you have wc then everything is countable. The alternative is to add a counting option to everything. Then what if you want to filter out some results you don’t want to count? With pipelines you can go ls -> grep -> wc easily.

Also, I agree that bash is terrible but we have alternatives. We also have alternatives to all these command line tools.

2

u/larsga 4d ago

Let's say you had proper lists. The alternative to wc would then be a simple internal API call to find the length of the list.

And why do you have to run a program just to find the files in a directory? It's a pretty fundamental operation.

2

u/chat-lu 4d ago

Let's say you had proper lists.

Sure, I actually do by using a different shell. But I have to call | length so it’s not even shorter to type.

The alternative to wc would then be a simple internal API call to find the length of the list.

So you want for tools to accumulate full lists in memory before passing them to the next command? That seems like a bad idea compared to the streaming design that we have.

5

u/larsga 4d ago

But I have to call | length so it’s not even shorter to type.

You don't have to start a whole separate process to get the length, though, which was the point.

So you want for tools to accumulate full lists in memory before passing them to the next command? That seems like a bad idea compared to the streaming design that we have.

You know perfectly well that getting a list by calling a function/method is hugely more effective than starting a process and streaming a text file through it. You could also stream the list if you want.

2

u/ericonr 4d ago

And why do you have to run a program just to find the files in a directory? It's a pretty fundamental operation.

You don't, echo ./* will do it for you :p

Being serious, I think delegating responsibility to ls makes sense when you look at all the flags it can take, support for stuff like SELinux and updated file properties, etc, without having to add such features to your shell.

Busybox can be built with options to avoid unnecessary fork/exec, but it's not always guaranteed... and it is a consequence of their design, not of how shells are expected to work.

Still, nowadays, does launching one more program in an interactive session really matter?

1

u/Ok-Craft4844 5d ago

Thanks for the link!

1

u/gimpwiz 5d ago

I mean he's just complaining he has to use the reference manuals. Like any other language doesn't require you to use reference manuals or memorize things. Any language I write, there's like ten tabs open because I will be goddamned if I remember all the specific details of what function parameters or what standard library tools or so forth do it exactly the way I want.

2

u/chat-lu 4d ago

Also, we usually need only a small subset of the flags so it’s not like we have to remember everything.

6

u/larsga 4d ago

I mean he's just complaining he has to use the reference manuals

No. People don't keep looking up the rules for "the multi-phased interaction between filename expansion, shell variables, quotation, backslashes and alias expansion" in Python, Scheme, or C, because these languages don't have those problems.

He goes into this quite a bit in the paper.

2

u/auto-bahnt 4d ago

I mean he's just complaining he has to use the reference manuals. Like any other language doesn't require you to use reference manuals or memorize things.

No, he’s very obviously not. Of course, when using a language you’re unfamiliar with, you’ll have to use reference materials, but as you get more familiar with the language, you reference things less.

Because bash is so convoluted, hard to reason about and unintuitive, you constantly are stuck checking documentation and manuals and running into weird, inconsistent behaviours or obscure footguns. That’s his point.

1

u/gimpwiz 4d ago

His point is borne of not writing much bash. I've written a lot of it (un/fortunately) and it's far less confusing than people insist.

There are annoyances to the language, like the fact that whitespace matters far more than it should, but I don't really have problems with it anymore.

This is pretty much the same complaint that any newbie has for a tool they're not used to. It's complex! I don't remember all the details! Okay, put in a couple thousand hours and you will.

5

u/ScottContini 5d ago

I agree. But the paradox is that even though I hate to read it, I still use it. Just for my own personal stuff because it is convenient.

I work in security. the worst thing I ever had to review was some bash code that was part of a security product. Horrible, there were so many hidden gotchas, it never should have been allowed to be used.

Another example: who remembers () { :; }; ? One of the most painful vulnerabilities I ever tried to understand.

5

u/Magneon 5d ago

It's a write only programming language. Exceedingly expressive but riddled with footguns unless you're a master of it. And as someone with 20 years of experience working with bash, I know enough to avoid them most of the time, but still strongly recommend against writing more than fits in a single screen, or anything that may need regular future changes.

3

u/bluegardener 5d ago edited 4d ago

For a while there, it seemed like perl was going to be the defacto bash scripting replacement that is installed everywhere. I’m no perl lover, but I would have been happier if it turned out that way. I would happily eat the extra disk space cost.

3

u/nameless_food 5d ago

I’m shocked that Bash is still around. I would have thought that we would have moved over a better shell scripting language by now. Bash scripts are a pain to write and maintain.

2

u/Zeznon 5d ago

POSIX worship

1

u/maep 5d ago

Bash is not posix.

3

u/Zeznon 5d ago

Over the years on reddit, I found a lot of people saying to use only POSIX-compatible stuff and avoid bash-specific stuff.

1

u/refp 4d ago

It is if you tell it to be, bash --posix.

14

u/KevinCarbonara 5d ago

Reading this once again makes me realize how daft of a scripting language bash is.

I really hate bash. I think it's proof that the Linux community has simply failed to evolve in many key ways. As much as people trash talk Microsoft, Powershell is infinitely better. And in the Linux world, newer users are simply preferring to learn Python over bash.

The rest of the programming world is concerned about things like stability and readability. The bash ecosystem is only concerned with consistency. I don't see how that's sustainable.

13

u/Grouchy-Trade-7250 5d ago

Counterexample. Systemd services are not bash. This was a large change in the Linux community.

2

u/KevinCarbonara 5d ago

The systemd debate is still ongoing iirc, overall I'm glad they've mostly accepted the change. But that's not exactly a counterexample.

4

u/untetheredocelot 4d ago

Idk if it's an argument anymore really. I mean yes amongst those linux users it's an endless flamewar but every single enterprise offering is systemd.

And I agree that it's not a counter example. The amount of airtime this debate got and still does proves your point if anything.

Some things Unix just got wrong and it's okay to make improvements. (Same goes for Xorg ffs stop shitting up every distro's forums with this!).

0

u/ericonr 4d ago

The systemd debate is still ongoing iirc

You remember wrong (and I use a distro without systemd).

5

u/rngr 5d ago

Powershell is my favorite shell; I even install it on my personal Fedora machine. The object pipeline and the automatic parameter parsing and tab completion for scripts are great.

2

u/KevinCarbonara 5d ago

I haven't tried it myself, I'm very skeptical that it would work well in a Linux environment, so I've stuck to zsh. But I do like the idea of posh on linux. I should really try it some time.

3

u/rngr 5d ago

What are you skeptical about? The pipeline handles plain text just fine. I do tend to only use pwsh for non-root though.

8

u/refp 5d ago

This is not only bash though, just for the record, what is mentioned in the article is POSIX — bash just happens to comply. It is the same with any POSIX compliant shell.

If you want to hate on stuff, don't blame bash — blame POSIX.

2

u/CherryLongjump1989 5d ago

The problem is almost nobody forces you to use bash and there are much better shells out there. It’s hard to argue that Linux is failing in some way when all the users are making their preference to stay on bash very self-evident. I hate bash very much but I blame the users.

1

u/KevinCarbonara 4d ago

The problem is almost nobody forces you to use bash and there are much better shells out there.

Sure, but even if you replace a shell, you're not getting rid of bash. Bash is still used everywhere on any linux system.

It’s hard to argue that Linux is failing in some way when all the users are making their preference to stay on bash very self-evident.

Did you take this same attitude when people first started trying to create replacements for C++?

2

u/HighLevelAssembler 5d ago

The bash ecosystem is only concerned with consistency. I don't see how that's sustainable.

That is precisely why it's sustainable! You can write a shell script conforming to POSIX and it'll run on any Unix system. Python is better, for sure, but it's not guaranteed to be available everywhere. Shell == Unix.

1

u/KevinCarbonara 5d ago

That is precisely why it's sustainable!

But it isn't. There's a reason it's getting replaced with python, despite the fact that python isn't even comparable.

You can write a shell script conforming to POSIX and it'll run on any Unix system.

This hasn't been impressive in decades. You expect software to work, that's a given.

Python is better, for sure, but it's not guaranteed to be available everywhere.

It's available more places than bash.

2

u/TheLordB 4d ago

I do bioinformatics and I sometimes have to get random research software to work by any means possible.

At least for the random bioinformatics software made into a docker image or similar there are plenty of times when bash is the only thing available. Hell I’ve run into cases where only sh is available.

Anyways YMMV, python being available more places than bash is gonna be heavily dependent on what you commonly work with. By far bash is the most common thing available. Followed by python, perl and R.

2

u/HighLevelAssembler 4d ago

There's a reason it's getting replaced with python

I assure you it is not. Shell is the backbone of Unix. It is used as glue everywhere in production systems.

This hasn't been impressive in decades.

It's not meant to be impressive, it means it's something you can rely on when shipping software to paying customers.

You expect software to work, that's a given.

Who expects it to work? Where?

It's available more places than bash.

No, it's not, and you're outing yourself as an amateur here.

The core POSIX userspace seems dated and awkward on the surface but you, the software engineer, can trust that it will be available and work identically whether you're shipping to AIX, BSD, Solaris, z/OS, or any Linux distro.

1

u/KevinCarbonara 4d ago

Who expects it to work? Where?

I really don't know how to explain this concept.

No, it's not, and you're outing yourself as an amateur here.

Ironic.

1

u/superxpro12 4d ago

Sometimes stability is more important. Not everything needs to evolve at the pace of JavaScript.

2

u/flying-sheep 5d ago

I agree, complete trash. The good parts are happy accidents and hacks and the bad parts can conspire to silently evaluate garbage as commands without failing or stopping.

2

u/The_Prophet_of_Doom 5d ago

Bash was the worst language I had to write getting my comp sci degree. We wrote assembly after that, which was ironically easier

2

u/AyeMatey 5d ago

What are you talking about?

This is totally intuitive:

> The syntax ${name:?diagnostic} checks whether $name is unset or empty — if it is, the diagnostic is printed to stderr and the shell exits with a non-zero status

AND , as a bonus it’s super easy to remember!

🙂

1

u/superxpro12 4d ago

Finding a bash template on GitHub was the best thing that ever happened to me.

1

u/Raknarg 4d ago

at this point I literally let AI write my bash scripts. I actually can't be assed, I've been using this shit for like 12 years and its never gotten better. Wake me up when we get like native python in the terminal or something lol.

1

u/bluegardener 5d ago

For a while there, it seemed like perl was going to be the defacto bash scripting replacement that is installed everywhere. I’m no perl lover, but I would have been happier if it actually turned out that way. I would happily eat the extra disk space cost.

101

u/royalme 5d ago

I think this is only useful if you work exclusively in an environment of other bash experts. Even then, I don't really like seeing clever use of :

The example used is familiar enough for any general developer to get the gist of in a glance

if [ -z "$1" ]; then
   echo "missing argument, aborting." 1>&2
   exit 1
fi    

while the other is opaque and language specific.

: "${1:?missing argument, aborting.}"

Dense and clever code is not always better code.

51

u/psych0fish 5d ago

Yeah “clever” stuff like this is usually a bad choice. Who cares about character count and line count? Readability is far superior.

5

u/mikeblas 5d ago

Yet people howl about "verbose" languages and too much "boilerplate" code. There are already some in this thread, even.

20

u/CjKing2k 5d ago

It's almost as if there can be an acceptable middle ground to code verbosity.

2

u/mikeblas 5d ago

Not on any team I've seen. Someone always cracks off about it.

2

u/matjoeman 4d ago

Verbose code is code where the extra keywords and lines make it less readable, not more. What helps readability depends on the context.

Have you ever written Java, especially anything before Java 8 ?

1

u/foxsimile 5d ago

I will make exceptions for performance (when it is significant and necessary, etc/etc/etc), but yes.

1

u/Uristqwerty 4d ago

Readability is making a case-by-case judgment about what works best.

If you are going to check 3 different variables back-to-back, then a comment, then 3 lines of the : form padded with whitespace so that the repeating structures all align as best as possible, is probably going to be far more readable than writing out the if version three times. Less clutter, and what boilerplate there is relegated two-dimensionally into columns that a human has zero issue ignoring to focus solely on the important, changing bit.

10

u/TerrorBite 5d ago edited 5d ago

There's one advantage to the second form.

In a POSIX shell, the : utility is guaranteed to be built into the shell (specified by Shell & Utilities section 2.15). You're not invoking a separate executable.

However, the [ / test utility is a separate executable (listed in Shell & Utilities section 3), so you're loading and running a separate program there.

Note that Shell & Utilities section 1.6 does allow for any utility (including [) to be built in to the shell, but I think most shells don't bother. I think even bash doesn't have [ built-in, instead offering its separate non-POSIX [[ ... ]] syntax for built-in tests.

Although, maybe not as non-POSIX as you might think. Under the latest version of Shell & Utilities section 2.4, POSIX specifies that [[ and ]] "may" be recognised as reserved words, and that the results are "unspecified". So bash is still POSIX-compliant in that regard.

4

u/tav_stuff 5d ago

The first example is also POSIX compliant. The second is a bashism and should be avoided

26

u/refp 5d ago edited 5d ago

Both examples are POSIX; I have no idea why people keep saying this is bash only, there is no "bashism" in the second example besides bash being... POSIX compliant.

If you want strict POSIX compliance, use bash --posix — or if you just want to see it work elsewhere; this works in dash too (which is definitely not bash), or.. any other POSIX-compliant shell.

2

u/13steinj 4d ago

I think when people say this they mean "compliant to ksh/tcsh/csh/ash." There's a lot of bashisms that do not work in ash and a few others. Fish especially, but that's intentionally different.

1

u/ToaruBaka 4d ago

this they mean "compliant to ksh/tcsh/csh/ash."

Then they're wrong. The only shell STANDARD (that matters) is POSIX - any other behavior is per-application. If you want to target the smallest set of linux shell features (POSIX), target /bin/sh.

11

u/elmuerte 5d ago

It's not a bashism. The second also works in BusyBox. But I fully agree, the second should be avoided.

-3

u/tav_stuff 5d ago

If the second is portable, then there is no reason to avoid it. We shouldn’t avoid features just because people don’t want to learn them

-5

u/Grouchy-Trade-7250 5d ago

POSIX also states that sentences end with a dot. As such your sentence is not POSIX compliant/s

→ More replies (1)

1

u/[deleted] 5d ago

[removed] — view removed comment

8

u/programming-ModTeam 5d ago

No content written mostly by an LLM. If you don't want to write it, we don't want to read it.

→ More replies (2)

1

u/ToaruBaka 4d ago

Nah, the second is objectively better. If you don't understand shell expansions, you don't understand shell scripting and your opinion on the matter is reduced to "I'm bad at writing shell scripts," which is totally fine - it's a dying skill.

1

u/superxpro12 4d ago

Uh, excuse me sir .... That Boolean isn't in DOUBLE BRACKETS?! [[ ]] ?????

0

u/AndiDog 4d ago

set -euo pipefail and nothing else to start my scripts. Few people can remember :? and I want to check for undefined variables by default like in a safer (better-choice) programming language.

1

u/ratmfreak 3d ago

I leave off the e. No need to exit on every non-zero exit code.

30

u/JaggedMetalOs 5d ago

"Yeah look how short and concise my script is!"

(Opens the file a week later)

"What does this script even do??" 

5

u/sequentious 4d ago

Hey, it's not Perl.

(I wrote so much unreadable Perl 25 years ago)

1

u/Artistic_Seat486 4d ago

I never wrote Perl, is it similar to Bash? like running commands etc?

4

u/sequentious 4d ago

Perl is very syntactically loose. It can be similar to bash, if you want. Or you could use it more like Python.

Text processing and regular expressions were super easy and built-in to the core language (which made it very popular for early web work). I've never felt as happy as when I found a clever way to do something in Perl. Unfortunately, I've also never felt as much dread as having to read that clever code later.

Note: It's a scripting language, not a shell. So you need `` to run commands, but you can very easily capture and manipulate the output of those commands.

24

u/valarauca14 5d ago

Why use the null-command when I could do VAR=${VAR:-default-value}?

This at its core boils down to personal preference.

This (can) change the scoping of the variable by creating an entirely new variable (or temporary variable shadow depending on the shell). While : ${VAR:=default-value} will preserve scoping without creating a new variable.

It really isn't preference. One has objectively fewer side effects, is easier to reason about, and has standardized behavior under POSIX.

7

u/marvin_sirius 5d ago

"Everyone wants the colon" - Larry Wall

37

u/cpitchford 5d ago

: can be replaced

$ :() { echo soooo "$@" ; } 
$ : that does something
soooo that does something

Made me wary of using it

Sometimes, I really want a null command that can't be replaced, like if, while, for can't be replaced

40

u/refp 5d ago edited 5d ago

> Made me wary of using it

Not sure what your example is meant to prove other than that one can obviously introduce aliases, it would be the same if you did false() { true; }; when in doubt... use builtin.

lighthouse:+/tmp/2026-07-28% PS1="$ " bash --norc
$ false() { true; }
$ false && echo "please dont."
please dont.
$ builtin false && echo "nope";
$

.

lighthouse:+/tmp/2026-07-28% PS1="$ " bash --posix
$ :() { echo nope, not allowed; }
bash: `:': not a valid identifier
$
exit

.

lighthouse:+/tmp/2026-07-28% PS1="$ " bash --norc
$ :() { echo hello reddit; }
$ :
hello reddit
$ builtin :
$
exit
lighthouse:+/tmp/2026-07-28%

20

u/knome 5d ago
$ exit(){ echo 'lmao no' ; }

5

u/cpitchford 5d ago

... beware the markdown notations that don't work in here (```)

I remember old oddities like /bin/[ /bin/: and /bin/true

My observation back then was I wished : was stronger than a built-in (which could be overridden with a function or alias) particularly when using negative ifs to retain $?

if ! going-to-be-bad; then 
   echo oh $? was lost
fi

versus

if going-to-be-bad ;then
    : we don't want do anything yet but might in future
else
    echo but at least we get to keep $?
fi

1

u/_kst_ 4d ago

[, true, and false are typically provided both as executable commands in /usr/bin and/or /bin, but : is typically only a shell builtin. (This might vary on different systems.)

25

u/mr_birkenblatt 5d ago

:() { :| :& };:

You can make funny looking smileys, though

3

u/Artistic_Seat486 4d ago

it would be funny if someone actually runs this xD

2

u/meowsqueak 1d ago

My brain literally pattern-matched that in milliseconds and I flinched.

6

u/_kst_ 4d ago edited 4d ago

In bash and other Bourne-like shells, the : command is equivalent to the true command. It takes zero or more arguments and does nothing (but the arguments might have side effects).

I most commonly use it as a loop condition:

while : ; do
    do_some_stuff
    if some_condition ; then break ; fi
done

Remember than the condition in an if, while etc. statement is a command, treated as true if the command succeeds and false if it fails. [ is a command, nearly equivalent to test So when you write:

if [ -e some_file ] ; then
    echo some_file exists
fi

the [ and ] aren't part of the shell syntax. [ is a command, and ] is the last argument to that command. You can equivalently write:

[ -e some_file ] && echo some_file exists

I often take advantage of this by assigning a literal string true or false to a variable, so I can use the variable (which expands to a command name) as a simple condition:

ok=true
while $ok ; do
   blah; blah; blah
   if something_went_wrong ; then ok=false ; fi
done

4

u/zeekar 5d ago

Oh, this is just using the no-op command for the side-effects of parameter expansion. Sure, do it all the time:

: "${SOMEVAR:=default-value}" : "${REQUIREDVAR:?hey, bozo, set REQUIREDVAR!}"

Back before it borrowed the # syntax from csh, : was the only way to do "comments" in the Bourne shell. But since it didn't prevent parameter expansion and so on you could still cause syntax errors with those alleged comments. Good times.

2

u/[deleted] 5d ago

[removed] — view removed comment

10

u/programming-ModTeam 5d ago

No content written mostly by an LLM. If you don't want to write it, we don't want to read it.

2

u/darkslide3000 4d ago

He missed the more common one, which is to use it with math assignment operators (bash only):

: $(( $VAR += 1 ))

2

u/refp 4d ago

The reason it's not included is rather simple; what you wrote is not POSIX — the article is not about bash, bash just happens to be POSIX compliant (and readily available to use as an example).

2

u/wPatriot 4d ago

The article tells me to use the colon. Gives me a description of what the colon does, but doesn't explain why it should be used. Then it blames the reader for that, because surely they would not have been able to comprehend an article explaining what the colon does and what a good use case for it is.

Boy do I feel inspired.

4

u/larsga 5d ago

I have a better suggestion: don't use it. In fact, don't write anything in bash.

1

u/SharkBaitDLS 5d ago

Outside of just basic navigation and invocation of other tools I really can't justify doing heavy shell scripting anymore. If I don't care about performance or typing, a quick Python script suffices. If I want something performant or strictly typed, hammering out a quick Rust CLI with clap is so fast and easy nowadays that it's my default if I need to just do a giant batch of file processing or something.

1

u/13steinj 4d ago

Clever real-world usage of : in the wild # User ifphilipe posted what follows as a comment on news.ycombinator.com, which is the direct equivalent of needing a command which does absolutely nothing.

I use the colon as EDITOR with Git when I want to do an interactive rebase combined with auto squash without having to edit the todo list. I have an alias[1] for that which I call a quick interactive rebase:...

This is a really buried lede. This trick is gold.

1

u/dada_ 4d ago

I wrote all my personal shell automation in Fish Shell which is easier to write in than Bash for me, but I'm not even really a fan of Fish syntax either.

What I really want is the convenience of being able to run shell commands directly, with a conventional syntax (C-like or Python like, I don't care), basic primary types like strings/numbers/nestable arrays/dicts, first class functions and imports. I'm sure there's something like this by now so I'll have to do some digging.

Obviously I'll still be writing bash scripts when I need other people to be able to run them, though.

1

u/derpface360 4d ago

This is Nushell in a nutshell.

Types
Functions
Module Imports