Why do I get "/bin/sh: Argument list too long" when passing quoted arguments? Why do I get "/bin/sh: Argument list too long" when passing quoted arguments? bash bash

Why do I get "/bin/sh: Argument list too long" when passing quoted arguments?


TL;DR

A single argument must be shorter than MAX_ARG_STRLEN.

Analysis

According to this link:

And as additional limit since 2.6.23, one argument must not be longer than MAX_ARG_STRLEN (131072). This might become relevant if you generate a long call like "sh -c 'generated with long arguments'".

This is exactly the "problem" identified by the OP. While the number of arguments allowed may be quite large (see getconf ARG_MAX), when you pass a quoted command to /bin/sh the shell interprets the quoted command as a single string. In the OP's example, it is this single string that exceeds the MAX_ARG_STRLEN limit, not the length of the expanded argument list.

Implementation Specific

Argument limits are implementation specific. However, this Linux Journal article suggests several ways to work around them, including increasing system limits. This may not be directly applicable to the OP, but it nonetheless useful in the general case.

Do Something Else

The OP's issue isn't actually a real problem. The question is imposing an arbitrary constraint that doesn't solve a real-world problem.

You can work around this easily enough by using loops. For example, with Bash 4:

for i in {1..100000}; do /bin/sh -c "/bin/true $i"; done

works just fine. It will certainly be slow, since you're spawning a process on each pass through the loop, but it certainly gets around the command-line limit you're experiencing.

Describe Your Real Problem

If a loop doesn't resolve your issue, please update the question to describe the problem you're actually trying to solve using really long argument lists. Exploring arbitrary line-length limits is an academic exercise, and not on-topic for Stack Overflow.


I don't get that error message. My secret? Single quotes:

/bin/sh -c '/bin/true $(seq 1 100000)'

If I use double quotes, I get that error with every shell:

$ /bin/sh -c "/bin/true $(seq 1 100000)"-bash: /bin/sh: Argument list too long$ /bin/bash -c "/bin/true $(seq 1 100000)"-bash: /bin/bash: Argument list too long$ /bin/ksh -c "/bin/true $(seq 1 100000)"-bash: /bin/ksh: Argument list too long$ /bin/zsh -c "/bin/true $(seq 1 100000)"-bash: /bin/zsh: Argument list too long

The argument list gets expanded in the current shell when double quotes are used as evidenced by the fact that Bash is the one issuing the error "-bash: ..." regardless of the shell being used to run the command. On my system sh is Dash, by the way.

This holds true even for other "host" shells:

$ dash$ /bin/bash -c '/bin/true $(seq 1 100000)'$ /bin/bash -c "/bin/true $(seq 1 100000)"dash: /bin/bash: Argument list too long

Patient: Doctor, it hurts when I do this."
Doctor: Don't do that.


Make a file with #!/bin/sh as the first line, then the put rest of your command on subsequent lines? :)

More seriously, you can also read commands from STDIN using the -s option, so you can generate your long command line and pipe it in to /bin/sh -s