bash and readline: tab completion in a user input loop?

After trying a custom completion script that I know works (I use it every day) and running into the same issue (when rigging it up similar to yours), I decided to snoop through the bash 4.1 source, and found this interesting block in bash-4.1/builtins/read.def:edit_line():

old_attempted_completion_function = rl_attempted_completion_function;rl_attempted_completion_function = (rl_completion_func_t *)NULL;if (itext)  {    old_startup_hook = rl_startup_hook;    rl_startup_hook = set_itext;    deftext = itext;  }ret = readline (p);rl_attempted_completion_function = old_attempted_completion_function;old_attempted_completion_function = (rl_completion_func_t *)NULL;

It appears that before readline() is called, it resets the completion function to null for some reason that only a bash-hacking long beard might know. Thus, doing this with the read builtin may simply be hard-coded to be disabled.

EDIT: Some more on this: The wrapping code to stop completion in the read builtin occurred between bash-2.05a and bash-2.05b. I found this note in that version's bash-2.05b/CWRU/changelog file:

• edit_line (called by read -e) now just does readline's filename completion by setting rl_attempted_completion_function to NULL, since e.g., doing command completion for the first word on the line wasn't really useful

I think it's a legacy oversight, and since programmable completion has come a long way, what you're doing is useful. Maybe you can ask them to add it back in, or just patch it yourself, if that'd be feasible for what you're doing.

Afraid I don't have a different solution aside from what you've come up with so far, but at least we know why it doesn't work with read.

EDIT2: Right, here's a patch I just tested that seems to "work". Passes all unit and reg tests, and shows this output from your script when run using the patched bash, as you expected:

$ ./tabcompl.shwaiting for commands-> **<TAB>**TAB     hit     output  should  these   this    when    words   you->

As you'll see, I just commented out those 4 lines and some timer code to reset the rl_attempted_completion_function when read -t is specified and a timeout occurs, which is no longer necessary. If you're going to send Chet something, you may wish to excise the entirety of the rl_attempted_completion_function junk first, but this will at least allow your script to behave properly.

Patch:

--- bash-4.1/builtins/read.def     2009-10-09 00:35:46.000000000 +0900+++ bash-4.1-patched/builtins/read.def     2011-01-20 07:14:43.000000000 +0900@@ -394,10 +394,12 @@        }       old_alrm = set_signal_handler (SIGALRM, sigalrm);       add_unwind_protect (reset_alarm, (char *)NULL);+/* #if defined (READLINE)       if (edit)        add_unwind_protect (reset_attempted_completion_function, (char *)NULL); #endif+*/       falarm (tmsec, tmusec);     }@@ -914,8 +916,10 @@   if (bash_readline_initialized == 0)     initialize_readline ();+/*   old_attempted_completion_function = rl_attempted_completion_function;   rl_attempted_completion_function = (rl_completion_func_t *)NULL;+*/   if (itext)     {       old_startup_hook = rl_startup_hook;@@ -923,8 +927,10 @@       deftext = itext;     }   ret = readline (p);+/*   rl_attempted_completion_function = old_attempted_completion_function;   old_attempted_completion_function = (rl_completion_func_t *)NULL;+*/   if (ret == 0)     return ret;

Keep in mind the patched bash would have to be distributed or made available somehow wherever people would be using your script...

I've been struggling with same issue for some time now and I think I have a solution that works, in my real world case I'm using compgen to generate possible completions. But here is an example that illustrates the core logic:

#!/bin/bashset -o emacs;tab() {  READLINE_LINE="foobar"  READLINE_POINT="${#READLINE_LINE}"}bind -x '"\t":"tab"';read -ep "$ ";

Set the emacs option to enable key binding, bind the tab key to a function, change READLINE_LINE to update the line after the prompt, and set READLINE_POINT to reflect the line's new longer length.

In my use case I actually mimic the COMP_WORDS, COMP_CWORD and COMPREPLY variables but this should be sufficient to understand how to go about adding custom tab completion when using read -ep.

You must update READLINE_LINE to change the prompt line (completion single match), printing to stdin prints before the prompt as readline has put the terminal in raw mode and is capturing input.

Well, it seems i finally stumped on the answer, and it sadly is: actually there isn't full support for readline when interfacing it via "read -e".

The answer is given by the BASH maintainer, Chet Ramey. In this thread the exact same issue is addressed:

I'm writing a script with a command line interpreter and I can most thingsworking (eg. history etc.) except for one thing. The filename completionworks well for some of the commands, but I'd like to use other completionoptions for others. Works well from the "real" command line, but I can'tget it to work properly in my "read -e, eval" loop..

You won't be able to do it. `read -e' uses only the readline defaultcompletions.

Chet

So, unless i'm missing something //rant// while bash hands to the programmer the "read -e" mechanism as the mean for full, proper CLI user interfacing, the functionality is crippled, even though the underlying mechanism (readline) works and integrates with the rest of bash flawlessly //end rant//

I have exposed the question to the kind folks at #bash in freenode and been suggested to try with a Readline wrapper like rlfe or rlwrap.

Finally, i contacted Chet himself by mail yesterday, and he confirmed that this is by design, and that he doesn't feel like changing it as the only use case for programmable completion into "read", i.e. presenting a list of commands to the script user, doesn't look like a compelling reason to spend time working on this. Nevertheless he expressed that in case someone actually does the work he would certainly look at the result.

IMHO, not considering worth of the effort the ability to bring up a full CLI with just 5 lines of code, something one wish were possible in a lot of languages, is a mistake.

In this context, i think Simon's answer is brilliant and right in place. I'll try to follow your steps and perhaps with some luck i'll get more info. Been a while since i don't hack in C however, and i assume the amount of code i'll have to grasp to implement will not be trivial. But anyway i'll try.