How to automatically decompress a custom compressed file when opened in emacs?

Maybe, the following code is helpful. By the way, it is interesting that the author of jka-compr did not take the evaluation of the program arguments into consideration and did not provide filename as one of the possible evaluable arguments.

I do not know your compression/uncompression program. Therefore, I just used cat for files ending with .cat instead for testing.

(defadvice jka-compr-info-compress-args (around eval-args activate)  "Evaluate program arguments"  (setq ad-return-value (mapcar 'eval (aref info 3))))(defadvice jka-compr-info-uncompress-args (around eval-args activate)  "Evaluate program arguments"  (setq ad-return-value (mapcar 'eval (aref info 6))))(add-to-list 'jka-compr-compression-info-list ["\\.cat\\'" "cat" "cat" ("-")                           "cat uncompress" "cat" (filename) nil t ""])(add-to-list 'auto-mode-alist '("\\.cat\\'" nil jka-compr))(add-to-list 'file-name-handler-alist '("\\.cat\\'" . jka-compr-handler))

Corresponding enhancement-request:

http://debbugs.gnu.org/cgi/bugreport.cgi?msg=5;att=1;bug=16454

It's hard to test this without knowing what your custom compression and decompression tools are, but using xargs seems to work for me on Linux:

(if (fboundp 'auto-compression-mode)    (auto-compression-mode 0)  (require 'jka-compr))(add-to-list 'jka-compr-compression-info-list             ["\\.customcomm\\'"              "custom compressing" "xargs" ("customcom" "-c")              "custom decompressing" "xargs" ("customcom" "-d")              nil t])(auto-compression-mode 1)

Note that you may still have problems if the output of this command is not to STDOUT.

You might like to try and use ("-d" "-") as argument, in case customcom follows the custom of interpreting - to mean "use stdin". Or depending on your OS, you could try ("-d" "/dev/stdin"): it should work under GNU/Linux at least.