If I write my own shell---which I may very well do at some point---there's a particular process model I'd like to embed in it. To wit: in UNIX right now, each program in a pipeline has a single input stream and two output streams, with files/sockets/&c for other kinds of communication.

A pipeline of functions foo | bar | baz looks kind of like this

keyboard -> stdin     stdout -> stdin     stdout -> stdin     stdout ---+
                 >foo<               >bar<               >baz<          |
                      stderr -+           stderr -+           stderr -+ |
                              |                   |                   | |
                              v                   v                   v v
                            [..................terminal...................]

Which works pretty well. You can do some pretty nice things with redirecting stderr to here and stdin from there and so forth, and it enables some nice terse shell invocations.

I'd like that basic system to be preserved, but with the ability to easily create other named streams. For example, imagine a hypothetical version of wc which still outputs the relevant data to stdout, but also has three other streams with these names:

                 / newlines
                 | words
stdin -> wc-s -> | bytes
                 | stdout
                 \ stderr

You can always see the normal output of wc on stdout:

gdsh$ wc-s *
       2       3       6 this.txt
      10      20      30 that.c
     100    1000   10000 whatever.py

But you could also extract an individual stream from that invocation using special redirection operators:

gdsh$ wc-s * stdout>/dev/null bytes>&stdout
3
20
1000

We could also have multiple input channels. I imagine an fmt command which can interpolate named streams, e.g.

gdsh$ printf "1\n2 3\n" | wc-s | fmt "bytes: {bytes}\n words: {words}\n nl: {newlines}\n"
bytes: 6
words: 3
newlines: 2

We can then have a handful of other utilities and built-in shell operators for manipulating these other streams:

gdsh$ wc-s * | select words
3
20
1000
gdsh$ fmt "this is {X}\n" X<this.txt
1
2 3
gdsh$ !X='cat this.txt' fmt "this is {X}\n"
1
2 3
gdsh$ @X=this.txt fmt "this is {X}\n"
1
2 3
gdsh$ ^Y=recidivism fmt "Y is {Y}\n"
recidivism
gdsh$ wc-s * words>words.txt bytes>bytes.txt newlines>newlines.txt
gdsh$ wc -s * | split words=[sort >sorted-word-count.txt] bytes=[uniq >uniq-bytes.txt]

This could also enable new idioms for programming, e.g. verbose output, rather than being controlled by a flag, could be output consistently on another (usually hidden) stream:

gdsh$ myprog
myprog: config file not found
gdsh$ myprog stderr>/dev/null verbose>stderr
Setting up context
Looking in user dir... NOT FOUND
Looking in global dir... NOT FOUND
myprog: file not found
Tearing down context
Completed

Or maybe you could have human-readable error messages on stderr and machine-readable error messages on jsonerr:

gdsh$ thatprog
ERROR: no filename given
gdsh$ thatprog stderr>/dev/null jsonerr>stderr
{"error-type":"fatal","error-code":30,"error-msg":"no filename given"}

There are other considerations I've glossed over here, but here are a few notes, advantages, and interactions:

• I glossed over the distinction between input/output streams. In practice, the shell has no trouble disambiguating the two, but a given program may wish to consider the distinction between words.in and words.out; to this end, we could rename the existing streams std.in and std.out and err.out (it being an error to read from err.in in most cases.²)

• This is obviously not POSIX-compliant, but could be made to work with the existing UNIX process model by e.g. having a standard environment variable for stream-aware programs to look at which maps stream names to file descriptors. That way, programs which don't expect these special streams still use fds 0, 1, and 2 as expected, while programs that do handle these can read STREAM_DESC to find out which 'streams' correspond to which file descriptors. In that case, you can almost use these commands with an existing shell by doing something like

  sh$ echo foo >&77 | STREAM_DESC='std.in:0 std.out:1 err.out:2 foo.in:77' fmt "foo is {foo}\n"
  

• If we do use an existing UNIX system to write this, then we also should integrate libraries for this, and the API for it is unknown. Presumably a C interface would have int get_stream(char* stream_name) that could return -1 on failure or something like it, but maybe another interface would be preferable.

• This would interact really interestingly with a semi-graphical shell¹ that could visualize the stream relationships between commands as well as a shell with a higher-level understanding of data types.³

serializing/deserializing that a system is doing and just pass in some raw in-memory structure. You also get some nice operations that subsume the role of sed and awk and its ilk, e.g.

    lsh$ (map (\ ((fname nl wd cs)) (fmt #f "%s has %d bytes." fname cs))
              (wc "this.txt" "that.c"))
    ("this.txt has 6 bytes" "that.c has 100 bytes")

You could then use piping and shells to do some _pretty_ elaborate stuff,

including transferring higher-level objects (like HTTP requests or opaque OS-level objects like users or processes) in a universally understood way.

    jsh$ ps | map (fun (x) { return [x.cmd, x.args, x.pid, x.user]; })
    [ ["jsh",   [],          4440,  "gdritter"]
    , ["emacs", ["post.md"], 12893, "gdritter"]
    , ["ps",    [],          29678, "gdritter"]
    ]

So those are some ideas that have been drifting around in my head for a while. No idea if I'll ever implement any of them, or if they'd even be worth implementing, but I might get around to it at some point. We'll see.