18 August 2013


by mo

ri IO.pipe tells me:

Creates a pair of pipe endpoints (connected to each other) and returns them as a two-element array of IO objects: [ read_io, write_io ].

Essentially what this does is create two IO objects. One that you can write to, and one that you can read from. Picture a pipe where water can only flow one way. At one end of the pipe you can pour water in (writer) and at the other end of the pipe the water pours out (reader).

Here’s an example:

IO.pipe do |reader, writer|
  p reader.readpartial(4)

In the above example IO.pipe creates a reader and writer IO objects that we can read and write to/from. At one end of the pipe we can write ‘blah’ too. At the other end we can read from the pipe. (Note the use of readpartial instead of read. read will block until an EOF is written to the pipe. The only way I know to do that is to just close the IO object.)

An alternate way to write the above code is:

reader, writer = IO.pipe


The following will raise an error, because the reader IO object cannot be written to.

  IO.pipe do |reader, writer|
    reader.write('blah') # `write': not opened for writing (IOError)

The following will also raise an error because the writer IO object anno be read from.

  IO.pipe do |reader, writer|
    writer.read() # `read': not opened for reading (IOError)

Pipes become incredibly useful in the context of forking processes. When you need to pipe input from out process into another. Let’s look at another example:

reader, writer = IO.pipe
writer.write('HELLO') # fill pipe with data from the parent process

pid = fork do
  data_from_parent = reader.readpartial(512)
  writer.close # close writer end of pipe and send EOF

p reader.readpartial(512) # read contents from child process

Because a child process gets a copy of all open file descriptors from the parent process at the moment that it was forked from the parent, it now has access to the reader and writer ends of the pipe. This allows it to read data from the parent process or write data back up to the parent process.

Pipes are one way. At one end of the pipe you can push data in and at the other end of the pipe you can pull data out.

When you execute a command like the following from your shell. What is happening under the hood is your shell is forking itself (this includes all environment variables and open file descriptors. Then exec the first command and piping output from one forked process into the other.

  $ ls /var/log | grep log

In this case a process is forked for ls and for grep. The grep process will wait to read from a reader end of the pipe. The ls process will write to a writer end of the pipe and finally the grep process will write back out to stdout (standard out).

For more information check out Jesse Storimers blog series on building a unix shell.

named pipes

You can also create named pipes. An easy way to do this is from your shell.

λ mkfifo my.pipe
λ ls -al my.pipe 
prw-rw-r-- 1 mo 0 Aug 18 15:32 my.pipe|

The mkfifo command is used to a create a new file called ‘my.pipe’. The main difference between this file and any other file is that in the file attributes it is marked with a ‘p’ instead of a ‘-‘.

Now if you open up two separate shells. You can write to the pipe in one shell and read from the pipe in another.

λ echo 'hello' > my.pipe
λ cat my.pipe 

You might find that your shell blocks when you write to the pipe then unblocks when you read from the pipe.

The advantage of the named pipe is that it is persistent and exists beyond the life of a process. As your process starts up you can connect to the named pipe.


unix 💎