Pico Pipes

Table of Contents

• Pipe
  • Pipe Documentation
    • General help
    • Subs
    • Pubs
    • Pipes
• Pipe-web
  • Pipe-web Documentation
    • Subscribe to a topic
      • Parameters
      • Query Parameters
      • Responses
      • Example cURL
    • Publish to a topic
      • Parameters
      • Query Parameters
      • Responses
      • Example cURL
    • Subscribe to a topic
      • Parameters
      • Query Parameters
      • Responses
      • Example cURL
    • Publish to a topic
      • Parameters
      • Query Parameters
      • Responses
      • Example cURL

Pipe #

Pipe is a simple and secure way of putting together composable directional streams of data, just like a *nix | operator!

For example, maybe you have a *nix pipe you're using on the command line like so:

1tail -f -n 0 /tmp/foo.log | grep --line-buffered "ERROR" | xargs -I{} -L1 osascript -e 'display notification "{}" with title "Pipe Notification"'

This simple one liner will grab new messages from a log file, check if it contains ERROR and then send a notification for macOS using AppleScript. This works if you run your app locally, but what if you run the app on a remote server? You can use pipe to connect remote and local without ever leaving the command line!

On your remote side, you would start a tail and pub it to a topic:

1tail -f -n 0 /tmp/foo.log | ssh pipe.pico.sh pub foo.log

On your local side, you would sub it to the notify command:

1ssh pipe.pico.sh sub foo.log | grep --line-buffered "ERROR" | xargs -I{} -L1 osascript -e 'display notification "{}" with title "Pipe Notification"'

The beauty of this method is that the command will wait until the sub is started before any data is consumed, ensuring you never miss a log line. If you didn't want it to wait for a sub, you can add -b=false (b for blocking) to the pub to prevent it from blocking.

Once you stop the pub command, the sub will also exit. You can also prevent this by adding -k (k for keepalive) to the sub command. With both blocking disabled and keepalive enabled, the commands would look like so:

Remote:

1tail -f -n 0 /tmp/foo.log | ssh pipe.pico.sh pub -b=false foo.log

Local:

1ssh pipe.pico.sh sub -k foo.log | grep --line-buffered "ERROR" | xargs -I{} -L1 osascript -e 'display notification "{}" with title "Pipe Notification"'

We can combine this with any commands we want and create a pretty robust pub/sub system. We can even send full command output through a pipe:

1ssh pipe.pico.sh sub htop

1htop | ssh pipe.pico.sh pub htop

Now what if you wanted to have bi-directional IO? That's where our last command of Pipe comes in, pipe! Pipe allow you to open a bi-directional client on any topic. It's fully non-blocking and can allow you to do things like have fully interactive chat over pipe. For example, running the following in two terminals:

1ssh pipe.pico.sh pipe chat

Will allow you to type and read messages as if you were sitting at the same terminal!

Pipe Documentation #

General help #

 1~$ ssh pipe.pico.sh help
 2Command: ssh pipe.pico.sh <help | ls | pub | sub | pipe> <topic> [-h | args...]
 3
 4The simplest authenticated pubsub system.  Send messages through
 5user-defined topics.  Topics are private to the authenticated
 6ssh user.  The default pubsub model is multicast with bidirectional
 7blocking, meaning a publisher ("pub") will send its message to all
 8subscribers ("sub").  Further, both "pub" and "sub" will wait for
 9at least one event to be sent or received. Pipe ("pipe") allows
10for bidirectional messages to be sent between any clients connected
11to a pipe.
12
13Think of these different commands in terms of the direction the
14data is being sent:
15
16- pub => writes to client
17- sub => reads from client
18- pipe => read and write between clients

Subs #

1~$ ssh pipe.pico.sh sub -h
2Usage: sub <topic> [args...]
3Args:
4  -a string
5        Comma separated list of pico usernames or ssh-key fingerprints to allow access to a topic
6  -c    Don't send status messages
7  -k    Keep the subscription alive even after the publisher has died
8  -p    Subscribe to a public topic

Pubs #

 1~$ ssh pipe.pico.sh pub -h
 2Usage: pub <topic> [args...]
 3Args:
 4  -a string
 5        Comma separated list of pico usernames or ssh-key fingerprints to allow access to a topic
 6  -b    Block writes until a subscriber is available (default true)
 7  -c    Don't send status messages
 8  -e    Send an empty message to subs
 9  -p    Publish message to public topic
10  -t duration
11        Timeout as a Go duration to block for a subscriber to be available. Valid time units are 'ns', 'us' (or 'µs'), 'ms', 's', 'm', 'h'. Default is 30 days. (default 720h0m0s)

Pipes #

1~$ ssh pipe.pico.sh pipe -h
2Usage: pipe <topic> [args...]
3Args:
4  -a string
5        Comma separated list of pico usernames or ssh-key fingerprints to allow access to a topic
6  -c    Don't send status messages
7  -p    Pipe to a public topic
8  -r    Replay messages to the client that sent it

Pipe-web #

Now what if you don't have a terminal available? Not a problem! Pipe has a web component that works side by side. For example, let's start a notification sub through the terminal like so (not p for public, so anyone can send us a notification):

1ssh pipe.pico.sh sub -p -k notifications | xargs -I{} -L1 osascript -e 'display notification "{}" with title "Pipe Notification"'

We can send a POST to the Pipe-web service to send a message onto that topic like so:

1echo "Hello world" | curl -X POST https://pipe.pico.sh/topic/notifications --data-binary @-

And a notification will pop up! Now it's important to note that this is risky, anyone can use a "public" topic (on both the terminal or Pipe-web). We can make this less risky by starting the notifications topic with an access list set. And if we want Pipe-web to access it, we need to provide "pico" to the access list setting:

1ssh pipe.pico.sh sub -a pico -p -k notifications | xargs -I{} -L1 osascript -e 'display notification "{}" with title "Pipe Notification"'

Now, only Pipe-web and yourself are able to access this public topic.

Pipe-web comes with a few caveats, namely all topics need to be public for it to work. You can set an access list on the topic, but Pipe-web is an unauthenticated service. Therefore, anyone can send a post request to the process.

Pipe-web Documentation #

Subscribe to a topic #

Publish to a topic #

Subscribe to a topic #

Publish to a topic #