ALPHABET-GRID(1) ALPHABET-GRID(1)
NAME
grid - peer-to-peer data distribution
SYNOPSIS
load alphabet
typeset /grid
type /grid/endpoint
DESCRIPTION
Grid is a typeset for alphabet (see sh-alphabet(1)) which
enables allows direct interconnection of remote and local
processing components. It defines one new type, endpoint,
which represents a place in the network to which two parties
can connect and exchange data.
In the following descriptions, if a type is not endpoint or
a type defined in the root typeset (see alphabet-main(1)),
it is assumed to be of type /string.
Modules currently provided within the /grid typeset include:
farm [-lnkavA] endpoint addr tasktype [arg...] -> endpoint
Farm connects to a grid labour exchange (see
scheduler(4)) at addr, starts a new job of type
workflow, and passes all the data read from
endpoint to be processed by the currently avail-
able labour. The data is split into records, each
one of which will be processed on a worker node by
tasktype, with its associated arguments. Other
than -A, which specifies unauthenticated access to
the scheduler, the various options are all passed
verbatim to workflow: -l causes it to split its
input on newline-separated records; -n specifies
that no record separation is necessary on output;
-k specifies that intermediate data for failed
tasks should be kept around; -a specifies that
intermediate data for all tasks should be kept
around, and -v specifies that workflow should pro-
duce a wordy description of what it is doing.
local endpoint -> /fd
Local reads everything from endpoint, and writes
it to fd.
remote [-a addr] fd -> endpoint
Remote is the inverse of local: it reads data from
fd and writes it to a newly created endpoint. If
-a is given, addr specifies the network address of
an endpoint server on which to create the new end-
point.
Page 1 Plan 9 (printed 12/21/24)
ALPHABET-GRID(1) ALPHABET-GRID(1)
rexec [-A] endpoint addr cmd -> endpoint
Rexec connects to a remote execution server at
addr (unauthenticated if -A is specified), and
arranges to execute the alphabet expression cmd
there. The expression should be compatible with
usage fd -> fd. Data from the argument endpoint
will be piped through this expression, and made
available as the resulting endpoint endpoint.
EXAMPLES
The examples below that a local endpoint is available, and
the following alphabet declarations:
load alphabet
typeset /grid
type /string /endpoint /fd
import /grid/local /grid/remote
autoconvert fd endpoint remote
Set up a rendering pipeline:
-{/read /tmp/somedata |
remote |
rexec tcp!node1!rexec "{(/fd); /filter $1 "{os render_stage1}} |
rexec tcp!node2!rexec "{(/fd); /filter $1 "{os render_stage2}} |
/create /tmp/somedata.result
}
SOURCE
/appl/alphabet/grid.b, /appl/alphabet/gridtypes.b
/appl/cmd/grid/*.b
SEE ALSO
sh-alphabet(1), alphabet-main(1), sh(1)
Page 2 Plan 9 (printed 12/21/24)