AOE(3) AOE(3)
NAME
aoe - ATA-over-Ethernet (AoE) interface
SYNOPSIS
bind -a #æ /dev
/dev/aoe/ctl
/dev/aoe/log
/dev/aoe/shelf.slot/config
/dev/aoe/shelf.slot/ctl
/dev/aoe/shelf.slot/devlink/0
...
/dev/aoe/shelf.slot/devlink/i
/dev/aoe/shelf.slot/ident
...
DESCRIPTION
The AoE (ATA-over-Ethernet) interface serves a three-level
directory providing control and access to AoE targets. The
interface provided is primarily intended for low-level con-
trol of the AoE initiator. See sdaoe(3) for the standard
interface.
Top-level files
In order to access AoE targets, one or more Ethernet con-
trollers need to be bound to the AoE initiator. By default,
the system starts with no interfaces bound. For automatic
binding of interfaces on boot, the aoeif configuration vari-
able is set in plan9.ini(8). Ethernet interfaces are speci-
fied as ethern, not as #ln. To bind the first and second
Ethernet devices on boot, add
aoeif=ether0 ether1
To bind ether1 to a running system:
% echo bind '#l1/ether1' >/dev/aoe/ctl
And to unbind it
% echo unbind '#l1/ether1' >/dev/aoe/ctl
When an interface is unbound, targets depending on that
interface are removed.
Each local interface is called a netlink. The mapping of AoE
targets to netlinks is called a devlink. Each devlink may
see multiple interfaces per target. For example, if the
local machine has one Ethernet address bound and the target
has two interfaces on the same Ethernet segment, this will
result in one netlink and one devlink with two Ethernet
Page 1 Plan 9 (printed 12/21/24)
AOE(3) AOE(3)
addresses. AoE frames are sent in round-robin fashion.
Each successive frame is sent on the next address available
on the next available devlink (local interface).
Normally the initiator automatically discovers and adds new
device directories on startup. New devices are not added
except as new interfaces are bound to the initiator. Sev-
eral messages can be written to /dev/aoe/ctl which alter
this behavior:
autodiscover toggle
If toggle is absent, the state of autodiscover is tog-
gled. If it is the string on, it is turned on. Any
other string turns autodisover off. This option is not
useful after Ethernet devices have been bound.
discover shelf.slot
Attempt to find the named target on all bound inter-
faces.
remove shelf.slot
The converse of discover: remove the named target if it
exists.
rediscover toggle
Allow or disallow rediscovery. This allows for auto-
matic discovery of new targets. Unfortunately, it also
allows automatic modification or loss of existing tar-
gets. This option is considered dangerous.
Reading /dev/aoe/ctl returns a list of colon-separated lines
with keywords and their values:
debug
autodiscover
rediscover Returns the current state of the variable named
by the keyword. Writing the variable's name to
the control file toggles the state of that vari-
able.
ifn path Path to nth bound Ethernet device.
ifn ea Ethernet address of this device.
ifn flag A flag of ``Up'' indicates that this interface
is available.
ifn lostjumbo
Number of consecutive lost jumbograms.
ifn datamtu Incorrect and unused.
Page 2 Plan 9 (printed 12/21/24)
AOE(3) AOE(3)
Target subdirectories
Once configured, each AoE target is accessed via files in
the directory named for its shelf and slot. For example,
shelf 42, slot 0 would be accessed through the path
`/dev/aoe/42.0'. The ident file contains the read-only,
verbatim result of the identify unit ATA command. The
config file contains the target's AoE configuration string.
Writing to this file sets the targets configuration string.
Reading a target's ctl file returns a list of colon-
separated lines with the following keywords and values:
state ``Up'' or ``down''.
nopen Number of clients using this target.
nout Number of outstanding AoE frames.
nmaxout Maximum number of outstanding frames allowed.
nframes Maximum number of outstanding frames. Nframes is
greater than nmaxout when the initiator is reduc-
ing the number of in-flight frames due to packet
loss. It is assumed that packet loss is due to an
overwhelmed target and not poor network condi-
tions.
maxbcount Maximum number of data bytes per AoE frame. Using
standard frames, maxbcount is 1024 or two sectors.
AoE ATA headers are 36 bytes.
model
serial
firmware The respective fields from the ATA identify unit
command.
flag List of flags useful for debugging. The flag
jumbo indicates that jumbo frames are accepted,
not that they are being used. Maxbcount should be
consulted for this purpose.
Writing to the ctl file, the following commands may be
issued:
failio fail outstanding i/o.
identify send an ata `identify device' command to the tar-
get.
maxbno n set the maximum number of block sent per packet.
Page 3 Plan 9 (printed 12/21/24)
AOE(3) AOE(3)
mtu n set the maximum number of bytes (including header)
sent per packet.
nofail never fail this target. This is useful if your
root device is on this target.
setsize n with no arguments, reset the device size to the
size claimed. Otherwise, assume the device is the
given size.
The data file may be read or written like a normal file
except that reads and writes to this file are converted to
AoE commands to the target, so transfers should be 512 or
1024 bytes long (or a larger multiple of 512 iff jumbo pack-
ets are in use). The size of this file is the usable size
of the target.
The devlink directory contains one file for each interface
the target was discovered on. The files are numbers from 0
to n and contain a list of colon-separated lines with key-
words and their values:
addr A space-separated list of the target's Ethernet
addresses visible from this interface.
npkt The number of frames sent on this interface.
resent The number of frames re-sent. Frames are re-sent
when they have been outstanding twice the RTT
average.
flag ``Up'' when the netlink is up.
rttavg
mintimer Minimum timer and RTT average as per Congestion
Avoidance and Control.
nl path Path of the Ethernet device.
nl ea Ethernet address of the local Ethernet device.
nl flag ``Up'' if the local interface is up.
nl lostjumbo
Number of consecutive jumbograms lost.
nl datamtu
Unused.
SOURCE
/sys/src/9/port/devaoe.c
Page 4 Plan 9 (printed 12/21/24)
AOE(3) AOE(3)
SEE ALSO
sd(3), sdaoe(3), aoesrv(8), snoopy(8)
http://support.coraid.com/documents/AoEr11.txt
Van Jacobson and Michael J. Karels, ``Congestion Avoidance
and Control'', ACM Computer Communication Review; Proceed-
ings of the Sigcomm '88 Symposium in Stanford, CA, August,
1988.
BUGS
There is no raw file for executing arbitrary commands.
This is a fairly primitive interface; sdaoe(3) is usually
more suitable.
Page 5 Plan 9 (printed 12/21/24)