1.\" Copyright (c) 1996-1999 Whistle Communications, Inc.
2.\" All rights reserved.
3.\"
4.\" Subject to the following obligations and disclaimer of warranty, use and
5.\" redistribution of this software, in source or object code forms, with or
6.\" without modifications are expressly permitted by Whistle Communications;
7.\" provided, however, that:
8.\" 1. Any and all reproductions of the source or object code must include the
9.\" copyright notice above and the following disclaimer of warranties; and
10.\" 2. No rights are granted, in any manner or form, to use Whistle
11.\" Communications, Inc. trademarks, including the mark "WHISTLE
12.\" COMMUNICATIONS" on advertising, endorsements, or otherwise except as
13.\" such appears in the above copyright notice or in the software.
14.\"
15.\" THIS SOFTWARE IS BEING PROVIDED BY WHISTLE COMMUNICATIONS "AS IS", AND
16.\" TO THE MAXIMUM EXTENT PERMITTED BY LAW, WHISTLE COMMUNICATIONS MAKES NO
17.\" REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, REGARDING THIS SOFTWARE,
18.\" INCLUDING WITHOUT LIMITATION, ANY AND ALL IMPLIED WARRANTIES OF
19.\" MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.
20.\" WHISTLE COMMUNICATIONS DOES NOT WARRANT, GUARANTEE, OR MAKE ANY
21.\" REPRESENTATIONS REGARDING THE USE OF, OR THE RESULTS OF THE USE OF THIS
22.\" SOFTWARE IN TERMS OF ITS CORRECTNESS, ACCURACY, RELIABILITY OR OTHERWISE.
23.\" IN NO EVENT SHALL WHISTLE COMMUNICATIONS BE LIABLE FOR ANY DAMAGES
24.\" RESULTING FROM OR ARISING OUT OF ANY USE OF THIS SOFTWARE, INCLUDING
25.\" WITHOUT LIMITATION, ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
26.\" PUNITIVE, OR CONSEQUENTIAL DAMAGES, PROCUREMENT OF SUBSTITUTE GOODS OR
27.\" SERVICES, LOSS OF USE, DATA OR PROFITS, HOWEVER CAUSED AND UNDER ANY
28.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
30.\" THIS SOFTWARE, EVEN IF WHISTLE COMMUNICATIONS IS ADVISED OF THE POSSIBILITY
31.\" OF SUCH DAMAGE.
32.\"
33.\" Authors: Julian Elischer <julian@FreeBSD.org>
34.\" Archie Cobbs <archie@FreeBSD.org>
35.\"
36.\" $FreeBSD: head/share/man/man4/netgraph.4 69922 2000-12-12 18:52:14Z julian $
37.\" $Whistle: netgraph.4,v 1.7 1999/01/28 23:54:52 julian Exp $
38.\"
39.Dd January 19, 1999
40.Dt NETGRAPH 4
41.Os FreeBSD
42.Sh NAME
43.Nm netgraph
44.Nd graph based kernel networking subsystem
45.Sh DESCRIPTION
46The
47.Nm
48system provides a uniform and modular system for the implementation
49of kernel objects which perform various networking functions. The objects,
50known as
51.Em nodes ,
52can be arranged into arbitrarily complicated graphs. Nodes have
53.Em hooks
54which are used to connect two nodes together, forming the edges in the graph.
55Nodes communicate along the edges to process data, implement protocols, etc.
56.Pp
57The aim of
58.Nm
59is to supplement rather than replace the existing kernel networking
60infrastructure. It provides:
61.Pp
62.Bl -bullet -compact -offset 2n
63.It
64A flexible way of combining protocol and link level drivers
65.It
66A modular way to implement new protocols
67.It
68A common framework for kernel entities to inter-communicate
69.It
70A reasonably fast, kernel-based implementation
71.El
72.Sh Nodes and Types
73The most fundamental concept in
74.Nm
75is that of a
76.Em node .
77All nodes implement a number of predefined methods which allow them
78to interact with other nodes in a well defined manner.
79.Pp
80Each node has a
81.Em type ,
82which is a static property of the node determined at node creation time.
83A node's type is described by a unique
84.Tn ASCII
85type name.
86The type implies what the node does and how it may be connected
87to other nodes.
88.Pp
89In object-oriented language, types are classes and nodes are instances
90of their respective class. All node types are subclasses of the generic node
91type, and hence inherit certain common functionality and capabilities
92(e.g., the ability to have an
93.Tn ASCII
94name).
95.Pp
96Nodes may be assigned a globally unique
97.Tn ASCII
98name which can be
99used to refer to the node.
100The name must not contain the characters
101.Dq \&.
102or
103.Dq \&:
104and is limited to
105.Dv "NG_NODELEN + 1"
106characters (including NUL byte).
107.Pp
108Each node instance has a unique
109.Em ID number
110which is expressed as a 32-bit hex value. This value may be used to
111refer to a node when there is no
112.Tn ASCII
113name assigned to it.
114.Sh Hooks
115Nodes are connected to other nodes by connecting a pair of
116.Em hooks ,
117one from each node. Data flows bidirectionally between nodes along
118connected pairs of hooks. A node may have as many hooks as it
119needs, and may assign whatever meaning it wants to a hook.
120.Pp
121Hooks have these properties:
122.Pp
123.Bl -bullet -compact -offset 2n
124.It
125A hook has an
126.Tn ASCII
127name which is unique among all hooks
128on that node (other hooks on other nodes may have the same name).
129The name must not contain a
130.Dq \&.
131or a
132.Dq \&:
133and is
134limited to
135.Dv "NG_HOOKLEN + 1"
136characters (including NUL byte).
137.It
138A hook is always connected to another hook. That is, hooks are
139created at the time they are connected, and breaking an edge by
140removing either hook destroys both hooks.
141.It
142A hook can be set into a state where incoming packets are always queued
143by the input queuing system, rather than being delivered directly. This
144is used when the two joined nodes need to be decoupled, e.g. if they are
145running at different processor priority levels. (spl)
146.El
147.Pp
148A node may decide to assign special meaning to some hooks.
149For example, connecting to the hook named
150.Dq debug
151might trigger
152the node to start sending debugging information to that hook.
153.Sh Data Flow
154Two types of information flow between nodes: data messages and
155control messages. Data messages are passed in mbuf chains along the edges
156in the graph, one edge at a time. The first mbuf in a chain must have the
157.Dv M_PKTHDR
158flag set. Each node decides how to handle data coming in on its hooks.
159.Pp
160Control messages are type-specific C structures sent from one node
161directly to some arbitrary other node. Control messages have a common
162header format, followed by type-specific data, and are binary structures
163for efficiency. However, node types also may support conversion of the
164type specific data between binary and
165.Tn ASCII
166for debugging and human interface purposes (see the
167.Dv NGM_ASCII2BINARY
168and
169.Dv NGM_BINARY2ASCII
170generic control messages below). Nodes are not required to support
171these conversions.
172.Pp
173There are three ways to address a control message. If
174there is a sequence of edges connecting the two nodes, the message
175may be
176.Dq source routed
177by specifying the corresponding sequence
178of
179.Tn ASCII
180hook names as the destination address for the message (relative
181addressing). If the destination is adjacent to the source, then the source
182node may simply specify (as a pointer in the code) the hook across which the
183message should be sent. Otherwise, the recipient node global
184.Tn ASCII
185name
186(or equivalent ID based name) is used as the destination address
187for the message (absolute addressing). The two types of
188.Tn ASCII
189addressing
190may be combined, by specifying an absolute start node and a sequence
191of hooks. Only the
192.Tn ASCII
193addressing modes are available to control programs outside the kernel,
194as use of direct pointers is limited of course to kernel modules.
195.Pp
196Messages often represent commands that are followed by a reply message
197in the reverse direction. To facilitate this, the recipient of a
198control message is supplied with a
199.Dq return address
200that is suitable for addressing a reply.
201In addition, depending on the topology of
202the graph and whether the source has requested it, a pointer to a
203pointer that can be read by the source node may also be supplied.
204This allows the destination node to directly respond in a
205synchronous manner when control returns to the source node, by
206simply pointing the supplied pointer to the response message.
207Such synchronous message responses are more efficient but are not always possible.
208.Pp
209Each control message contains a 32 bit value called a
210.Em typecookie
211indicating the type of the message, i.e., how to interpret it.
212Typically each type defines a unique typecookie for the messages
213that it understands. However, a node may choose to recognize and
214implement more than one type of message.
215.Pp
216If a message is delivered to an address that implies that it arrived
217at that node through a particular hook, (as opposed to having been directly
218addressed using its ID or global name), then that hook is identified to the
219receiving node. This allows a message to be rerouted or passed on, should
220a node decide that this is required, in much the same way that data packets
221are passed around between nodes. A set of standard
222messages for flow control and link management purposes are
223defined by the base system that are usually
224passed around in this manner. Flow control message would usually travel
225in the opposite direction to the data to which they pertain.
226.Pp
227Since flow control packets can also result from data being sent, it is also
228possible to return a synchronous message response to a data packet being
229sent between nodes. (See later).
230.Sh Netgraph is Functional
231In order to minimize latency, most
232.Nm
233operations are functional.
234That is, data and control messages are delivered by making function
235calls rather than by using queues and mailboxes. For example, if node
236A wishes to send a data mbuf to neighboring node B, it calls the
237generic
238.Nm
239data delivery function. This function in turn locates
240node B and calls B's
241.Dq receive data
242method. There are exceptions to this.
243.Pp
244It is allowable for nodes to reject a data packet, or to pass it back to the
245caller in a modified or completely replaced form. The caller can notify the
246node being called that it does not wish to receive any such packets
247by using the
248.Fn NG_SEND_DATA
249and
250.Fn NG_SEND_DATA_ONLY
251macros, in which case, the second node should just discard rejected packets.
252If the sender knows how to handle returned packets, it must use the
253.Fn NG_SEND_DATA_RET
254macro, which will adjust the parameters to point to the returned data
255or NULL if no data was returned to the caller. No packet return is possible
256across a queuing link (though an explicitly sent return is of course possible,
257it doesn't mean quite the same thing).
258.Pp
259While this mode of operation
260results in good performance, it has a few implications for node
261developers:
262.Pp
263.Bl -bullet -compact -offset 2n
264.It
265Whenever a node delivers a data or control message, the node
266may need to allow for the possibility of receiving a returning
267message before the original delivery function call returns.
268.It
269Netgraph nodes and support routines generally run at
270.Fn splnet .
271However, some nodes may want to send data and control messages
272from a different priority level. Netgraph supplies a mechanism which
273utilizes the NETISR system to move message and data delivery to
274.Fn splnet .
275Nodes that run at other priorities (e.g. interfaces) can be directly
276linked to other nodes so that the combination runs at the other priority,
277however any interaction with nodes running at splnet MUST be achieved via the
278queueing functions, (which use the
279.Fn netisr
280feature of the kernel).
281Note that messages are always received at
282.Fn splnet .
283.It
284It's possible for an infinite loop to occur if the graph contains cycles.
285.El
286.Pp
287So far, these issues have not proven problematical in practice.
288.Sh Interaction With Other Parts of the Kernel
289A node may have a hidden interaction with other components of the
290kernel outside of the
291.Nm
292subsystem, such as device hardware,
293kernel protocol stacks, etc. In fact, one of the benefits of
294.Nm
295is the ability to join disparate kernel networking entities together in a
296consistent communication framework.
297.Pp
298An example is the node type
299.Em socket
300which is both a netgraph node and a
301.Xr socket 2
302BSD socket in the protocol family
303.Dv PF_NETGRAPH .
304Socket nodes allow user processes to participate in
305.Nm .
306Other nodes communicate with socket nodes using the usual methods, and the
307node hides the fact that it is also passing information to and from a
308cooperating user process.
309.Pp
310Another example is a device driver that presents
311a node interface to the hardware.
312.Sh Node Methods
313Nodes are notified of the following actions via function calls
314to the following node methods (all at
315.Fn splnet )
316and may accept or reject that action (by returning the appropriate
317error code):
318.Bl -tag -width xxx
319.It Creation of a new node
320The constructor for the type is called. If creation of a new node is
321allowed, the constructor must call the generic node creation
322function (in object-oriented terms, the superclass constructor)
323and then allocate any special resources it needs. For nodes that
324correspond to hardware, this is typically done during the device
325attach routine. Often a global
326.Tn ASCII
327name corresponding to the
328device name is assigned here as well.
329.It Creation of a new hook
330The hook is created and tentatively
331linked to the node, and the node is told about the name that will be
332used to describe this hook. The node sets up any special data structures
333it needs, or may reject the connection, based on the name of the hook.
334.It Successful connection of two hooks
335After both ends have accepted their
336hooks, and the links have been made, the nodes get a chance to
337find out who their peer is across the link and can then decide to reject
338the connection. Tear-down is automatic. This is also the time at which
339a node may decide whether to set a particular hook (or its peer) into
340.Em queuing
341mode.
342.It Destruction of a hook
343The node is notified of a broken connection. The node may consider some hooks
344to be critical to operation and others to be expendable: the disconnection
345of one hook may be an acceptable event while for another it
346may effect a total shutdown for the node.
347.It Shutdown of a node
348This method allows a node to clean up
349and to ensure that any actions that need to be performed
350at this time are taken. The method must call the generic (i.e., superclass)
351node destructor to get rid of the generic components of the node.
352Some nodes (usually associated with a piece of hardware) may be
353.Em persistent
354in that a shutdown breaks all edges and resets the node,
355but doesn't remove it, in which case the generic destructor is not called.
356.El
357.Sh Sending and Receiving Data
358Two other methods are also supported by all nodes:
359.Bl -tag -width xxx
360.It Receive data message
361An mbuf chain is passed to the node.
362The node is notified on which hook the data arrived,
363and can use this information in its processing decision.
364The receiving node must always
365.Fn m_freem
366the mbuf chain on completion or error, pass it back (reject it), or pass
367it on to another node
368(or kernel module) which will then be responsible for freeing it.
369If a node passes a packet back to the caller, it does not have to be the
370same mbuf, in which case the original must be freed. Passing a packet
371back allows a module to modify the original data (e.g. encrypt it),
372or in some other way filter it (e.g. packet filtering).
373.Pp
374In addition to the mbuf chain itself there is also a pointer to a
375structure describing meta-data about the message
376(e.g. priority information). This pointer may be
377.Dv NULL
378if there is no additional information. The format for this information is
379described in
380.Pa sys/netgraph/netgraph.h .
381The memory for meta-data must allocated via
382.Fn malloc
383with type
384.Dv M_NETGRAPH .
385As with the data itself, it is the receiver's responsibility to
386.Fn free
387the meta-data. If the mbuf chain is freed the meta-data must
388be freed at the same time. If the meta-data is freed but the
389real data on is passed on, then a
390.Dv NULL
391pointer must be substituted.
392Meta-data may be passed back in the same way that mbuf data may be passed back.
393As with mbuf data, the rejected or returned meta-data pointer may point to
394the same or different meta-data as that passed in,
395and if it is different, the original must be freed.
396.Pp
397The receiving node may decide to defer the data by queueing it in the
398.Nm
399NETISR system (see below). It achieves this by setting the
400.Dv HK_QUEUE
401flag in the flags word of the hook on which that data will arrive.
402The infrastructure will respect that bit and queue the data for delivery at
403a later time, rather than deliver it directly. A node may decide to set
404the bit on the
405.Em peer
406node, so that it's own output packets are queued. This is used
407by device drivers running at different processor priorities to transfer
408packet delivery to the splnet() level at which the bulk of
409.Nm
410runs.
411.Pp
412The structure and use of meta-data is still experimental, but is
413presently used in frame-relay to indicate that management packets
414should be queued for transmission
415at a higher priority than data packets. This is required for
416conformance with Frame Relay standards.
417.Pp
418The node may also receive information allowing it to send a synchronous
419message response to one of the originators of the data. it is envisionned
420that such a message would contain error or flow-control information.
421Standard messages for these purposes have been defined in
422.Pa sys/netgraph/netgraph.h .
423.It Receive control message
424This method is called when a control message is addressed to the node.
425A return address is always supplied, giving the address of the node
426that originated the message so a reply message can be sent anytime later.
427.Pp
428It is possible for a synchronous reply to be made, and in fact this
429is more common in practice.
430This is done by setting a pointer (supplied as an extra function parameter)
431to point to the reply.
432Then when the control message delivery function returns,
433the caller can check if this pointer has been made non-NULL,
434and if so then it points to the reply message allocated via
435.Fn malloc
436and containing the synchronous response. In both directions,
437(request and response) it is up to the
438receiver of that message to
439.Fn free
440the control message buffer. All control messages and replies are
441allocated with
442.Fn malloc
443type
444.Dv M_NETGRAPH .
445.Pp
446If the message was delivered via a specific hook, that hook will
447also be made known, which allows the use of such things as flow-control
448messages, and status change messages, where the node may want to forward
449the message out another hook to that on which it arrived.
450.El
451.Pp
452Much use has been made of reference counts, so that nodes being
453free'd of all references are automatically freed, and this behaviour
454has been tested and debugged to present a consistent and trustworthy
455framework for the
456.Dq type module
457writer to use.
458.Sh Addressing
459The
460.Nm
461framework provides an unambiguous and simple to use method of specifically
462addressing any single node in the graph. The naming of a node is
463independent of its type, in that another node, or external component
464need not know anything about the node's type in order to address it so as
465to send it a generic message type. Node and hook names should be
466chosen so as to make addresses meaningful.
467.Pp
468Addresses are either absolute or relative. An absolute address begins
469with a node name, (or ID), followed by a colon, followed by a sequence of hook
470names separated by periods. This addresses the node reached by starting
471at the named node and following the specified sequence of hooks.
472A relative address includes only the sequence of hook names, implicitly
473starting hook traversal at the local node.
474.Pp
475There are a couple of special possibilities for the node name.
476The name
477.Dq \&.
478(referred to as
479.Dq \&.: )
480always refers to the local node.
481Also, nodes that have no global name may be addressed by their ID numbers,
482by enclosing the hex representation of the ID number within square brackets.
483Here are some examples of valid netgraph addresses:
484.Bd -literal -offset 4n -compact
485
486 .:
487 foo:
488 .:hook1
489 foo:hook1.hook2
490 [d80]:hook1
491.Ed
492.Pp
493Consider the following set of nodes might be created for a site with
494a single physical frame relay line having two active logical DLCI channels,
495with RFC-1490 frames on DLCI 16 and PPP frames over DLCI 20:
496.Pp
497.Bd -literal
498[type SYNC ] [type FRAME] [type RFC1490]
499[ "Frame1" ](uplink)<-->(data)[<un-named>](dlci16)<-->(mux)[<un-named> ]
500[ A ] [ B ](dlci20)<---+ [ C ]
501 |
502 | [ type PPP ]
503 +>(mux)[<un-named>]
504 [ D ]
505.Ed
506.Pp
507One could always send a control message to node C from anywhere
508by using the name
509.Em "Frame1:uplink.dlci16" .
510In this case, node C would also be notified that the message
511reached it via its hook
512.Dq mux .
513Similarly,
514.Em "Frame1:uplink.dlci20"
515could reliably be used to reach node D, and node A could refer
516to node B as
517.Em ".:uplink" ,
518or simply
519.Em "uplink" .
520Conversely, B can refer to A as
521.Em "data" .
522The address
523.Em "mux.data"
524could be used by both nodes C and D to address a message to node A.
525.Pp
526Note that this is only for
527.Em control messages .
528In each of these cases, where a relative addressing mode is
529used, the recipient is notified of the hook on which the
530message arrived, as well as
531the originating node.
532This allows the option of hop-by-hop distibution of messages and
533state information.
534Data messages are
535.Em only
536routed one hop at a time, by specifying the departing
537hook, with each node making
538the next routing decision. So when B receives a frame on hook
539.Dq data
540it decodes the frame relay header to determine the DLCI,
541and then forwards the unwrapped frame to either C or D.
542.Pp
543In a similar way, flow control messages may be routed in the reverse
544direction to outgoing data. For example a "buffer nearly full" message from
545.Em "Frame1:
546would be passed to node
547.Em B
548which might decide to send similar messages to both nodes
549.Em C
550and
551.Em D .
552The nodes would use
553.Em "Direct hook pointer"
554addressing to route the messages. The message may have travelled from
555.Em "Frame1:
556to
557.Em B
558as a synchronous reply, saving time and cycles.
559
560
561.Pp
562A similar graph might be used to represent multi-link PPP running
563over an ISDN line:
564.Pp
565.Bd -literal
566[ type BRI ](B1)<--->(link1)[ type MPP ]
567[ "ISDN1" ](B2)<--->(link2)[ (no name) ]
568[ ](D) <-+
569 |
570 +----------------+
571 |
572 +->(switch)[ type Q.921 ](term1)<---->(datalink)[ type Q.931 ]
573 [ (no name) ] [ (no name) ]
574.Ed
575.Sh Netgraph Structures
576Interesting members of the node and hook structures are shown below
577however you should
578check
579.Pa sys/netgraph/netgraph.h
580on your system for more up-to-date versions.
581
582.Bd -literal
583struct ng_node {
584 char *name; /* Optional globally unique name */
585 void *private; /* Node implementation private info */
586 struct ng_type *type; /* The type of this node */
587 int refs; /* Number of references to this struct */
588 int numhooks; /* Number of connected hooks */
589 hook_p hooks; /* Linked list of (connected) hooks */
590};
591typedef struct ng_node *node_p;
592
593struct ng_hook {
594 char *name; /* This node's name for this hook */
595 void *private; /* Node implementation private info */
596 int refs; /* Number of references to this struct */
597 struct ng_node *node; /* The node this hook is attached to */
598 struct ng_hook *peer; /* The other hook in this connected pair */
599 struct ng_hook *next; /* Next in list of hooks for this node */
600};
601typedef struct ng_hook *hook_p;
602.Ed
603.Pp
604The maintenance of the name pointers, reference counts, and linked list
605of hooks for each node is handled automatically by the
606.Nm
607subsystem.
608Typically a node's private info contains a back-pointer to the node or hook
609structure, which counts as a new reference that must be registered by
610incrementing
611.Dv "node->refs" .
612.Pp
613From a hook you can obtain the corresponding node, and from
614a node the list of all active hooks.
615.Pp
616Node types are described by the structures below:
617.Bd -literal
618/** How to convert a control message from binary <-> ASCII */
619struct ng_cmdlist {
620 u_int32_t cookie; /* typecookie */
621 int cmd; /* command number */
622 const char *name; /* command name */
623 const struct ng_parse_type *mesgType; /* args if !NGF_RESP */
624 const struct ng_parse_type *respType; /* args if NGF_RESP */
625};
626
627struct ng_type {
628 u_int32_t version; /* Must equal NG_VERSION */
629 const char *name; /* Unique type name */
630
631 /* Module event handler */
632 modeventhand_t mod_event; /* Handle load/unload (optional) */
633
634 /* Constructor */
635 int (*constructor)(node_p *node); /* Create a new node */
636
637 /** Methods using the node **/
638 int (*rcvmsg)(node_p node, /* Receive control message */
639 struct ng_mesg *msg, /* The message */
640 const char *retaddr, /* Return address */
641 struct ng_mesg **resp /* Synchronous response */
642 hook_p lasthook); /* last hook traversed */
643 int (*shutdown)(node_p node); /* Shutdown this node */
644 int (*newhook)(node_p node, /* create a new hook */
645 hook_p hook, /* Pre-allocated struct */
646 const char *name); /* Name for new hook */
647
648 /** Methods using the hook **/
649 int (*connect)(hook_p hook); /* Confirm new hook attachment */
650 int (*rcvdata)(hook_p hook, /* Receive data on a hook */
651 struct mbuf *m, /* The data in an mbuf */
652 meta_p meta, /* Meta-data, if any */
653 struct mbuf **ret_m, /* return data here */
654 meta_p *ret_meta, /* return Meta-data here */
655 struct ng_message **resp); /* Synchronous reply info */
656 int (*disconnect)(hook_p hook); /* Notify disconnection of hook */
657
658 /** How to convert control messages binary <-> ASCII */
659 const struct ng_cmdlist *cmdlist; /* Optional; may be NULL */
660};
661.Ed
662.Pp
663Control messages have the following structure:
664.Bd -literal
665#define NG_CMDSTRLEN 15 /* Max command string (16 with null) */
666
667struct ng_mesg {
668 struct ng_msghdr {
669 u_char version; /* Must equal NG_VERSION */
670 u_char spare; /* Pad to 2 bytes */
671 u_short arglen; /* Length of cmd/resp data */
672 u_long flags; /* Message status flags */
673 u_long token; /* Reply should have the same token */
674 u_long typecookie; /* Node type understanding this message */
675 u_long cmd; /* Command identifier */
676 u_char cmdstr[NG_CMDSTRLEN+1]; /* Cmd string (for debug) */
677 } header;
678 char data[0]; /* Start of cmd/resp data */
679};
680
681#define NG_VERSION 3 /* Netgraph version */
682#define NGF_ORIG 0x0000 /* Command */
683#define NGF_RESP 0x0001 /* Response */
684.Ed
685.Pp
686Control messages have the fixed header shown above, followed by a
687variable length data section which depends on the type cookie
688and the command. Each field is explained below:
689.Bl -tag -width xxx
690.It Dv version
691Indicates the version of netgraph itself. The current version is
692.Dv NG_VERSION .
693.It Dv arglen
694This is the length of any extra arguments, which begin at
695.Dv data .
696.It Dv flags
697Indicates whether this is a command or a response control message.
698.It Dv token
699The
700.Dv token
701is a means by which a sender can match a reply message to the
702corresponding command message; the reply always has the same token.
703.Pp
704.It Dv typecookie
705The corresponding node type's unique 32-bit value.
706If a node doesn't recognize the type cookie it must reject the message
707by returning
708.Er EINVAL .
709.Pp
710Each type should have an include file that defines the commands,
711argument format, and cookie for its own messages.
712The typecookie
713insures that the same header file was included by both sender and
714receiver; when an incompatible change in the header file is made,
715the typecookie
716.Em must
717be changed.
718The de facto method for generating unique type cookies is to take the
719seconds from the epoch at the time the header file is written
720(i.e., the output of
721.Dv "date -u +'%s'" ) .
722.Pp
723There is a predefined typecookie
724.Dv NGM_GENERIC_COOKIE
725for the
726.Dq generic
727node type, and
728a corresponding set of generic messages which all nodes understand.
729The handling of these messages is automatic.
730.It Dv command
731The identifier for the message command. This is type specific,
732and is defined in the same header file as the typecookie.
733.It Dv cmdstr
734Room for a short human readable version of
735.Dq command
736(for debugging purposes only).
737.El
738.Pp
739Some modules may choose to implement messages from more than one
740of the header files and thus recognize more than one type cookie.
741.Sh Control Message ASCII Form
742Control messages are in binary format for efficiency. However, for
743debugging and human interface purposes, and if the node type supports
744it, control messages may be converted to and from an equivalent
745.Tn ASCII
746form. The
747.Tn ASCII
748form is similar to the binary form, with two exceptions:
749.Pp
750.Bl -tag -compact -width xxx
751.It o
752The
753.Dv cmdstr
754header field must contain the
755.Tn ASCII
756name of the command, corresponding to the
757.Dv cmd
758header field.
759.It o
760The
761.Dv args
762field contains a NUL-terminated
763.Tn ASCII
764string version of the message arguments.
765.El
766.Pp
767In general, the arguments field of a control messgage can be any
768arbitrary C data type. Netgraph includes parsing routines to support
769some pre-defined datatypes in
770.Tn ASCII
771with this simple syntax:
772.Pp
773.Bl -tag -compact -width xxx
774.It o
775Integer types are represented by base 8, 10, or 16 numbers.
776.It o
777Strings are enclosed in double quotes and respect the normal
778C language backslash escapes.
779.It o
780IP addresses have the obvious form.
781.It o
782Arrays are enclosed in square brackets, with the elements listed
783consecutively starting at index zero. An element may have an optional
784index and equals sign preceeding it. Whenever an element
785does not have an explicit index, the index is implicitly the previous
786element's index plus one.
787.It o
788Structures are enclosed in curly braces, and each field is specified
789in the form
790.Dq fieldname=value .
791.It o
792Any array element or structure field whose value is equal to its
793.Dq default value
794may be omitted. For integer types, the default value
795is usually zero; for string types, the empty string.
796.It o
797Array elements and structure fields may be specified in any order.
798.El
799.Pp
800Each node type may define its own arbitrary types by providing
801the necessary routines to parse and unparse.
802.Tn ASCII
803forms defined
804for a specific node type are documented in the documentation for
805that node type.
806.Sh Generic Control Messages
807There are a number of standard predefined messages that will work
808for any node, as they are supported directly by the framework itself.
809These are defined in
810.Pa ng_message.h
811along with the basic layout of messages and other similar information.
812.Bl -tag -width xxx
813.It Dv NGM_CONNECT
814Connect to another node, using the supplied hook names on either end.
815.It Dv NGM_MKPEER
816Construct a node of the given type and then connect to it using the
817supplied hook names.
818.It Dv NGM_SHUTDOWN
819The target node should disconnect from all its neighbours and shut down.
820Persistent nodes such as those representing physical hardware
821might not disappear from the node namespace, but only reset themselves.
822The node must disconnect all of its hooks.
823This may result in neighbors shutting themselves down, and possibly a
824cascading shutdown of the entire connected graph.
825.It Dv NGM_NAME
826Assign a name to a node. Nodes can exist without having a name, and this
827is the default for nodes created using the
828.Dv NGM_MKPEER
829method. Such nodes can only be addressed relatively or by their ID number.
830.It Dv NGM_RMHOOK
831Ask the node to break a hook connection to one of its neighbours.
832Both nodes will have their
833.Dq disconnect
834method invoked.
835Either node may elect to totally shut down as a result.
836.It Dv NGM_NODEINFO
837Asks the target node to describe itself. The four returned fields
838are the node name (if named), the node type, the node ID and the
839number of hooks attached. The ID is an internal number unique to that node.
840.It Dv NGM_LISTHOOKS
841This returns the information given by
842.Dv NGM_NODEINFO ,
843but in addition
844includes an array of fields describing each link, and the description for
845the node at the far end of that link.
846.It Dv NGM_LISTNAMES
847This returns an array of node descriptions (as for
848.Dv NGM_NODEINFO ")"
849where each entry of the array describes a named node.
850All named nodes will be described.
851.It Dv NGM_LISTNODES
852This is the same as
853.Dv NGM_LISTNAMES
854except that all nodes are listed regardless of whether they have a name or not.
855.It Dv NGM_LISTTYPES
856This returns a list of all currently installed netgraph types.
857.It Dv NGM_TEXT_STATUS
858The node may return a text formatted status message.
859The status information is determined entirely by the node type.
860It is the only "generic" message
861that requires any support within the node itself and as such the node may
862elect to not support this message. The text response must be less than
863.Dv NG_TEXTRESPONSE
864bytes in length (presently 1024). This can be used to return general
865status information in human readable form.
866.It Dv NGM_BINARY2ASCII
867This message converts a binary control message to its
868.Tn ASCII
869form.
870The entire control message to be converted is contained within the
871arguments field of the
872.Dv Dv NGM_BINARY2ASCII
873message itself. If successful, the reply will contain the same control
874message in
875.Tn ASCII
876form.
877A node will typically only know how to translate messages that it
878itself understands, so the target node of the
879.Dv NGM_BINARY2ASCII
880is often the same node that would actually receive that message.
881.It Dv NGM_ASCII2BINARY
882The opposite of
883.Dv NGM_BINARY2ASCII .
884The entire control message to be converted, in
885.Tn ASCII
886form, is contained
887in the arguments section of the
888.Dv NGM_ASCII2BINARY
889and need only have the
890.Dv flags ,
891.Dv cmdstr ,
892and
893.Dv arglen
894header fields filled in, plus the NUL-terminated string version of
895the arguments in the arguments field. If successful, the reply
896contains the binary version of the control message.
897.El
898
899.Sh Flow Control Messages
900In addition to the control messages that affect nodes with respect to the
901graph, there are also a number of
902.Em Flow-control
903messages defined. At present these are
904.Em NOT
905handled automatically by the system, so
906nodes need to handle them if they are going to be used in a graph utilising
907flow control, and will be in the likely path of these messages. The
908default action of a node that doesn't understand these messages should
909be to pass them onto the next node. Hopefully some helper functions
910will assist in this eventually. These messages are also defined in
911.Pa sys/netgraph/ng_message.h
912and have a separate cookie
913.Em NG_FLOW_COOKIE
914to help identify them. They will not be covered in depth here.
915.Sh Metadata
916Data moving through the
917.Nm
918system can be accompanied by meta-data that describes some
919aspect of that data. The form of the meta-data is a fixed header,
920which contains enough information for most uses, and can optionally
921be supplemented by trailing
922.Em option
923structures, which contain a
924.Em cookie
925(see the section on control messages), an identifier, a length and optional
926data. If a node does not recognize the cookie associated with an option,
927it should ignore that option.
928.Pp
929Meta data might include such things as priority, discard eligibility,
930or special processing requirements. It might also mark a packet for
931debug status, etc. The use of meta-data is still experimental.
932.Sh INITIALIZATION
933The base
934.Nm
935code may either be statically compiled
936into the kernel or else loaded dynamically as a KLD via
937.Xr kldload 8 .
938In the former case, include
939.Bd -literal -offset 4n -compact
940
941 options NETGRAPH
942
943.Ed
944in your kernel configuration file. You may also include selected
945node types in the kernel compilation, for example:
946.Bd -literal -offset 4n -compact
947
948 options NETGRAPH
949 options NETGRAPH_SOCKET
950 options NETGRAPH_ECHO
951
952.Ed
953.Pp
954Once the
955.Nm
956subsystem is loaded, individual node types may be loaded at any time
957as KLD modules via
958.Xr kldload 8 .
959Moreover,
960.Nm
961knows how to automatically do this; when a request to create a new
962node of unknown type
963.Em type
964is made,
965.Nm
966will attempt to load the KLD module
967.Pa ng_type.ko .
968.Pp
969Types can also be installed at boot time, as certain device drivers
970may want to export each instance of the device as a netgraph node.
971.Pp
972In general, new types can be installed at any time from within the
973kernel by calling
974.Fn ng_newtype ,
975supplying a pointer to the type's
976.Dv struct ng_type
977structure.
978.Pp
979The
980.Fn NETGRAPH_INIT
981macro automates this process by using a linker set.
982.Sh EXISTING NODE TYPES
983Several node types currently exist. Each is fully documented
984in its own man page:
985.Bl -tag -width xxx
986.It SOCKET
987The socket type implements two new sockets in the new protocol domain
988.Dv PF_NETGRAPH .
989The new sockets protocols are
990.Dv NG_DATA
991and
992.Dv NG_CONTROL ,
993both of type
994.Dv SOCK_DGRAM .
995Typically one of each is associated with a socket node.
996When both sockets have closed, the node will shut down. The
997.Dv NG_DATA
998socket is used for sending and receiving data, while the
999.Dv NG_CONTROL
1000socket is used for sending and receiving control messages.
1001Data and control messages are passed using the
1002.Xr sendto 2
1003and
1004.Xr recvfrom 2
1005calls, using a
1006.Dv struct sockaddr_ng
1007socket address.
1008.Pp
1009.It HOLE
1010Responds only to generic messages and is a
1011.Dq black hole
1012for data, Useful for testing. Always accepts new hooks.
1013.Pp
1014.It ECHO
1015Responds only to generic messages and always echoes data back through the
1016hook from which it arrived. Returns any non generic messages as their
1017own response. Useful for testing. Always accepts new hooks.
1018.Pp
1019.It TEE
1020This node is useful for
1021.Dq snooping .
1022It has 4 hooks:
1023.Dv left ,
1024.Dv right ,
1025.Dv left2right ,
1026and
1027.Dv right2left .
1028Data entering from the right is passed to the left and duplicated on
1029.Dv right2left,
1030and data entering from the left is passed to the right and
1031duplicated on
1032.Dv left2right .
1033Data entering from
1034.Dv left2right
1035is sent to the right and data from
1036.Dv right2left
1037to left.
1038.Pp
1039.It RFC1490 MUX
1040Encapsulates/de-encapsulates frames encoded according to RFC 1490.
1041Has a hook for the encapsulated packets
1042.Pq Dq downstream
1043and one hook
1044for each protocol (i.e., IP, PPP, etc.).
1045.Pp
1046.It FRAME RELAY MUX
1047Encapsulates/de-encapsulates Frame Relay frames.
1048Has a hook for the encapsulated packets
1049.Pq Dq downstream
1050and one hook
1051for each DLCI.
1052.Pp
1053.It FRAME RELAY LMI
1054Automatically handles frame relay
1055.Dq LMI
1056(link management interface) operations and packets.
1057Automatically probes and detects which of several LMI standards
1058is in use at the exchange.
1059.Pp
1060.It TTY
1061This node is also a line discipline. It simply converts between mbuf
1062frames and sequential serial data, allowing a tty to appear as a netgraph
1063node. It has a programmable
1064.Dq hotkey
1065character.
1066.Pp
1067.It ASYNC
1068This node encapsulates and de-encapsulates asynchronous frames
1069according to RFC 1662. This is used in conjunction with the TTY node
1070type for supporting PPP links over asynchronous serial lines.
1071.Pp
1072.It INTERFACE
1073This node is also a system networking interface. It has hooks representing
1074each protocol family (IP, AppleTalk, IPX, etc.) and appears in the output of
1075.Xr ifconfig 8 .
1076The interfaces are named
1077.Em ng0 ,
1078.Em ng1 ,
1079etc.
1080.It ONE2MANY
1081This node implements a simple round-robin multiplexer. It can be used
1082for example to make several LAN ports act together to get a higher speed
1083link between two machines.
1084.It Various PPP related nodes.
1085There is a full multilink PPP implementation that runs in Netgraph.
1086The
1087.Em Mpd
1088port can use these modules to make a very low latency high
1089capacity ppp system. It also supports
1090.Em PPTP
1091vpns using the
1092.Em PPTP
1093node.
1094.It PPPOE
1095A server and client side implememtation of PPPoE. Used in conjunction with
1096either
1097.Xr ppp 8
1098or the
1099.Em mpd port.
1100.It BRIDGE
1101This node, togther with the ethernet nodes allows a very flexible
1102bridging system to be implemented.
1103.It KSOCKET
1104This intriguing node looks like a socket to the system but diverts
1105all data to and from the netgraph system for further processing. This allows
1106such things as UDP tunnels to be almost trivially implemented from the
1107command line.
1108
1109.El
1110.Pp
1111Refer to the section at the end of this man page for more nodes types.
1112.Sh NOTES
1113Whether a named node exists can be checked by trying to send a control message
1114to it (e.g.,
1115.Dv NGM_NODEINFO
1116).
1117If it does not exist,
1118.Er ENOENT
1119will be returned.
1120.Pp
1121All data messages are mbuf chains with the M_PKTHDR flag set.
1122.Pp
1123Nodes are responsible for freeing what they allocate.
1124There are three exceptions:
1125.Bl -tag -width xxxx
1126.It 1
1127Mbufs sent across a data link are never to be freed by the sender,
1128unless it is returned from the recipient.
1129.It 2
1130Any meta-data information traveling with the data has the same restriction.
1131It might be freed by any node the data passes through, and a
1132.Dv NULL
1133passed onwards, but the caller will never free it.
1134Two macros
1135.Fn NG_FREE_META "meta"
1136and
1137.Fn NG_FREE_DATA "m" "meta"
1138should be used if possible to free data and meta data (see
1139.Pa netgraph.h ) .
1140.It 3
1141Messages sent using
1142.Fn ng_send_message
1143are freed by the recipient. As in the case above, the addresses
1144associated with the message are freed by whatever allocated them so the
1145recipient should copy them if it wants to keep that information.
1146.El
1147.Sh FILES
1148.Bl -tag -width xxxxx -compact
1149.It Pa /sys/netgraph/netgraph.h
1150Definitions for use solely within the kernel by
1151.Nm
1152nodes.
1153.It Pa /sys/netgraph/ng_message.h
1154Definitions needed by any file that needs to deal with
1155.Nm
1156messages.
1157.It Pa /sys/netgraph/ng_socket.h
1158Definitions needed to use
1159.Nm
1160socket type nodes.
1161.It Pa /sys/netgraph/ng_{type}.h
1162Definitions needed to use
1163.Nm
1164{type}
1165nodes, including the type cookie definition.
1166.It Pa /modules/netgraph.ko
1167Netgraph subsystem loadable KLD module.
1168.It Pa /modules/ng_{type}.ko
1169Loadable KLD module for node type {type}.
1170.El
1171.Sh USER MODE SUPPORT
1172There is a library for supporting user-mode programs that wish
1173to interact with the netgraph system. See
1174.Xr netgraph 3
1175for details.
1176.Pp
1177Two user-mode support programs,
1178.Xr ngctl 8
1179and
1180.Xr nghook 8 ,
1181are available to assist manual configuration and debugging.
1182.Pp
1183There are a few useful techniques for debugging new node types.
1184First, implementing new node types in user-mode first
1185makes debugging easier.
1186The
1187.Em tee
1188node type is also useful for debugging, especially in conjunction with
1189.Xr ngctl 8
1190and
1191.Xr nghook 8 .
1192.Pp
1193Also look in /usr/share/examples/netgraph for solutions to several
1194common networking problems, solved using
1195.Nm .
1196.Sh SEE ALSO
1197.Xr socket 2 ,
1198.Xr netgraph 3 ,
1199.Xr ng_async 4 ,
1200.Xr ng_bridge 4 ,
1201.Xr ng_bpf 4 ,
1202.Xr ng_cisco 4 ,
1203.Xr ng_ether 4 ,
1204.Xr ng_echo 4 ,
1205.Xr ng_ether 4 ,
1206.Xr ng_frame_relay 4 ,
1207.Xr ng_hole 4 ,
1208.Xr ng_iface 4 ,
1209.Xr ng_ksocket 4 ,
1210.Xr ng_lmi 4 ,
1211.Xr ng_mppc 4 ,
1212.Xr ng_ppp 4 ,
1213.Xr ng_pppoe 4 ,
1214.Xr ng_pptpgre 4 ,
1215.Xr ng_rfc1490 4 ,
1216.Xr ng_socket 4 ,
1217.Xr ng_tee 4 ,
1218.Xr ng_tty 4 ,
1219.Xr ng_UI 4 ,
1220.Xr ng_vjc 4 ,
1221.Xr ng_{type} 4 ,
1222.Xr ngctl 8 ,
1223.Xr nghook 8
1224.Sh HISTORY
1225The
1226.Nm
1227system was designed and first implemented at Whistle Communications, Inc.
1228in a version of
1229.Fx 2.2
1230customized for the Whistle InterJet.
1231It first made its debut in the main tree in
1232.Fx 3.4 .
1233.Sh AUTHORS
1234.An -nosplit
1235.An Julian Elischer Aq julian@FreeBSD.org ,
1236with contributions by
1237.An Archie Cobbs Aq archie@FreeBSD.org .
2.\" All rights reserved.
3.\"
4.\" Subject to the following obligations and disclaimer of warranty, use and
5.\" redistribution of this software, in source or object code forms, with or
6.\" without modifications are expressly permitted by Whistle Communications;
7.\" provided, however, that:
8.\" 1. Any and all reproductions of the source or object code must include the
9.\" copyright notice above and the following disclaimer of warranties; and
10.\" 2. No rights are granted, in any manner or form, to use Whistle
11.\" Communications, Inc. trademarks, including the mark "WHISTLE
12.\" COMMUNICATIONS" on advertising, endorsements, or otherwise except as
13.\" such appears in the above copyright notice or in the software.
14.\"
15.\" THIS SOFTWARE IS BEING PROVIDED BY WHISTLE COMMUNICATIONS "AS IS", AND
16.\" TO THE MAXIMUM EXTENT PERMITTED BY LAW, WHISTLE COMMUNICATIONS MAKES NO
17.\" REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, REGARDING THIS SOFTWARE,
18.\" INCLUDING WITHOUT LIMITATION, ANY AND ALL IMPLIED WARRANTIES OF
19.\" MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT.
20.\" WHISTLE COMMUNICATIONS DOES NOT WARRANT, GUARANTEE, OR MAKE ANY
21.\" REPRESENTATIONS REGARDING THE USE OF, OR THE RESULTS OF THE USE OF THIS
22.\" SOFTWARE IN TERMS OF ITS CORRECTNESS, ACCURACY, RELIABILITY OR OTHERWISE.
23.\" IN NO EVENT SHALL WHISTLE COMMUNICATIONS BE LIABLE FOR ANY DAMAGES
24.\" RESULTING FROM OR ARISING OUT OF ANY USE OF THIS SOFTWARE, INCLUDING
25.\" WITHOUT LIMITATION, ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
26.\" PUNITIVE, OR CONSEQUENTIAL DAMAGES, PROCUREMENT OF SUBSTITUTE GOODS OR
27.\" SERVICES, LOSS OF USE, DATA OR PROFITS, HOWEVER CAUSED AND UNDER ANY
28.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
30.\" THIS SOFTWARE, EVEN IF WHISTLE COMMUNICATIONS IS ADVISED OF THE POSSIBILITY
31.\" OF SUCH DAMAGE.
32.\"
33.\" Authors: Julian Elischer <julian@FreeBSD.org>
34.\" Archie Cobbs <archie@FreeBSD.org>
35.\"
36.\" $FreeBSD: head/share/man/man4/netgraph.4 69922 2000-12-12 18:52:14Z julian $
37.\" $Whistle: netgraph.4,v 1.7 1999/01/28 23:54:52 julian Exp $
38.\"
39.Dd January 19, 1999
40.Dt NETGRAPH 4
41.Os FreeBSD
42.Sh NAME
43.Nm netgraph
44.Nd graph based kernel networking subsystem
45.Sh DESCRIPTION
46The
47.Nm
48system provides a uniform and modular system for the implementation
49of kernel objects which perform various networking functions. The objects,
50known as
51.Em nodes ,
52can be arranged into arbitrarily complicated graphs. Nodes have
53.Em hooks
54which are used to connect two nodes together, forming the edges in the graph.
55Nodes communicate along the edges to process data, implement protocols, etc.
56.Pp
57The aim of
58.Nm
59is to supplement rather than replace the existing kernel networking
60infrastructure. It provides:
61.Pp
62.Bl -bullet -compact -offset 2n
63.It
64A flexible way of combining protocol and link level drivers
65.It
66A modular way to implement new protocols
67.It
68A common framework for kernel entities to inter-communicate
69.It
70A reasonably fast, kernel-based implementation
71.El
72.Sh Nodes and Types
73The most fundamental concept in
74.Nm
75is that of a
76.Em node .
77All nodes implement a number of predefined methods which allow them
78to interact with other nodes in a well defined manner.
79.Pp
80Each node has a
81.Em type ,
82which is a static property of the node determined at node creation time.
83A node's type is described by a unique
84.Tn ASCII
85type name.
86The type implies what the node does and how it may be connected
87to other nodes.
88.Pp
89In object-oriented language, types are classes and nodes are instances
90of their respective class. All node types are subclasses of the generic node
91type, and hence inherit certain common functionality and capabilities
92(e.g., the ability to have an
93.Tn ASCII
94name).
95.Pp
96Nodes may be assigned a globally unique
97.Tn ASCII
98name which can be
99used to refer to the node.
100The name must not contain the characters
101.Dq \&.
102or
103.Dq \&:
104and is limited to
105.Dv "NG_NODELEN + 1"
106characters (including NUL byte).
107.Pp
108Each node instance has a unique
109.Em ID number
110which is expressed as a 32-bit hex value. This value may be used to
111refer to a node when there is no
112.Tn ASCII
113name assigned to it.
114.Sh Hooks
115Nodes are connected to other nodes by connecting a pair of
116.Em hooks ,
117one from each node. Data flows bidirectionally between nodes along
118connected pairs of hooks. A node may have as many hooks as it
119needs, and may assign whatever meaning it wants to a hook.
120.Pp
121Hooks have these properties:
122.Pp
123.Bl -bullet -compact -offset 2n
124.It
125A hook has an
126.Tn ASCII
127name which is unique among all hooks
128on that node (other hooks on other nodes may have the same name).
129The name must not contain a
130.Dq \&.
131or a
132.Dq \&:
133and is
134limited to
135.Dv "NG_HOOKLEN + 1"
136characters (including NUL byte).
137.It
138A hook is always connected to another hook. That is, hooks are
139created at the time they are connected, and breaking an edge by
140removing either hook destroys both hooks.
141.It
142A hook can be set into a state where incoming packets are always queued
143by the input queuing system, rather than being delivered directly. This
144is used when the two joined nodes need to be decoupled, e.g. if they are
145running at different processor priority levels. (spl)
146.El
147.Pp
148A node may decide to assign special meaning to some hooks.
149For example, connecting to the hook named
150.Dq debug
151might trigger
152the node to start sending debugging information to that hook.
153.Sh Data Flow
154Two types of information flow between nodes: data messages and
155control messages. Data messages are passed in mbuf chains along the edges
156in the graph, one edge at a time. The first mbuf in a chain must have the
157.Dv M_PKTHDR
158flag set. Each node decides how to handle data coming in on its hooks.
159.Pp
160Control messages are type-specific C structures sent from one node
161directly to some arbitrary other node. Control messages have a common
162header format, followed by type-specific data, and are binary structures
163for efficiency. However, node types also may support conversion of the
164type specific data between binary and
165.Tn ASCII
166for debugging and human interface purposes (see the
167.Dv NGM_ASCII2BINARY
168and
169.Dv NGM_BINARY2ASCII
170generic control messages below). Nodes are not required to support
171these conversions.
172.Pp
173There are three ways to address a control message. If
174there is a sequence of edges connecting the two nodes, the message
175may be
176.Dq source routed
177by specifying the corresponding sequence
178of
179.Tn ASCII
180hook names as the destination address for the message (relative
181addressing). If the destination is adjacent to the source, then the source
182node may simply specify (as a pointer in the code) the hook across which the
183message should be sent. Otherwise, the recipient node global
184.Tn ASCII
185name
186(or equivalent ID based name) is used as the destination address
187for the message (absolute addressing). The two types of
188.Tn ASCII
189addressing
190may be combined, by specifying an absolute start node and a sequence
191of hooks. Only the
192.Tn ASCII
193addressing modes are available to control programs outside the kernel,
194as use of direct pointers is limited of course to kernel modules.
195.Pp
196Messages often represent commands that are followed by a reply message
197in the reverse direction. To facilitate this, the recipient of a
198control message is supplied with a
199.Dq return address
200that is suitable for addressing a reply.
201In addition, depending on the topology of
202the graph and whether the source has requested it, a pointer to a
203pointer that can be read by the source node may also be supplied.
204This allows the destination node to directly respond in a
205synchronous manner when control returns to the source node, by
206simply pointing the supplied pointer to the response message.
207Such synchronous message responses are more efficient but are not always possible.
208.Pp
209Each control message contains a 32 bit value called a
210.Em typecookie
211indicating the type of the message, i.e., how to interpret it.
212Typically each type defines a unique typecookie for the messages
213that it understands. However, a node may choose to recognize and
214implement more than one type of message.
215.Pp
216If a message is delivered to an address that implies that it arrived
217at that node through a particular hook, (as opposed to having been directly
218addressed using its ID or global name), then that hook is identified to the
219receiving node. This allows a message to be rerouted or passed on, should
220a node decide that this is required, in much the same way that data packets
221are passed around between nodes. A set of standard
222messages for flow control and link management purposes are
223defined by the base system that are usually
224passed around in this manner. Flow control message would usually travel
225in the opposite direction to the data to which they pertain.
226.Pp
227Since flow control packets can also result from data being sent, it is also
228possible to return a synchronous message response to a data packet being
229sent between nodes. (See later).
230.Sh Netgraph is Functional
231In order to minimize latency, most
232.Nm
233operations are functional.
234That is, data and control messages are delivered by making function
235calls rather than by using queues and mailboxes. For example, if node
236A wishes to send a data mbuf to neighboring node B, it calls the
237generic
238.Nm
239data delivery function. This function in turn locates
240node B and calls B's
241.Dq receive data
242method. There are exceptions to this.
243.Pp
244It is allowable for nodes to reject a data packet, or to pass it back to the
245caller in a modified or completely replaced form. The caller can notify the
246node being called that it does not wish to receive any such packets
247by using the
248.Fn NG_SEND_DATA
249and
250.Fn NG_SEND_DATA_ONLY
251macros, in which case, the second node should just discard rejected packets.
252If the sender knows how to handle returned packets, it must use the
253.Fn NG_SEND_DATA_RET
254macro, which will adjust the parameters to point to the returned data
255or NULL if no data was returned to the caller. No packet return is possible
256across a queuing link (though an explicitly sent return is of course possible,
257it doesn't mean quite the same thing).
258.Pp
259While this mode of operation
260results in good performance, it has a few implications for node
261developers:
262.Pp
263.Bl -bullet -compact -offset 2n
264.It
265Whenever a node delivers a data or control message, the node
266may need to allow for the possibility of receiving a returning
267message before the original delivery function call returns.
268.It
269Netgraph nodes and support routines generally run at
270.Fn splnet .
271However, some nodes may want to send data and control messages
272from a different priority level. Netgraph supplies a mechanism which
273utilizes the NETISR system to move message and data delivery to
274.Fn splnet .
275Nodes that run at other priorities (e.g. interfaces) can be directly
276linked to other nodes so that the combination runs at the other priority,
277however any interaction with nodes running at splnet MUST be achieved via the
278queueing functions, (which use the
279.Fn netisr
280feature of the kernel).
281Note that messages are always received at
282.Fn splnet .
283.It
284It's possible for an infinite loop to occur if the graph contains cycles.
285.El
286.Pp
287So far, these issues have not proven problematical in practice.
288.Sh Interaction With Other Parts of the Kernel
289A node may have a hidden interaction with other components of the
290kernel outside of the
291.Nm
292subsystem, such as device hardware,
293kernel protocol stacks, etc. In fact, one of the benefits of
294.Nm
295is the ability to join disparate kernel networking entities together in a
296consistent communication framework.
297.Pp
298An example is the node type
299.Em socket
300which is both a netgraph node and a
301.Xr socket 2
302BSD socket in the protocol family
303.Dv PF_NETGRAPH .
304Socket nodes allow user processes to participate in
305.Nm .
306Other nodes communicate with socket nodes using the usual methods, and the
307node hides the fact that it is also passing information to and from a
308cooperating user process.
309.Pp
310Another example is a device driver that presents
311a node interface to the hardware.
312.Sh Node Methods
313Nodes are notified of the following actions via function calls
314to the following node methods (all at
315.Fn splnet )
316and may accept or reject that action (by returning the appropriate
317error code):
318.Bl -tag -width xxx
319.It Creation of a new node
320The constructor for the type is called. If creation of a new node is
321allowed, the constructor must call the generic node creation
322function (in object-oriented terms, the superclass constructor)
323and then allocate any special resources it needs. For nodes that
324correspond to hardware, this is typically done during the device
325attach routine. Often a global
326.Tn ASCII
327name corresponding to the
328device name is assigned here as well.
329.It Creation of a new hook
330The hook is created and tentatively
331linked to the node, and the node is told about the name that will be
332used to describe this hook. The node sets up any special data structures
333it needs, or may reject the connection, based on the name of the hook.
334.It Successful connection of two hooks
335After both ends have accepted their
336hooks, and the links have been made, the nodes get a chance to
337find out who their peer is across the link and can then decide to reject
338the connection. Tear-down is automatic. This is also the time at which
339a node may decide whether to set a particular hook (or its peer) into
340.Em queuing
341mode.
342.It Destruction of a hook
343The node is notified of a broken connection. The node may consider some hooks
344to be critical to operation and others to be expendable: the disconnection
345of one hook may be an acceptable event while for another it
346may effect a total shutdown for the node.
347.It Shutdown of a node
348This method allows a node to clean up
349and to ensure that any actions that need to be performed
350at this time are taken. The method must call the generic (i.e., superclass)
351node destructor to get rid of the generic components of the node.
352Some nodes (usually associated with a piece of hardware) may be
353.Em persistent
354in that a shutdown breaks all edges and resets the node,
355but doesn't remove it, in which case the generic destructor is not called.
356.El
357.Sh Sending and Receiving Data
358Two other methods are also supported by all nodes:
359.Bl -tag -width xxx
360.It Receive data message
361An mbuf chain is passed to the node.
362The node is notified on which hook the data arrived,
363and can use this information in its processing decision.
364The receiving node must always
365.Fn m_freem
366the mbuf chain on completion or error, pass it back (reject it), or pass
367it on to another node
368(or kernel module) which will then be responsible for freeing it.
369If a node passes a packet back to the caller, it does not have to be the
370same mbuf, in which case the original must be freed. Passing a packet
371back allows a module to modify the original data (e.g. encrypt it),
372or in some other way filter it (e.g. packet filtering).
373.Pp
374In addition to the mbuf chain itself there is also a pointer to a
375structure describing meta-data about the message
376(e.g. priority information). This pointer may be
377.Dv NULL
378if there is no additional information. The format for this information is
379described in
380.Pa sys/netgraph/netgraph.h .
381The memory for meta-data must allocated via
382.Fn malloc
383with type
384.Dv M_NETGRAPH .
385As with the data itself, it is the receiver's responsibility to
386.Fn free
387the meta-data. If the mbuf chain is freed the meta-data must
388be freed at the same time. If the meta-data is freed but the
389real data on is passed on, then a
390.Dv NULL
391pointer must be substituted.
392Meta-data may be passed back in the same way that mbuf data may be passed back.
393As with mbuf data, the rejected or returned meta-data pointer may point to
394the same or different meta-data as that passed in,
395and if it is different, the original must be freed.
396.Pp
397The receiving node may decide to defer the data by queueing it in the
398.Nm
399NETISR system (see below). It achieves this by setting the
400.Dv HK_QUEUE
401flag in the flags word of the hook on which that data will arrive.
402The infrastructure will respect that bit and queue the data for delivery at
403a later time, rather than deliver it directly. A node may decide to set
404the bit on the
405.Em peer
406node, so that it's own output packets are queued. This is used
407by device drivers running at different processor priorities to transfer
408packet delivery to the splnet() level at which the bulk of
409.Nm
410runs.
411.Pp
412The structure and use of meta-data is still experimental, but is
413presently used in frame-relay to indicate that management packets
414should be queued for transmission
415at a higher priority than data packets. This is required for
416conformance with Frame Relay standards.
417.Pp
418The node may also receive information allowing it to send a synchronous
419message response to one of the originators of the data. it is envisionned
420that such a message would contain error or flow-control information.
421Standard messages for these purposes have been defined in
422.Pa sys/netgraph/netgraph.h .
423.It Receive control message
424This method is called when a control message is addressed to the node.
425A return address is always supplied, giving the address of the node
426that originated the message so a reply message can be sent anytime later.
427.Pp
428It is possible for a synchronous reply to be made, and in fact this
429is more common in practice.
430This is done by setting a pointer (supplied as an extra function parameter)
431to point to the reply.
432Then when the control message delivery function returns,
433the caller can check if this pointer has been made non-NULL,
434and if so then it points to the reply message allocated via
435.Fn malloc
436and containing the synchronous response. In both directions,
437(request and response) it is up to the
438receiver of that message to
439.Fn free
440the control message buffer. All control messages and replies are
441allocated with
442.Fn malloc
443type
444.Dv M_NETGRAPH .
445.Pp
446If the message was delivered via a specific hook, that hook will
447also be made known, which allows the use of such things as flow-control
448messages, and status change messages, where the node may want to forward
449the message out another hook to that on which it arrived.
450.El
451.Pp
452Much use has been made of reference counts, so that nodes being
453free'd of all references are automatically freed, and this behaviour
454has been tested and debugged to present a consistent and trustworthy
455framework for the
456.Dq type module
457writer to use.
458.Sh Addressing
459The
460.Nm
461framework provides an unambiguous and simple to use method of specifically
462addressing any single node in the graph. The naming of a node is
463independent of its type, in that another node, or external component
464need not know anything about the node's type in order to address it so as
465to send it a generic message type. Node and hook names should be
466chosen so as to make addresses meaningful.
467.Pp
468Addresses are either absolute or relative. An absolute address begins
469with a node name, (or ID), followed by a colon, followed by a sequence of hook
470names separated by periods. This addresses the node reached by starting
471at the named node and following the specified sequence of hooks.
472A relative address includes only the sequence of hook names, implicitly
473starting hook traversal at the local node.
474.Pp
475There are a couple of special possibilities for the node name.
476The name
477.Dq \&.
478(referred to as
479.Dq \&.: )
480always refers to the local node.
481Also, nodes that have no global name may be addressed by their ID numbers,
482by enclosing the hex representation of the ID number within square brackets.
483Here are some examples of valid netgraph addresses:
484.Bd -literal -offset 4n -compact
485
486 .:
487 foo:
488 .:hook1
489 foo:hook1.hook2
490 [d80]:hook1
491.Ed
492.Pp
493Consider the following set of nodes might be created for a site with
494a single physical frame relay line having two active logical DLCI channels,
495with RFC-1490 frames on DLCI 16 and PPP frames over DLCI 20:
496.Pp
497.Bd -literal
498[type SYNC ] [type FRAME] [type RFC1490]
499[ "Frame1" ](uplink)<-->(data)[<un-named>](dlci16)<-->(mux)[<un-named> ]
500[ A ] [ B ](dlci20)<---+ [ C ]
501 |
502 | [ type PPP ]
503 +>(mux)[<un-named>]
504 [ D ]
505.Ed
506.Pp
507One could always send a control message to node C from anywhere
508by using the name
509.Em "Frame1:uplink.dlci16" .
510In this case, node C would also be notified that the message
511reached it via its hook
512.Dq mux .
513Similarly,
514.Em "Frame1:uplink.dlci20"
515could reliably be used to reach node D, and node A could refer
516to node B as
517.Em ".:uplink" ,
518or simply
519.Em "uplink" .
520Conversely, B can refer to A as
521.Em "data" .
522The address
523.Em "mux.data"
524could be used by both nodes C and D to address a message to node A.
525.Pp
526Note that this is only for
527.Em control messages .
528In each of these cases, where a relative addressing mode is
529used, the recipient is notified of the hook on which the
530message arrived, as well as
531the originating node.
532This allows the option of hop-by-hop distibution of messages and
533state information.
534Data messages are
535.Em only
536routed one hop at a time, by specifying the departing
537hook, with each node making
538the next routing decision. So when B receives a frame on hook
539.Dq data
540it decodes the frame relay header to determine the DLCI,
541and then forwards the unwrapped frame to either C or D.
542.Pp
543In a similar way, flow control messages may be routed in the reverse
544direction to outgoing data. For example a "buffer nearly full" message from
545.Em "Frame1:
546would be passed to node
547.Em B
548which might decide to send similar messages to both nodes
549.Em C
550and
551.Em D .
552The nodes would use
553.Em "Direct hook pointer"
554addressing to route the messages. The message may have travelled from
555.Em "Frame1:
556to
557.Em B
558as a synchronous reply, saving time and cycles.
559
560
561.Pp
562A similar graph might be used to represent multi-link PPP running
563over an ISDN line:
564.Pp
565.Bd -literal
566[ type BRI ](B1)<--->(link1)[ type MPP ]
567[ "ISDN1" ](B2)<--->(link2)[ (no name) ]
568[ ](D) <-+
569 |
570 +----------------+
571 |
572 +->(switch)[ type Q.921 ](term1)<---->(datalink)[ type Q.931 ]
573 [ (no name) ] [ (no name) ]
574.Ed
575.Sh Netgraph Structures
576Interesting members of the node and hook structures are shown below
577however you should
578check
579.Pa sys/netgraph/netgraph.h
580on your system for more up-to-date versions.
581
582.Bd -literal
583struct ng_node {
584 char *name; /* Optional globally unique name */
585 void *private; /* Node implementation private info */
586 struct ng_type *type; /* The type of this node */
587 int refs; /* Number of references to this struct */
588 int numhooks; /* Number of connected hooks */
589 hook_p hooks; /* Linked list of (connected) hooks */
590};
591typedef struct ng_node *node_p;
592
593struct ng_hook {
594 char *name; /* This node's name for this hook */
595 void *private; /* Node implementation private info */
596 int refs; /* Number of references to this struct */
597 struct ng_node *node; /* The node this hook is attached to */
598 struct ng_hook *peer; /* The other hook in this connected pair */
599 struct ng_hook *next; /* Next in list of hooks for this node */
600};
601typedef struct ng_hook *hook_p;
602.Ed
603.Pp
604The maintenance of the name pointers, reference counts, and linked list
605of hooks for each node is handled automatically by the
606.Nm
607subsystem.
608Typically a node's private info contains a back-pointer to the node or hook
609structure, which counts as a new reference that must be registered by
610incrementing
611.Dv "node->refs" .
612.Pp
613From a hook you can obtain the corresponding node, and from
614a node the list of all active hooks.
615.Pp
616Node types are described by the structures below:
617.Bd -literal
618/** How to convert a control message from binary <-> ASCII */
619struct ng_cmdlist {
620 u_int32_t cookie; /* typecookie */
621 int cmd; /* command number */
622 const char *name; /* command name */
623 const struct ng_parse_type *mesgType; /* args if !NGF_RESP */
624 const struct ng_parse_type *respType; /* args if NGF_RESP */
625};
626
627struct ng_type {
628 u_int32_t version; /* Must equal NG_VERSION */
629 const char *name; /* Unique type name */
630
631 /* Module event handler */
632 modeventhand_t mod_event; /* Handle load/unload (optional) */
633
634 /* Constructor */
635 int (*constructor)(node_p *node); /* Create a new node */
636
637 /** Methods using the node **/
638 int (*rcvmsg)(node_p node, /* Receive control message */
639 struct ng_mesg *msg, /* The message */
640 const char *retaddr, /* Return address */
641 struct ng_mesg **resp /* Synchronous response */
642 hook_p lasthook); /* last hook traversed */
643 int (*shutdown)(node_p node); /* Shutdown this node */
644 int (*newhook)(node_p node, /* create a new hook */
645 hook_p hook, /* Pre-allocated struct */
646 const char *name); /* Name for new hook */
647
648 /** Methods using the hook **/
649 int (*connect)(hook_p hook); /* Confirm new hook attachment */
650 int (*rcvdata)(hook_p hook, /* Receive data on a hook */
651 struct mbuf *m, /* The data in an mbuf */
652 meta_p meta, /* Meta-data, if any */
653 struct mbuf **ret_m, /* return data here */
654 meta_p *ret_meta, /* return Meta-data here */
655 struct ng_message **resp); /* Synchronous reply info */
656 int (*disconnect)(hook_p hook); /* Notify disconnection of hook */
657
658 /** How to convert control messages binary <-> ASCII */
659 const struct ng_cmdlist *cmdlist; /* Optional; may be NULL */
660};
661.Ed
662.Pp
663Control messages have the following structure:
664.Bd -literal
665#define NG_CMDSTRLEN 15 /* Max command string (16 with null) */
666
667struct ng_mesg {
668 struct ng_msghdr {
669 u_char version; /* Must equal NG_VERSION */
670 u_char spare; /* Pad to 2 bytes */
671 u_short arglen; /* Length of cmd/resp data */
672 u_long flags; /* Message status flags */
673 u_long token; /* Reply should have the same token */
674 u_long typecookie; /* Node type understanding this message */
675 u_long cmd; /* Command identifier */
676 u_char cmdstr[NG_CMDSTRLEN+1]; /* Cmd string (for debug) */
677 } header;
678 char data[0]; /* Start of cmd/resp data */
679};
680
681#define NG_VERSION 3 /* Netgraph version */
682#define NGF_ORIG 0x0000 /* Command */
683#define NGF_RESP 0x0001 /* Response */
684.Ed
685.Pp
686Control messages have the fixed header shown above, followed by a
687variable length data section which depends on the type cookie
688and the command. Each field is explained below:
689.Bl -tag -width xxx
690.It Dv version
691Indicates the version of netgraph itself. The current version is
692.Dv NG_VERSION .
693.It Dv arglen
694This is the length of any extra arguments, which begin at
695.Dv data .
696.It Dv flags
697Indicates whether this is a command or a response control message.
698.It Dv token
699The
700.Dv token
701is a means by which a sender can match a reply message to the
702corresponding command message; the reply always has the same token.
703.Pp
704.It Dv typecookie
705The corresponding node type's unique 32-bit value.
706If a node doesn't recognize the type cookie it must reject the message
707by returning
708.Er EINVAL .
709.Pp
710Each type should have an include file that defines the commands,
711argument format, and cookie for its own messages.
712The typecookie
713insures that the same header file was included by both sender and
714receiver; when an incompatible change in the header file is made,
715the typecookie
716.Em must
717be changed.
718The de facto method for generating unique type cookies is to take the
719seconds from the epoch at the time the header file is written
720(i.e., the output of
721.Dv "date -u +'%s'" ) .
722.Pp
723There is a predefined typecookie
724.Dv NGM_GENERIC_COOKIE
725for the
726.Dq generic
727node type, and
728a corresponding set of generic messages which all nodes understand.
729The handling of these messages is automatic.
730.It Dv command
731The identifier for the message command. This is type specific,
732and is defined in the same header file as the typecookie.
733.It Dv cmdstr
734Room for a short human readable version of
735.Dq command
736(for debugging purposes only).
737.El
738.Pp
739Some modules may choose to implement messages from more than one
740of the header files and thus recognize more than one type cookie.
741.Sh Control Message ASCII Form
742Control messages are in binary format for efficiency. However, for
743debugging and human interface purposes, and if the node type supports
744it, control messages may be converted to and from an equivalent
745.Tn ASCII
746form. The
747.Tn ASCII
748form is similar to the binary form, with two exceptions:
749.Pp
750.Bl -tag -compact -width xxx
751.It o
752The
753.Dv cmdstr
754header field must contain the
755.Tn ASCII
756name of the command, corresponding to the
757.Dv cmd
758header field.
759.It o
760The
761.Dv args
762field contains a NUL-terminated
763.Tn ASCII
764string version of the message arguments.
765.El
766.Pp
767In general, the arguments field of a control messgage can be any
768arbitrary C data type. Netgraph includes parsing routines to support
769some pre-defined datatypes in
770.Tn ASCII
771with this simple syntax:
772.Pp
773.Bl -tag -compact -width xxx
774.It o
775Integer types are represented by base 8, 10, or 16 numbers.
776.It o
777Strings are enclosed in double quotes and respect the normal
778C language backslash escapes.
779.It o
780IP addresses have the obvious form.
781.It o
782Arrays are enclosed in square brackets, with the elements listed
783consecutively starting at index zero. An element may have an optional
784index and equals sign preceeding it. Whenever an element
785does not have an explicit index, the index is implicitly the previous
786element's index plus one.
787.It o
788Structures are enclosed in curly braces, and each field is specified
789in the form
790.Dq fieldname=value .
791.It o
792Any array element or structure field whose value is equal to its
793.Dq default value
794may be omitted. For integer types, the default value
795is usually zero; for string types, the empty string.
796.It o
797Array elements and structure fields may be specified in any order.
798.El
799.Pp
800Each node type may define its own arbitrary types by providing
801the necessary routines to parse and unparse.
802.Tn ASCII
803forms defined
804for a specific node type are documented in the documentation for
805that node type.
806.Sh Generic Control Messages
807There are a number of standard predefined messages that will work
808for any node, as they are supported directly by the framework itself.
809These are defined in
810.Pa ng_message.h
811along with the basic layout of messages and other similar information.
812.Bl -tag -width xxx
813.It Dv NGM_CONNECT
814Connect to another node, using the supplied hook names on either end.
815.It Dv NGM_MKPEER
816Construct a node of the given type and then connect to it using the
817supplied hook names.
818.It Dv NGM_SHUTDOWN
819The target node should disconnect from all its neighbours and shut down.
820Persistent nodes such as those representing physical hardware
821might not disappear from the node namespace, but only reset themselves.
822The node must disconnect all of its hooks.
823This may result in neighbors shutting themselves down, and possibly a
824cascading shutdown of the entire connected graph.
825.It Dv NGM_NAME
826Assign a name to a node. Nodes can exist without having a name, and this
827is the default for nodes created using the
828.Dv NGM_MKPEER
829method. Such nodes can only be addressed relatively or by their ID number.
830.It Dv NGM_RMHOOK
831Ask the node to break a hook connection to one of its neighbours.
832Both nodes will have their
833.Dq disconnect
834method invoked.
835Either node may elect to totally shut down as a result.
836.It Dv NGM_NODEINFO
837Asks the target node to describe itself. The four returned fields
838are the node name (if named), the node type, the node ID and the
839number of hooks attached. The ID is an internal number unique to that node.
840.It Dv NGM_LISTHOOKS
841This returns the information given by
842.Dv NGM_NODEINFO ,
843but in addition
844includes an array of fields describing each link, and the description for
845the node at the far end of that link.
846.It Dv NGM_LISTNAMES
847This returns an array of node descriptions (as for
848.Dv NGM_NODEINFO ")"
849where each entry of the array describes a named node.
850All named nodes will be described.
851.It Dv NGM_LISTNODES
852This is the same as
853.Dv NGM_LISTNAMES
854except that all nodes are listed regardless of whether they have a name or not.
855.It Dv NGM_LISTTYPES
856This returns a list of all currently installed netgraph types.
857.It Dv NGM_TEXT_STATUS
858The node may return a text formatted status message.
859The status information is determined entirely by the node type.
860It is the only "generic" message
861that requires any support within the node itself and as such the node may
862elect to not support this message. The text response must be less than
863.Dv NG_TEXTRESPONSE
864bytes in length (presently 1024). This can be used to return general
865status information in human readable form.
866.It Dv NGM_BINARY2ASCII
867This message converts a binary control message to its
868.Tn ASCII
869form.
870The entire control message to be converted is contained within the
871arguments field of the
872.Dv Dv NGM_BINARY2ASCII
873message itself. If successful, the reply will contain the same control
874message in
875.Tn ASCII
876form.
877A node will typically only know how to translate messages that it
878itself understands, so the target node of the
879.Dv NGM_BINARY2ASCII
880is often the same node that would actually receive that message.
881.It Dv NGM_ASCII2BINARY
882The opposite of
883.Dv NGM_BINARY2ASCII .
884The entire control message to be converted, in
885.Tn ASCII
886form, is contained
887in the arguments section of the
888.Dv NGM_ASCII2BINARY
889and need only have the
890.Dv flags ,
891.Dv cmdstr ,
892and
893.Dv arglen
894header fields filled in, plus the NUL-terminated string version of
895the arguments in the arguments field. If successful, the reply
896contains the binary version of the control message.
897.El
898
899.Sh Flow Control Messages
900In addition to the control messages that affect nodes with respect to the
901graph, there are also a number of
902.Em Flow-control
903messages defined. At present these are
904.Em NOT
905handled automatically by the system, so
906nodes need to handle them if they are going to be used in a graph utilising
907flow control, and will be in the likely path of these messages. The
908default action of a node that doesn't understand these messages should
909be to pass them onto the next node. Hopefully some helper functions
910will assist in this eventually. These messages are also defined in
911.Pa sys/netgraph/ng_message.h
912and have a separate cookie
913.Em NG_FLOW_COOKIE
914to help identify them. They will not be covered in depth here.
915.Sh Metadata
916Data moving through the
917.Nm
918system can be accompanied by meta-data that describes some
919aspect of that data. The form of the meta-data is a fixed header,
920which contains enough information for most uses, and can optionally
921be supplemented by trailing
922.Em option
923structures, which contain a
924.Em cookie
925(see the section on control messages), an identifier, a length and optional
926data. If a node does not recognize the cookie associated with an option,
927it should ignore that option.
928.Pp
929Meta data might include such things as priority, discard eligibility,
930or special processing requirements. It might also mark a packet for
931debug status, etc. The use of meta-data is still experimental.
932.Sh INITIALIZATION
933The base
934.Nm
935code may either be statically compiled
936into the kernel or else loaded dynamically as a KLD via
937.Xr kldload 8 .
938In the former case, include
939.Bd -literal -offset 4n -compact
940
941 options NETGRAPH
942
943.Ed
944in your kernel configuration file. You may also include selected
945node types in the kernel compilation, for example:
946.Bd -literal -offset 4n -compact
947
948 options NETGRAPH
949 options NETGRAPH_SOCKET
950 options NETGRAPH_ECHO
951
952.Ed
953.Pp
954Once the
955.Nm
956subsystem is loaded, individual node types may be loaded at any time
957as KLD modules via
958.Xr kldload 8 .
959Moreover,
960.Nm
961knows how to automatically do this; when a request to create a new
962node of unknown type
963.Em type
964is made,
965.Nm
966will attempt to load the KLD module
967.Pa ng_type.ko .
968.Pp
969Types can also be installed at boot time, as certain device drivers
970may want to export each instance of the device as a netgraph node.
971.Pp
972In general, new types can be installed at any time from within the
973kernel by calling
974.Fn ng_newtype ,
975supplying a pointer to the type's
976.Dv struct ng_type
977structure.
978.Pp
979The
980.Fn NETGRAPH_INIT
981macro automates this process by using a linker set.
982.Sh EXISTING NODE TYPES
983Several node types currently exist. Each is fully documented
984in its own man page:
985.Bl -tag -width xxx
986.It SOCKET
987The socket type implements two new sockets in the new protocol domain
988.Dv PF_NETGRAPH .
989The new sockets protocols are
990.Dv NG_DATA
991and
992.Dv NG_CONTROL ,
993both of type
994.Dv SOCK_DGRAM .
995Typically one of each is associated with a socket node.
996When both sockets have closed, the node will shut down. The
997.Dv NG_DATA
998socket is used for sending and receiving data, while the
999.Dv NG_CONTROL
1000socket is used for sending and receiving control messages.
1001Data and control messages are passed using the
1002.Xr sendto 2
1003and
1004.Xr recvfrom 2
1005calls, using a
1006.Dv struct sockaddr_ng
1007socket address.
1008.Pp
1009.It HOLE
1010Responds only to generic messages and is a
1011.Dq black hole
1012for data, Useful for testing. Always accepts new hooks.
1013.Pp
1014.It ECHO
1015Responds only to generic messages and always echoes data back through the
1016hook from which it arrived. Returns any non generic messages as their
1017own response. Useful for testing. Always accepts new hooks.
1018.Pp
1019.It TEE
1020This node is useful for
1021.Dq snooping .
1022It has 4 hooks:
1023.Dv left ,
1024.Dv right ,
1025.Dv left2right ,
1026and
1027.Dv right2left .
1028Data entering from the right is passed to the left and duplicated on
1029.Dv right2left,
1030and data entering from the left is passed to the right and
1031duplicated on
1032.Dv left2right .
1033Data entering from
1034.Dv left2right
1035is sent to the right and data from
1036.Dv right2left
1037to left.
1038.Pp
1039.It RFC1490 MUX
1040Encapsulates/de-encapsulates frames encoded according to RFC 1490.
1041Has a hook for the encapsulated packets
1042.Pq Dq downstream
1043and one hook
1044for each protocol (i.e., IP, PPP, etc.).
1045.Pp
1046.It FRAME RELAY MUX
1047Encapsulates/de-encapsulates Frame Relay frames.
1048Has a hook for the encapsulated packets
1049.Pq Dq downstream
1050and one hook
1051for each DLCI.
1052.Pp
1053.It FRAME RELAY LMI
1054Automatically handles frame relay
1055.Dq LMI
1056(link management interface) operations and packets.
1057Automatically probes and detects which of several LMI standards
1058is in use at the exchange.
1059.Pp
1060.It TTY
1061This node is also a line discipline. It simply converts between mbuf
1062frames and sequential serial data, allowing a tty to appear as a netgraph
1063node. It has a programmable
1064.Dq hotkey
1065character.
1066.Pp
1067.It ASYNC
1068This node encapsulates and de-encapsulates asynchronous frames
1069according to RFC 1662. This is used in conjunction with the TTY node
1070type for supporting PPP links over asynchronous serial lines.
1071.Pp
1072.It INTERFACE
1073This node is also a system networking interface. It has hooks representing
1074each protocol family (IP, AppleTalk, IPX, etc.) and appears in the output of
1075.Xr ifconfig 8 .
1076The interfaces are named
1077.Em ng0 ,
1078.Em ng1 ,
1079etc.
1080.It ONE2MANY
1081This node implements a simple round-robin multiplexer. It can be used
1082for example to make several LAN ports act together to get a higher speed
1083link between two machines.
1084.It Various PPP related nodes.
1085There is a full multilink PPP implementation that runs in Netgraph.
1086The
1087.Em Mpd
1088port can use these modules to make a very low latency high
1089capacity ppp system. It also supports
1090.Em PPTP
1091vpns using the
1092.Em PPTP
1093node.
1094.It PPPOE
1095A server and client side implememtation of PPPoE. Used in conjunction with
1096either
1097.Xr ppp 8
1098or the
1099.Em mpd port.
1100.It BRIDGE
1101This node, togther with the ethernet nodes allows a very flexible
1102bridging system to be implemented.
1103.It KSOCKET
1104This intriguing node looks like a socket to the system but diverts
1105all data to and from the netgraph system for further processing. This allows
1106such things as UDP tunnels to be almost trivially implemented from the
1107command line.
1108
1109.El
1110.Pp
1111Refer to the section at the end of this man page for more nodes types.
1112.Sh NOTES
1113Whether a named node exists can be checked by trying to send a control message
1114to it (e.g.,
1115.Dv NGM_NODEINFO
1116).
1117If it does not exist,
1118.Er ENOENT
1119will be returned.
1120.Pp
1121All data messages are mbuf chains with the M_PKTHDR flag set.
1122.Pp
1123Nodes are responsible for freeing what they allocate.
1124There are three exceptions:
1125.Bl -tag -width xxxx
1126.It 1
1127Mbufs sent across a data link are never to be freed by the sender,
1128unless it is returned from the recipient.
1129.It 2
1130Any meta-data information traveling with the data has the same restriction.
1131It might be freed by any node the data passes through, and a
1132.Dv NULL
1133passed onwards, but the caller will never free it.
1134Two macros
1135.Fn NG_FREE_META "meta"
1136and
1137.Fn NG_FREE_DATA "m" "meta"
1138should be used if possible to free data and meta data (see
1139.Pa netgraph.h ) .
1140.It 3
1141Messages sent using
1142.Fn ng_send_message
1143are freed by the recipient. As in the case above, the addresses
1144associated with the message are freed by whatever allocated them so the
1145recipient should copy them if it wants to keep that information.
1146.El
1147.Sh FILES
1148.Bl -tag -width xxxxx -compact
1149.It Pa /sys/netgraph/netgraph.h
1150Definitions for use solely within the kernel by
1151.Nm
1152nodes.
1153.It Pa /sys/netgraph/ng_message.h
1154Definitions needed by any file that needs to deal with
1155.Nm
1156messages.
1157.It Pa /sys/netgraph/ng_socket.h
1158Definitions needed to use
1159.Nm
1160socket type nodes.
1161.It Pa /sys/netgraph/ng_{type}.h
1162Definitions needed to use
1163.Nm
1164{type}
1165nodes, including the type cookie definition.
1166.It Pa /modules/netgraph.ko
1167Netgraph subsystem loadable KLD module.
1168.It Pa /modules/ng_{type}.ko
1169Loadable KLD module for node type {type}.
1170.El
1171.Sh USER MODE SUPPORT
1172There is a library for supporting user-mode programs that wish
1173to interact with the netgraph system. See
1174.Xr netgraph 3
1175for details.
1176.Pp
1177Two user-mode support programs,
1178.Xr ngctl 8
1179and
1180.Xr nghook 8 ,
1181are available to assist manual configuration and debugging.
1182.Pp
1183There are a few useful techniques for debugging new node types.
1184First, implementing new node types in user-mode first
1185makes debugging easier.
1186The
1187.Em tee
1188node type is also useful for debugging, especially in conjunction with
1189.Xr ngctl 8
1190and
1191.Xr nghook 8 .
1192.Pp
1193Also look in /usr/share/examples/netgraph for solutions to several
1194common networking problems, solved using
1195.Nm .
1196.Sh SEE ALSO
1197.Xr socket 2 ,
1198.Xr netgraph 3 ,
1199.Xr ng_async 4 ,
1200.Xr ng_bridge 4 ,
1201.Xr ng_bpf 4 ,
1202.Xr ng_cisco 4 ,
1203.Xr ng_ether 4 ,
1204.Xr ng_echo 4 ,
1205.Xr ng_ether 4 ,
1206.Xr ng_frame_relay 4 ,
1207.Xr ng_hole 4 ,
1208.Xr ng_iface 4 ,
1209.Xr ng_ksocket 4 ,
1210.Xr ng_lmi 4 ,
1211.Xr ng_mppc 4 ,
1212.Xr ng_ppp 4 ,
1213.Xr ng_pppoe 4 ,
1214.Xr ng_pptpgre 4 ,
1215.Xr ng_rfc1490 4 ,
1216.Xr ng_socket 4 ,
1217.Xr ng_tee 4 ,
1218.Xr ng_tty 4 ,
1219.Xr ng_UI 4 ,
1220.Xr ng_vjc 4 ,
1221.Xr ng_{type} 4 ,
1222.Xr ngctl 8 ,
1223.Xr nghook 8
1224.Sh HISTORY
1225The
1226.Nm
1227system was designed and first implemented at Whistle Communications, Inc.
1228in a version of
1229.Fx 2.2
1230customized for the Whistle InterJet.
1231It first made its debut in the main tree in
1232.Fx 3.4 .
1233.Sh AUTHORS
1234.An -nosplit
1235.An Julian Elischer Aq julian@FreeBSD.org ,
1236with contributions by
1237.An Archie Cobbs Aq archie@FreeBSD.org .