Difference between revisions of "SPP Command Interface"

From ARL Wiki
Jump to navigationJump to search
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
[[Category:The SPP]]
 
[[Category:The SPP]]
  
>>>>> WARNING:  NOT READY FOR PUBLIC CONSUMPTION <<<<<
+
== System Resource Manager (SRM) and Resource Manager Proxy (RMP) ==
  
== System Resource Manager (SRM) ==
+
[[Image:SPP_Control_Software.png|thumb|right|400px|border|Major Control Software Modules]]
  
The SRM is the top level controller for the SPP and provides several services.
+
The page [[SPP Control Software]] describes the System Resource Manager (SRM) that runs on the SPP's Control Processor and the Resource Manager Proxy (RMP) that runs on each SPP's GPE, and it describes the programming API provided by the RMP.
These include acquiring slice definitions from SPP-PLC, instantiating slice definitions, reserving and assigning resources to slices and coordinating the initialization of the whole system.
+
This page describes the accompanying command line interface which
The SRM implements functions provided by the Node Manager on a conventional PlanetLab node, but must provide this functionality in the context of a system with a more complex internal structure, and a richer set of resources.
+
allows users to reserve and configure resources interactively or through a shell script.
 +
In brief, the SRM coordinates the use of various SPP resources while the RMP provides an interface through which user slices can request and configure resources.
 +
The RMP, in turn, may communicate with the Substrate Control Daemons (SCD) in the Line Card and the NPE to configure packet pathways.
  
The SRM polls SPP-PLC periodically to obtain new slice definitions.
 
When a new slice is detected, the SRM selects one of the two GPEs on which to instantiate the slice.
 
Slice instantiation involves creating a vServer on the selected slice, initializing it and configuring a login so that users can access their assigned vServer.
 
 
Once assigned to a vServer, a user can run programs that send and receive packets on the external interfaces.
 
Outgoing connections are subjected to port number translation at the Line Cards, as described in Section 4.
 
Users may also request the use of specific external port numbers in order to run servers that listen on specific ports.
 
 
User requests are made through an interface provided by the RMP on the user’s assigned GPE.
 
User requests are made through an interface provided by the RMP on the user’s assigned GPE.
 
The RMP forwards these requests to the SRM which manages all system level resources, including external port numbers, physical interface bandwidth and NPE resources.
 
The RMP forwards these requests to the SRM which manages all system level resources, including external port numbers, physical interface bandwidth and NPE resources.
 +
For example, a user can request the use of a specific external port number in order to run a server that listens on that port at the user's GPE.
 +
Or a user may want to reserve resources in advance (e.g., port bandwidth and NPE fastpath).
  
== Resource Manager Proxy (RMP) ==
+
That command line interface is currently provided through three programs:  ''scfg'' (slice configuration), ''ip_fpc'' (IPv4 filter configuration) and ''ip_fpd'' (IPv4 fastpath daemon).
 +
(<font color=red>'''Note:''' These names are likely to change in the near future.</font>
 +
The ''scfg'' program applies to any fastpath while ''ip_fpc'' and ''ip_fpd'' are specific to the IPv4 code option.
  
The RMP provides an API used by applications running in vServers.
+
{| align=center border=1 cellspacing=0 cellpadding=3
The API allows users to reserve resources in advance (such as external port bandwidth and NPE fastpaths), to acquire those resources when a reservation period starts and configure the resources as needed.
+
! Program            || Functionality
The RMP is implemented as a daemon that runs in the root context and is accessed through a
+
|-
set of library routines.
+
| ''scfg''        || Interface information, endpoint management, <br>fastpath management, queue management, <br>reservation management
A command line interface is also provided so that users can reserve and configure resources interactively, or through a shell script.
+
|-
The command line interface converts the given commands to API calls.
+
| ''ip_fpc''      || IPv4 filter management
 +
|-
 +
| ''ip_fpd'' || IPv4 fastpath configuration, LD and EX traffic handling
 +
|}
  
The main API calls are listed below in topical sub-sections, along with a brief description of how each call is used.
+
There will be other fastpath-specific programs as other code options get developed.
We use a representation that attempts to informally describe the interface semantics.
+
It is worth noting that all of these programs implement their functionallity by calling RMP library functions.
More precise descriptions are given in the reference manual.
+
The fact that these programs exist as separate programs is historic.
We use an abstract interface syntax that has the form
+
For example, one could have a single ''ipv4_control'' program that included all of the functionality of these programs.
''R &larr; F(A1,…,An)''” where F is the function name, ''Ai'' is the i-th argument, and ''R'' is the return value.
+
Furthermore, a new code option such as ''vnet'' would have a ''vnet_control'' program that included the functionality in the ''scfg''  program above and vnet-specific functionality for filters and handling LD and EX traffic.
Mnemonic names are used to convey usage while data type modifiers have been omitted.
+
(Note:  The programs ''ip_fpc'' and ''ip_fpd'' were called ''fltr'' and ''create_fp'' respectively in [[The GEC4 Demo]].)
The following abbreviations and mnemonics are used in argument names and descriptions:
+
 
 +
== Command Interface Syntax ==
 +
 
 +
The ''scfg'' command uses GNU-style ''long options'' (e.g., ''--cmd get_endpoint'', ''--ipaddr 10.1.2.3'').
 +
And although they both have a few arguments that control their configuration (e.g., message log noise volume), this page only describes the user commands.
 +
In the descriptions below, the following metasymbols are used in the syntactic descriptions:
  
 
{| align=center border=0 cellspacing=5 cellpadding=2
 
{| align=center border=0 cellspacing=5 cellpadding=2
 
|-
 
|-
| FP   || FastPath
+
| '''A'''   || IP address (e.g., 192.168.2.32)      || || '''K'''    || The number of Kilobits per second
 
|-
 
|-
|EP    || EndPoint (a logical interface used by a slice and <br> mapped to a physical interface)
+
| '''D'''    || Date in YYYMMDDHHMMSS format          || ||            ||  
 
|-
 
|-
|LC    || LineCard
+
| '''F'''    || Filter identifier                    || || '''N'''    || Number (non-negative)
 
|-
 
|-
|BW    || BandWidth
+
| '''FN'''  || Filename                              || || '''P'''    || Protocol number (e.g., 6 (TCP), 17 (UDP))
 
|-
 
|-
|DB    || DataBase
+
| '''I'''    || An identifier or index                || || '''Q'''    || Queue identifier
 
|-
 
|-
|Xdescr || X description where X is Q, EP or FP for Queue, <br> EndPoint, or FastPath (e.g., FPdescr)
+
| '''['''x''']'''|| Optional argument x              || || '''S'''    || Statistics index
 
|-
 
|-
|Xid   || X identifier where X is F, FP, MI, Q or S for Filter, <br> FastPath, MetaInterface, Queue, or Slice (e.g., FPid)
+
| '''{''' x '''xxx''' y '''}''' || Alternative arguments x and y || || '''X'''   || Hexadecimal mask or 0
 
|}
 
|}
 +
 +
When a command description needs to use multiple instances of a metasymbol, distinction between instances is indicated by following the metasymbol insances by one or more distinguishing symbols (e.g., quote, integer, lowercase letters) (e.g., N and N' or N1, N2 and N3).
 +
 +
== Command Interface ==
 +
 +
The following description of the command-line interface is meant as a quick reference.
 +
If you are looking for a tutorial format, consider [[The SPP Tutorial]] page which will contain links to [[The Hello GPE World Tutorial]] and [[Using the IPv4 Code Option]].
 +
 +
The command-line interface is provided through three programs:  ''scfg'' (slice configuration), ''ip_fpc'' (IPv4 fastpath filter configuration) and ''ip_fpd'' (IPv4 fastpath daemon).
 +
The ''scfg'' program applies to any fastpath while ''ip_fpc'' and ''ip_fpd'' are specific to the IPv4 code option.
 +
The primary function of the IPv4 daemon ''ip_fpd'' is to intialize the IPv4
 +
code option and handle local delivery and error packets for that code option.
 +
The program ''ip_fpc'' manages filters for the IPv4 code option.
 +
All other functions that are generic to any fastpath (e.g., getting information about physical interfaces, allocating resources, making resource reservations, managing fastpath queues) are done by the ''scfg'' program.
  
 
=== Getting Interface Information ===
 
=== Getting Interface Information ===
  
 
+
* scfg --cmd get_ifaces
  client --cmd get_ifaces
 
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
 
Display all interfaces
 
Display all interfaces
 
         </ul>
 
         </ul>
  client --cmd get_ifpeer --ifn N
+
* scfg --cmd get_ifpeer --ifn <font color=red>N</font>
        <ul style="padding-left: 0.5in;">
 
Display the peer of interface num N
 
        </ul>
 
  client --cmd get_ifattrs --ifn N
 
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Display the attributes of interface num N
+
Display the peer of interface num <font color=red>N</font>
 
         </ul>
 
         </ul>
  client --cmd get_ifn --ipaddr A
+
* scfg --cmd get_ifattrs --ifn <font color=red>N</font>
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Display the interface associated with IP address A
+
Display the attributes of interface num <font color=red>N</font>
 
         </ul>
 
         </ul>
  client --cmd get_endpoint --fpid I --mii I'
+
* scfg --cmd get_ifn --ipaddr <font color=red>A</font>
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Display the endpoint associated with (fastpath I, meta-interface I')
+
Display the interface associated with IP address <font color=red>A</font>
 
         </ul>
 
         </ul>
  
=== Endpoint Management ===
+
=== Reservation Management ===
  
  client --cmd alloc_endpoint --bw K --ipaddr A --port N --proto P
+
* scfg --cmd get_resrvs
        <ul style="padding-left: 0.5in;">
 
Create the K Kbps capacity LC endpoint (A, N, P)
 
        </ul>
 
  client --cmd free_endpoint --ipaddr A --port N --proto P
 
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Remove the K Kbps capacity LC endpoint (A, N, P)
+
Display all slice reservations
 
         </ul>
 
         </ul>
 
+
* scfg --cmd get_resrv [--date <font color=red>D</font>]
=== Fastpath Management ===
 
 
 
  client --cmd alloc_udp_tunnel --fpid I --bw  K --ipaddr A --port N
 
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Add (A, N) as a local end of a UDP tunnel with capacity K Kbps to fastpath I
+
Display slice reservation for period covered by date <font color=red>D</font> (default: current).
 +
<br>Date format is YYYYMMDDHHMMSS.
 
         </ul>
 
         </ul>
  client --cmd free_udp_tunnel --fpid I --ipaddr A --port N
+
* scfg --cmd make_resrv --xfile <font color=red>FN</font>
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Free (A, N) as a local end of a UDP tunnel
+
Make a reservation described by XML file <font color=red>FN</font>
 
         </ul>
 
         </ul>
  client --cmd alloc_fastpath --copt C --firm K --soft 0 --fltrs N1 --qs N2 --stats N3 --buffs N4 --sram 4096 --dram 0
+
* scfg --cmd cancel_resrv [--date <font color=red>D</font>]
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Allocate fastpath resources for code option C:
+
Cancel slice reservation for period covered by date <font color=red>D</font> (default: current).
  K Kbps firm bandwidth, N1 filters, N2 queues, N3 stats indices, N4 buffers
+
<br>Date format is YYYYMMDDHHMMSS.
 
         </ul>
 
         </ul>
  client --cmd free_fastpath --fpid I
+
* scfg --cmd update_resrv --xfile <font color=red>FN</font>
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Free the resources for fastpath I
+
Update slice reservation described by XML file <font color=red>FN</font>
 
         </ul>
 
         </ul>
  
=== Queue Management ===
+
=== Setup/Free ===
 +
 
  
  client --cmd get_queue_params --fpid I --qid I'
+
* scfg --cmd claim_resources
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Get the threshold and bandwidth for (fastpath I, queue I')
+
Allocate slowpath (GPE) slice resources
 
         </ul>
 
         </ul>
  client --cmd get_queue_len --fpid I --qid I'
+
* scfg --cmd free_sp_resources
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Get the length (bytes) of (fastpath I, queue I')
+
Free slowpath (GPE) slice resources
 
         </ul>
 
         </ul>
  client --cmd bind_queue --fpid I --miid I' --qid_list_type 0 --qid_list Q1 --qid_list Q2 ...
+
* scfg --cmd free_fp_resources --fp <font color=red>I</font>
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Bind queues Q1, Q2, ... to (fastpath I, meta-interface I')
+
Free fastpath slice resources for fastpath <font color=red>I</font>
 
         </ul>
 
         </ul>
  client --cmd bind_queue --fpid I --miid I' --qid_list_type 1 --qid_list Q1 --qid_list Q2
+
* scfg --cmd setup_sp_endpoint --bw <font color=red>K</font> --ipaddr <font color=red>A</font> --port <font color=red>N</font> --proto <font color=red>P</font>
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Bind queues in the range Q1 to Q2-1 inclusive to (fastpath I, meta-interface I')
+
Create the <font color=red>K</font> Kbps capacity slowpath LC endpoint (<font color=red>A</font>, <font color=red>N</font>, <font color=red>P</font>)
 
         </ul>
 
         </ul>
  client --cmd set_queue_params --fpid I --qid I' --threshold B --bw K
+
* scfg --cmd free_sp_endpoint --ipaddr <font color=red>A</font> --port <font color=red>N</font> --proto <font color=red>P</font>
 
         <ul style="padding-left: 0.5in;">
 
         <ul style="padding-left: 0.5in;">
Set the threshold and bandwidth of (fastpath I, queue I') to B bytes and K Kbps
+
Remove the slowpath LC endpoint (<font color=red>A</font>, <font color=red>N</font>, <font color=red>P</font>)
 
         </ul>
 
         </ul>
 
+
* scfg --cmd setup_fp_tunnel --fp <font color=red>I</font> --bw  <font color=red>K</font> --ipaddr <font color=red>A</font> --port <font color=red>N</font>
<ul>
+
        <ul style="padding-left: 0.5in;">
ifList &larr; get_ifaces(ifList)
+
Add (<font color=red>A</font>, <font color=red>N</font>) as a local end of a fastpath (UDP) tunnel with capacity <font color=red>K</font> Kbps to fastpath <font color=red>I</font>. Return the fastpath interface identifier in ''scfg'''s exit code which can be captured in the shell variable ''$?''.
        <ul>
 
          Return a list of all physical interfaces of the SPP.
 
        Slices configure MIs using the information from this list.
 
        The returned list indicates for each physical interface the
 
        attributes of the interface; i.e., interface number,
 
        the interface type (Internet or peering), the IP address,
 
        the total bandwidth and the available bandwidth.
 
 
         </ul>
 
         </ul>
ifNum &larr; get_ifn(EPaddr)
+
* scfg --cmd free_fp_tunnel --fp <font color=red>I</font> --fpif <font color=red>I'</font>
         <ul>
+
         <ul style="padding-left: 0.5in;">
          Return the physical interface number of the EP.
+
Free fastpath interface (<font color=red>I</font>, <font color=red>I'</font>) where <font color=red>I</font> is the fastpath identifier and <font color=red>I'</font> is the fastpath interface identifier
        </ul>
 
ifAttributes &larr; get_ifattrs(ifNum,ifAttributes)
 
        <ul>
 
          Return the attributes of the physical interface.
 
 
         </ul>
 
         </ul>
IPaddr &larr; get_ifpeer(ifNum)
 
        <ul>
 
          Return the IP address of the physical interface.
 
        </ul>
 
</ul>
 
  
=== GPE Interface Bandwidth ===
+
=== Queue Management ===
  
<ul>
+
* scfg --cmd get_queue_params --fp <font color=red>I</font> --qid <font color=red>Q</font>
rmpCode &larr; resrv_pl_ifbw(ifNum,BWkbps)
+
         <ul style="padding-left: 0.5in;">
         <ul>
+
Get the threshold and bandwidth for (fastpath <font color=red>I</font>, queue <font color=red>Q</font>)
          Reserve bandwidth (Kbps) on the physical interface.
 
 
         </ul>
 
         </ul>
rmpCode &larr; free_pl_ifbw(ifNum,BWkbps)
+
* scfg --cmd get_queue_len --fp <font color=red>I</font> --qid <font color=red>Q</font>
         <ul>
+
         <ul style="padding-left: 0.5in;">
          Release bandwidth (Kbps) from the physical interface.
+
Get the length (bytes) of (fastpath <font color=red>I</font>, queue <font color=red>Q</font>)
 
         </ul>
 
         </ul>
</ul>
+
* scfg --cmd bind_queue --fp <font color=red>I</font> --fpif <font color=red>I'</font> --qid <font color=red>Q1,Q2,...</font>
 
+
         <ul style="padding-left: 0.5in;">
=== GPE Endpoints ===
+
Bind queues <font color=red>Q1</font>, <font color=red>Q2</font>, ... to (fastpath <font color=red>I</font>, fastpath interface <font color=red>I'</font>)
 
 
<ul>
 
EPdescr &larr; alloc_endpoint(EPdescr)
 
         <ul>
 
          Given an EP description, allocate a new EP, and return
 
        a reference to the EP.
 
        A filter is installed in the LC to direct matching traffic to the GPE.
 
        For TCP or UDP, you can select the port number or have the
 
        system give you one.
 
 
         </ul>
 
         </ul>
RMPcode &larr; free_endpoint(EPdescr)
+
* scfg --cmd bind_queue --fp <font color=red>I</font> --fpif <font color=red>I'</font> --qid <font color=red>Q1-Q2</font>
         <ul>
+
         <ul style="padding-left: 0.5in;">
          Free the endpoint, de-install the LC filter for the EP,
+
Bind queues in the range <font color=red>Q1</font> to <font color=red>Q2</font> inclusive to (fastpath <font color=red>I</font>, fastpath interface <font color=red>I'</font>)
        and return the status.
 
 
         </ul>
 
         </ul>
</ul>
+
* scfg --cmd set_queue_params --fp <font color=red>I</font> --qid <font color=red>Q</font> --threshold <font color=red>B</font> --bw <font color=red>K</font>
 
+
         <ul style="padding-left: 0.5in;">
=== FastPaths ===
+
Set the threshold and bandwidth of (fastpath <font color=red>I</font>, queue <font color=red>Q</font>) to <font color=red>B</font> bytes and <font color=red>K</font> Kbps
 
 
<ul>
 
FPdescr &larr; alloc_fastpath(codeOpt,bwSpec,resSpec,memSpec,FPdescr)
 
         <ul>
 
          Given specifications for the aggregate bandwidth,
 
        other resource (filters, queues, buffers and stats) and
 
        memory, allocate a new FP for the code option, and
 
        return a reference to the FP description.
 
 
         </ul>
 
         </ul>
free_fastpath(FPid)
 
        <ul>
 
        Free the resources of the FP.
 
        </ul>
 
</ul>
 
  
=== FastPath Bandwidth ===
+
=== IPv4 Filter Management ===
  
<ul>
+
Filter management for the IPv4 code option is done through the ''ip_fpc'' program.
RMPcode &larr; resrv_fpath_ifbw(FPid,ifNum,BWkbps)
+
The stylized syntax of the ''ip_fpc'' command is:
        <ul>
 
        Reserve bandwidth (Kbps) on a physical interface for a FP.
 
        </ul>
 
RMPcode &larr; free_fpath_ifbw(FPid,ifNum,BWkbps)
 
        <ul>
 
        Free the bandwidth (Kbps) of a FP from a physical interface,
 
        and return the status.
 
        </ul>
 
</ul>
 
  
=== FastPath MetaInterfaces ===
+
  ip_fpc --cmd write_fltr --fpid I --fid I'                                      # ID
 +
    [--res_ld]                                                                  # SPECIAL
 +
   
 +
    --key_type {0|1}  --key_rxmi M                                              # KEY
 +
    --key_saddr Asrc  --key_sport Nsrc  --key_daddr Adst  --key_dport Ndst    # KEY
 +
    --key_proto P                                                                # KEY
 +
   
 +
    --mask_saddr Asrc' --mask_sport Nsrc' --mask_daddr Adst' --mask_dport Ndst'  # MASK
 +
    --mask_flags P'XXX                                                          # MASK
 +
   
 +
    --txdaddr Anxt    --txdport Nnxt                                            # RESULT
 +
    --qid Q            --sindx S                                                # RESULT
  
<ul>
+
(CAVEAT: XXXXX)
MIid &larr; alloc_udp_tunnel(FPid,EPdescr )
 
        <ul>
 
        Given a UDP tunnel EP description allocate the EP for the FP,
 
        and return the MI identifier.
 
        </ul>
 
RMPcode &larr; free_udp_tunnel(FPid,MIid)
 
        <ul>
 
        Free the MI of a FP, and return the status.
 
        </ul>
 
EPdescr &larr; get_endpoint(FPid,MIid,EPdescr)
 
        <ul>
 
        Return the UDP tunnel EP description for a given MI of a FP.
 
        </ul>
 
</ul>
 
  
=== FastPath Queue Management ===
+
* ID
 +
** x
 +
* SPECIAL
 +
** x
 +
* KEY
 +
** x
 +
* MASK
 +
** x
 +
* RESULT
 +
** x
  
<ul>
+
XXXXX
RMPcode &larr; bind_queue(FPid,MIid,qidListType,qidList)
+
        <ul style="padding-left: 0.5in;">
         <ul>
+
        Install a filter for fastpath I with filter identifier F.
         Associate the listed queues to the MI of the FP, and
+
        <br>
        return the status.
+
        match (key-mask), peer, qid, stats
 +
        <br>
 +
        key_type
 +
         <br>
 +
         F is from the range 0 to n-3 where n is the number of filter resources requested for the fastpath.
 
         </ul>
 
         </ul>
Qdescr &larr; get_queue_params(FPid,Qid,Qdescr)
+
  ip_fpc --cmd update_result --fpid I --fid I' RESULT
         <ul>
+
         <ul style="padding-left: 0.5in;">
         Return the parameters (threshold, bandwidth) for the FP queue,
+
         Update the result part of filter (I, I') where I is the fastpath identifier and I' is the filter identifier.
        and return a description of the queue.
 
 
         </ul>
 
         </ul>
BWkbps &larr; set_queue_params(FPid,Qid,Qdescr)
+
  ip_fpc --cmd get_fltr_bykey --fpid I KEY
         <ul>
+
         <ul style="padding-left: 0.5in;">
         Set the queue parameters (threshold, bandwidth) for the FP queue,
+
         Display the entire filter (key, mask, result) from fastpath I matching KEY.
        and return the bandwidth of the queue.
 
 
         </ul>
 
         </ul>
Qlen &larr; get_queue_len(FPid,Qid,Qlen)
+
  ip_fpc --cmd get_fltr_byfid --fpid I --fid I'
         <ul>
+
         <ul style="padding-left: 0.5in;">
         Return the length of the FP queue.
+
         Display the filter from fastpath I with filter identifier I'.
 
         </ul>
 
         </ul>
</ul>
+
  ip_fpc --cmd lookup_fltr --fpid I KEY
 
+
        <ul style="padding-left: 0.5in;">
=== Fastpath Filter Management ===
+
         Display the result part of the filter from fastpath I matching KEY.
 
 
<ul>
 
rmpCode &larr; write_fltr(FPid,Fid,Fltr)
 
        <ul>
 
         Install a FP filter, and return the status.
 
 
         </ul>
 
         </ul>
rmpCode &larr; update_result(FPid,Fid,Fltr)
+
  ip_fpc --cmd rem_fltr_bykey --fpid I KEY
         <ul>
+
         <ul style="padding-left: 0.5in;">
        Modify the FP filter, and return the status.
 
 
         </ul>
 
         </ul>
Fltr &larr; get_fltr_byfid(FPid,Fid,Fltr)
+
  ip_fpc --cmd rem_fltr_byfid --fpid I --fid I'
        <ul>
+
         <ul style="padding-left: 0.5in;">
        Return the FP filter given the filter ID.
+
         Remove the filter (I, I') where I is the fastpath identifier and I' is the filter identifier.
        </ul>
 
Fltr &larr; get_fltr_bykey(FPid,key,Fltr)
 
        <ul>
 
        Return the FP filter that matches the key.
 
        </ul>
 
fltrResult &larr; lookup_fltr(FPid,key,Fltr)
 
        <ul>
 
        Return the result part of the FP filter that matches the key.
 
        </ul>
 
rmpCode &larr; rem_fltr_byfid(FPid,Fid)
 
        <ul>
 
        Remove the FP filter given the filter ID, and return the status.
 
        </ul>
 
rmpCode &larr; rem_fltr_bykey(FPid,key)
 
        <ul>
 
        Remove the highest priority FP filter that matches the key, and
 
        return the status.
 
         </ul>
 
</ul>
 
 
 
=== FastPath Stats Management ===
 
 
 
<ul>
 
statsRecord &larr; read_stats(FPid,statsId,flags,statsRecord)
 
        <ul>
 
        Return the FP stats record (counter group) for the stats ID.
 
        The flags argument selects which counters to return.
 
        You can select the byte or packet counter and
 
        whether the preQ or postQ counter
 
        </ul>
 
rmpCode &larr; clear_stats(FPid,statsId,flags)
 
        <ul>
 
        Reset the FP stats counters for the stats ID.
 
        The flags argument selects which counters to return.
 
        </ul>
 
statsHandle &larr; create_periodic(FPid,statsId,period, historySize,flags)
 
        <ul>
 
        Create a periodic stats read event for the stats ID with
 
        the given period and history size, and
 
        return a handle for the operation.
 
        The flags argument indicates the retrieval method:
 
        either push the stats data to a registered port, or
 
        have the VM pull the data using the get_periodic command.
 
        </ul>
 
rmpCode &larr; delete_periodic(FPid,statsHandle)
 
        <ul>
 
         Remove the periodic event, remove the callback state, and
 
        return the status.
 
        </ul>
 
rmpCode &larr; set_callback( FPid,statsHandle,ipPortNum)
 
        <ul>
 
        Setup the callback for a periodic stats push model that
 
        sends stats records to the IP port number, and return the status.
 
        </ul>
 
statsRecord &larr; get_periodic(FPid,statsHandle,statsRecord)
 
        <ul>
 
        Return the stats record associated with the stats handle.
 
        </ul>
 
</ul>
 
=== FastPath Memory ===
 
 
 
Each code option is provided with a block of SRAM.
 
A slice can read/write to any location in this block.
 
A code option may elect to provide library functions to manipulate control structures within this block.
 
The valBuf argument to the read/write functions is a structure that includes the number of bytes in the buffer and the buffer itself.
 
 
 
<ul>
 
rmpCode &larr; mem_write(FPid,offset,valBuf)
 
        <ul>
 
          Write data to the SRAM starting at offset within the FP block, and
 
        return the status.
 
        The valBuf argument is a structure that includes the
 
        number of bytes and the data.
 
        </ul>
 
valBuf &larr; mem_read(FPid,offset,nbytes,valBuf)
 
        <ul>
 
        Read bytes into the value buffer, and return a reference to
 
        the value buffer.
 
        </ul>
 
</ul>
 
=== Reservation Management ===
 
 
 
<ul>
 
rmpCode &larr; make_reservation(rsvRecord)
 
        <ul>
 
        Make a reservation, and return the status.
 
        </ul>
 
rmpCode &larr; update_reservation(rsvRecord)
 
        <ul>
 
        Update a reservation.
 
        </ul>
 
rmpCode &larr; cancel_reservation(date)
 
        <ul>
 
        Cancel the reservation that includes the specified date and time.
 
        </ul>
 
</ul>
 
 
 
== Substrate Control Daemons (SCD) ==
 
 
 
The SCDs run on the xScale processors of the Line Card and NPE.
 
They provide a messaging interface, through which other control software components can exercise control.
 
These include messages to access traffic counters, add/remove TCAM packet filters, configure queue parameters (including WDRR weights and discard thresholds), read/write specific memory locations used for control and status registers, etc.
 
These are described in more detail below.
 
All functions have a context ID (''contextID'') as an argument. A context ID of 0 indicates a privileged operation performed by the substrate.
 
Any other context ID indicates a user context and is either a fastpath ID or internal slice ID.
 
Many of the functions (e.g. ''write_fltr'') appear to be similar to ones in the RMP.
 
This is expected because the evaluation of an RMP operation must often be relayed to an SCD for evaluation but with one important difference.
 
The SCD has a substrate view of objects whereas the RMP provides a higher-level of abstraction.
 
The Line Card SCD allows the SRM to control various elements of the Line Card data path.
 
This includes the TCAM-resident packet filters (on both input and output), interface addressing and bandwidth, NAT filter table configuration and queueing parameters.
 
The NPE SCD allows the SRM and the RMP to control various elements of the NPE data path.
 
This includes fast path configuration data, per-slice packet filters resident in the TCAM and queueing parameters.
 
 
 
=== Control Table Initialization ===
 
 
 
There are several tables and control blocks used by the control software.
 
 
 
<ul>
 
set_sched_params(contextId,Sid,ifNum,BWkbpsMax,BWkbpsMin,valBuf)
 
        <ul>
 
          Set the interface number and bandwidth characteristics
 
        for a Scheduler in the Per Scheduler Parameters table.
 
        </ul>
 
set_encap_cb(contextId,Sid,srcIPaddr,dstMACaddr,valBuf)
 
        <ul>
 
          On the NPE, set the source IP Address and destination
 
        MAC Address associated with the specified scheduler.
 
        </ul>
 
set_sched_mac(contextId,Sid,dstMACaddr,srcMACaddr,valBuf)
 
        <ul>
 
          On the LC, set the destination and source MAC Addresses for
 
        the specified scheduler.
 
        </ul>
 
set_encap_gpe(contextId,FPid,GPEipAddr,NPEipAddr,valBuf)
 
        <ul>
 
          On the NPE, for a fast path, set the GPE IP Address and
 
        NPE IP Address to be used for communication between
 
        the GPE and NPE for local delivery and exceptions.
 
        </ul>
 
set_fpmi_bw(contextId,FPid,Sid,MIid,BWkbps,valBuf)
 
        <ul>
 
          On the NPE, for a particular fast path, set the bandwidth
 
        for a MI using a particular scheduler.
 
        </ul>
 
SCDcode &larr; set_src_hwaddr(contextId,MACaddr)
 
        <ul>
 
        On the NPE, set the NPE’s source MAC Address.
 
        </ul>
 
SCDcode &larr; set_iface_table(contextId,ifTable)
 
        <ul>
 
          On the NPE, initialize the RX Interface ID table.
 
        This table translates the receive destination address
 
        on a packet to a 4 bit index which will be used in the Lookup key.
 
 
         </ul>
 
         </ul>
</ul>
 
  
=== FastPath (NPE SCD Only) ===
+
=== IPv4 Fastpath Creation ===
  
set_fast_path(contextId,FPid,codeOpt,vlanID, num_queues,num_filters,num_buffers,num_stats, SRAM_offset,SRAM_size,DRAM_offset,DRAM_size,valBuf)
+
  ip_fpd --fpName FN --myIP A --myPort N >> ip_fpd.log &
        <ul>
 
        On the NPE, create a new fast path.
 
        </ul>
 
rem_fast_path(contextId,FPid,valBuf)
 
        <ul>
 
        On the NPE, remove a fast path.
 
        </ul>
 
SCDcode &larr; set_gpe_info(contextId,EXport,LDport,EXqid,LDqid)
 
        <ul>
 
        On the NPE, for a particular fast path, set the
 
        Local Delivery and Exception traffic port numbers and QIDs.
 
        </ul>
 
  
=== Memory ===
 
  
<ul>
+
xxx
write_sram(contextId,offset,valBuf)
 
        <ul>
 
        On the NPE, write to the SRAM block for a particular fast path.
 
        </ul>
 
read_sram(contextId,offset,valBuf,count)
 
        <ul>
 
        On the NPE, read from the SRAM block for a particular fast path.
 
        </ul>
 
</ul>
 
  
=== Queue Management ===
+
 
  
<ul>
+
== Examples ==
SCDcode &larr; bind_queue(contextId,MIid,qidListType,qidVector)
 
        <ul>
 
        Associate the listed queues to the context’s MI, and return the status.
 
        </ul>
 
BWkbps &larr; set_queue_params(contextId,Qid,threshhold,BWkbps)
 
        <ul>
 
          Set the context’s queue parameters (threshold, bandwidth)
 
        for the queue, and return the bandwidth of the queue.
 
        </ul>
 
get_queue_params(contextId,Qid,threshhold,BWkbps)
 
        <ul>
 
          Return the context’s parameters (threshold, bandwidth)
 
        for the queue through the threshold and BWkbps parameters,
 
        and return a description of the queue.
 
        </ul>
 
get_queue_len(contextId,Qid,pktCnt,byteCnt)
 
        <ul>
 
          Return the length of the context’s queue through the pktCnt
 
        and byteCnt parameters.
 
        </ul>
 
set_queue_sched(contextId,Qid,Sid,valBuf)
 
        <ul>
 
        Associate a specified queue with the specified scheduler.
 
        </ul>
 
</ul>
 
 
 
=== NPE Filter Management ===
 
  
<ul>
+
xxx
SCDcode &larr; npe_write_fltr(contextId,Fid,substrateFltr)
 
        <ul>
 
        Install a context’s substrate (generic) filter with filter ID.
 
        </ul>
 
SCDcode &larr; npe_update_result(contextId,Fid,result)
 
        <ul>
 
          Modify the result part of a context’s substrate (generic)
 
          filter with filter ID.
 
        </ul>
 
substrateFltr &larr; npe_get_fltr_by_key(contextId,key,substrateFltr)
 
        <ul>
 
        Return the context’s substrate (generic) filter that matches the key.
 
        </ul>
 
substrateFltr &larr; npe_get_fltr_by_fid(contextId,Fid,substrateFltr)
 
        <ul>
 
        Return the context’s substrate filter given the filter ID.
 
        </ul>
 
substrateResult &larr; npe_lookup_fltr(contextId,key,substrateResult)
 
        <ul>
 
          Return the result part of the context’s substrate (generic) filter
 
        that matches the key.
 
        </ul>
 
SCDcode &larr; npe_rem_fltr_by_key(contextId,substrateKey)
 
        <ul>
 
          Remove the context’s highest priority substrate filter
 
        that matches the key, and return the status.
 
        </ul>
 
SCDcode &larr; npe_rem_fltr_by_fid(contextId,Fid)
 
        <ul>
 
          Remove the context’s substrate filter given the filter ID,
 
        and return the status.
 
        </ul>
 
</ul>
 
=== Line Card Filter Management ===
 
 
 
There are two Line Card filter databases: ingress and egress.
 
Ingress filters are used to determine which SPP component (e.g., NPE, GPE) should handle incoming packets.
 
Egress filters are used to determine which output interface to send outgoing packets.
 
The database ID (DBid) indicates the database to be used.
 
 
 
<ul>
 
write_fltr( contextId,DBid,Fid,key,mask,result,valBuf)
 
        <ul>
 
        Install a context’s LC filter (key, mask, result) in the given database.
 
        </ul>
 
update_result(contextId,DBid,Fid,result)
 
        <ul>
 
        Update a context’s LC filter result in the specified database.
 
        </ul>
 
get_fltr_by_key(contextId,DBid,key,mask,result,keyLen,resultLen)
 
        <ul>
 
        Given the key, retrieve a filter from the specified database.
 
        </ul>
 
get_fltr_by_fid(contextId,DBid,Fid,key,mask,result,keyLen, resultLen)
 
        <ul>
 
        Given the filter id, retrieve a filter from the specified database.
 
        </ul>
 
lookup_fltr(contextId,DBid,key,result,resultLen)
 
        <ul>
 
        Given the key, retrieve the filter result from the specified database.
 
        </ul>
 
rem_fltr_by_key(contextId,DBid,key,valBuf)
 
        <ul>
 
        Given the key, remove the filter from the specified database.
 
        </ul>
 
rem_fltr_by_fid(contextId,DBid,Fid,valBuf)
 
        <ul>
 
        Given the filter id, remove the filter from a specified database.
 
        </ul>
 
</ul>
 
 
 
=== Statistics Management ===
 
 
 
<ul>
 
statsRecord &larr; read_stats(contextId,statsId,flags,statsRecord)
 
        <ul>
 
          Return the context’s stats record (counter group) for the stats ID.
 
        The flags argument selects which counters to return.
 
        You can select the byte or packet counter and
 
        whether the preQ or postQ counter.
 
        </ul>
 
SCDcode &larr; clear_stats(contextId,statsId,flags)
 
        <ul>
 
          Reset the context’s stats counters for the stats ID, and
 
        return the status.
 
        The flags argument selects which counters to return.
 
        </ul>
 
statsHandle &larr; create_periodic(contextId,statsId,period,count, flags)
 
        <ul>
 
          Create a periodic stats read event for the stats ID of
 
        the context with the given period and history size, and
 
        return a handle for the operation.
 
        The flags argument indicates the retrieval method:
 
        either push the stats data to a registered port, or have
 
        the VM pull the data using the get_periodic command.
 
        </ul>
 
SCDcode &larr; del_periodic(contextId,statsHandle)
 
        <ul>
 
          Remove the context’s periodic event, remove the callback state,
 
        and return the status.
 
        </ul>
 
SCDcode &larr; set_callback(contextId,statsHandle,UDPport)
 
        <ul>
 
          Setup the context’s callback for a periodic stats push
 
        model that sends stats records to the UDP port number, and
 
        return the status.
 
        </ul>
 
statsRecordVector &larr; get_periodic(contextId,statsHandle, statsRecordVector)
 
        <ul>
 
        Return the context’s stats record associated with the stats handle.
 
        </ul>
 
</ul>
 
 
 
=== MicroEngine Management ===
 
 
 
<ul>
 
start_mes(contextId,valBuf)
 
        <ul>
 
        Start the MicroEngines on an NPU.
 
        </ul>
 
stop_mes(contextId,valBuf)
 
        <ul>
 
        Stop the MicroEngines on an NPU.
 
        </ul>
 
</ul>
 
 
 
=== NAT ===
 
 
 
<ul>
 
nat_filters(contextId,ingressStartFid,ingressEndFid, egressStartFid, egressEndFid)
 
        <ul>
 
          On the LC, initialize the NAT filter tables.
 
        This sets aside a block of the TCAM for the Ingress NAT filters
 
        and a block of the TCAM for the Egress NAT filters.
 
        </ul>
 
</ul>
 
=== MetaInterface Management ===
 
 
 
<ul>
 
SCDcode &larr; create_mi(contextId,FPid,MIid,Sid)
 
        <ul>
 
        On the NPE,cCreate a new meta-interface for a fast path.
 
        </ul>
 
SCDcode &larr; delete_mi(contextId,FPid,MIid)
 
        <ul>
 
          On the NPE, delete the specified meta-interface for
 
        the specified fast path.
 
        </ul>
 
SCDcode &larr; set_mi_bw(contextId,FPid,MIid,BWkbps)
 
        <ul>
 
          On the NPE, for the specified fast path, set the bandwidth
 
        for a meta-interface.
 
        </ul>
 
SCDcode &larr; bind_queue_sched(contextId,Qid,Sid)
 
        <ul>
 
        On the NPE, bind a queue to a scheduler.
 
        </ul>
 
SCDcode &larr; unbind_queue_sched(contextId,Qid)
 
        <ul>
 
          On the NPE, unbind a queue from a scheduler and
 
        release its bandwidth on that scheduler.
 
        </ul>
 
SCDcode &larr; unbind_queue(contextId,Qid)
 
        <ul>
 
          On the NPE, unbind a queue from a meta-interface and
 
        release its bandwidth on that meta-interface.
 
        </ul>
 
</ul>
 

Latest revision as of 16:02, 2 April 2010


System Resource Manager (SRM) and Resource Manager Proxy (RMP)

Major Control Software Modules

The page SPP Control Software describes the System Resource Manager (SRM) that runs on the SPP's Control Processor and the Resource Manager Proxy (RMP) that runs on each SPP's GPE, and it describes the programming API provided by the RMP. This page describes the accompanying command line interface which allows users to reserve and configure resources interactively or through a shell script. In brief, the SRM coordinates the use of various SPP resources while the RMP provides an interface through which user slices can request and configure resources. The RMP, in turn, may communicate with the Substrate Control Daemons (SCD) in the Line Card and the NPE to configure packet pathways.

User requests are made through an interface provided by the RMP on the user’s assigned GPE. The RMP forwards these requests to the SRM which manages all system level resources, including external port numbers, physical interface bandwidth and NPE resources. For example, a user can request the use of a specific external port number in order to run a server that listens on that port at the user's GPE. Or a user may want to reserve resources in advance (e.g., port bandwidth and NPE fastpath).

That command line interface is currently provided through three programs: scfg (slice configuration), ip_fpc (IPv4 filter configuration) and ip_fpd (IPv4 fastpath daemon). (Note: These names are likely to change in the near future. The scfg program applies to any fastpath while ip_fpc and ip_fpd are specific to the IPv4 code option.

Program Functionality
scfg Interface information, endpoint management,
fastpath management, queue management,
reservation management
ip_fpc IPv4 filter management
ip_fpd IPv4 fastpath configuration, LD and EX traffic handling

There will be other fastpath-specific programs as other code options get developed. It is worth noting that all of these programs implement their functionallity by calling RMP library functions. The fact that these programs exist as separate programs is historic. For example, one could have a single ipv4_control program that included all of the functionality of these programs. Furthermore, a new code option such as vnet would have a vnet_control program that included the functionality in the scfg program above and vnet-specific functionality for filters and handling LD and EX traffic. (Note: The programs ip_fpc and ip_fpd were called fltr and create_fp respectively in The GEC4 Demo.)

Command Interface Syntax

The scfg command uses GNU-style long options (e.g., --cmd get_endpoint, --ipaddr 10.1.2.3). And although they both have a few arguments that control their configuration (e.g., message log noise volume), this page only describes the user commands. In the descriptions below, the following metasymbols are used in the syntactic descriptions:

A IP address (e.g., 192.168.2.32) K The number of Kilobits per second
D Date in YYYMMDDHHMMSS format
F Filter identifier N Number (non-negative)
FN Filename P Protocol number (e.g., 6 (TCP), 17 (UDP))
I An identifier or index Q Queue identifier
[x] Optional argument x S Statistics index
{ x xxx y } Alternative arguments x and y X Hexadecimal mask or 0

When a command description needs to use multiple instances of a metasymbol, distinction between instances is indicated by following the metasymbol insances by one or more distinguishing symbols (e.g., quote, integer, lowercase letters) (e.g., N and N' or N1, N2 and N3).

Command Interface

The following description of the command-line interface is meant as a quick reference. If you are looking for a tutorial format, consider The SPP Tutorial page which will contain links to The Hello GPE World Tutorial and Using the IPv4 Code Option.

The command-line interface is provided through three programs: scfg (slice configuration), ip_fpc (IPv4 fastpath filter configuration) and ip_fpd (IPv4 fastpath daemon). The scfg program applies to any fastpath while ip_fpc and ip_fpd are specific to the IPv4 code option. The primary function of the IPv4 daemon ip_fpd is to intialize the IPv4 code option and handle local delivery and error packets for that code option. The program ip_fpc manages filters for the IPv4 code option. All other functions that are generic to any fastpath (e.g., getting information about physical interfaces, allocating resources, making resource reservations, managing fastpath queues) are done by the scfg program.

Getting Interface Information

  • scfg --cmd get_ifaces
    Display all interfaces
  • scfg --cmd get_ifpeer --ifn N
    Display the peer of interface num N
  • scfg --cmd get_ifattrs --ifn N
    Display the attributes of interface num N
  • scfg --cmd get_ifn --ipaddr A
    Display the interface associated with IP address A

Reservation Management

  • scfg --cmd get_resrvs
    Display all slice reservations
  • scfg --cmd get_resrv [--date D]
    Display slice reservation for period covered by date D (default: current).
    Date format is YYYYMMDDHHMMSS.
  • scfg --cmd make_resrv --xfile FN
    Make a reservation described by XML file FN
  • scfg --cmd cancel_resrv [--date D]
    Cancel slice reservation for period covered by date D (default: current).
    Date format is YYYYMMDDHHMMSS.
  • scfg --cmd update_resrv --xfile FN
    Update slice reservation described by XML file FN

Setup/Free

  • scfg --cmd claim_resources
    Allocate slowpath (GPE) slice resources
  • scfg --cmd free_sp_resources
    Free slowpath (GPE) slice resources
  • scfg --cmd free_fp_resources --fp I
    Free fastpath slice resources for fastpath I
  • scfg --cmd setup_sp_endpoint --bw K --ipaddr A --port N --proto P
    Create the K Kbps capacity slowpath LC endpoint (A, N, P)
  • scfg --cmd free_sp_endpoint --ipaddr A --port N --proto P
    Remove the slowpath LC endpoint (A, N, P)
  • scfg --cmd setup_fp_tunnel --fp I --bw K --ipaddr A --port N
    Add (A, N) as a local end of a fastpath (UDP) tunnel with capacity K Kbps to fastpath I. Return the fastpath interface identifier in scfg's exit code which can be captured in the shell variable $?.
  • scfg --cmd free_fp_tunnel --fp I --fpif I'
    Free fastpath interface (I, I') where I is the fastpath identifier and I' is the fastpath interface identifier

Queue Management

  • scfg --cmd get_queue_params --fp I --qid Q
    Get the threshold and bandwidth for (fastpath I, queue Q)
  • scfg --cmd get_queue_len --fp I --qid Q
    Get the length (bytes) of (fastpath I, queue Q)
  • scfg --cmd bind_queue --fp I --fpif I' --qid Q1,Q2,...
    Bind queues Q1, Q2, ... to (fastpath I, fastpath interface I')
  • scfg --cmd bind_queue --fp I --fpif I' --qid Q1-Q2
    Bind queues in the range Q1 to Q2 inclusive to (fastpath I, fastpath interface I')
  • scfg --cmd set_queue_params --fp I --qid Q --threshold B --bw K
    Set the threshold and bandwidth of (fastpath I, queue Q) to B bytes and K Kbps

IPv4 Filter Management

Filter management for the IPv4 code option is done through the ip_fpc program. The stylized syntax of the ip_fpc command is:

 ip_fpc --cmd write_fltr --fpid I --fid I'                                      # ID
   [--res_ld]                                                                   # SPECIAL
   
   --key_type {0|1}   --key_rxmi M                                              # KEY
   --key_saddr Asrc   --key_sport Nsrc   --key_daddr Adst   --key_dport Ndst    # KEY
   --key_proto P                                                                # KEY
   
   --mask_saddr Asrc' --mask_sport Nsrc' --mask_daddr Adst' --mask_dport Ndst'  # MASK
   --mask_flags P'XXX                                                           # MASK
   
   --txdaddr Anxt     --txdport Nnxt                                            # RESULT
   --qid Q            --sindx S                                                 # RESULT

(CAVEAT: XXXXX)

  • ID
    • x
  • SPECIAL
    • x
  • KEY
    • x
  • MASK
    • x
  • RESULT
    • x

XXXXX

    Install a filter for fastpath I with filter identifier F.
    match (key-mask), peer, qid, stats
    key_type
    F is from the range 0 to n-3 where n is the number of filter resources requested for the fastpath.
 ip_fpc --cmd update_result --fpid I --fid I' RESULT
    Update the result part of filter (I, I') where I is the fastpath identifier and I' is the filter identifier.
 ip_fpc --cmd get_fltr_bykey --fpid I KEY
    Display the entire filter (key, mask, result) from fastpath I matching KEY.
 ip_fpc --cmd get_fltr_byfid --fpid I --fid I'
    Display the filter from fastpath I with filter identifier I'.
 ip_fpc --cmd lookup_fltr --fpid I KEY
    Display the result part of the filter from fastpath I matching KEY.
 ip_fpc --cmd rem_fltr_bykey --fpid I KEY
 ip_fpc --cmd rem_fltr_byfid --fpid I --fid I'
    Remove the filter (I, I') where I is the fastpath identifier and I' is the filter identifier.

IPv4 Fastpath Creation

 ip_fpd --fpName FN --myIP A --myPort N >> ip_fpd.log &


xxx


Examples

xxx