Difference between revisions of "SPP Command Interface"

From ARL Wiki
Jump to navigationJump to search
(New page: Category:The SPP >>>>> WARNING: NOT READY FOR PUBLIC CONSUMPTION <<<<< == System Resource Manager (SRM) == The SRM is the top level controller for the SPP and provides several serv...)
 
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
[[Category:The SPP]]
 
[[Category:The SPP]]
  
−
>>>>> WARNING:  NOT READY FOR PUBLIC CONSUMPTION <<<<<
+
== System Resource Manager (SRM) and Resource Manager Proxy (RMP) ==
  
−
== System Resource Manager (SRM) ==
+
[[Image:SPP_Control_Software.png|thumb|right|400px|border|Major Control Software Modules]]
  
−
The SRM is the top level controller for the SPP and provides several services.
+
The page [[SPP Control Software]] describes the System Resource Manager (SRM) that runs on the SPP's Control Processor and the Resource Manager Proxy (RMP) that runs on each SPP's GPE, and it describes the programming API provided by the RMP.
−
These include acquiring slice definitions from SPP-PLC, instantiating slice definitions, reserving and assigning resources to slices and coordinating the initialization of the whole system.
+
This page describes the accompanying command line interface which
−
The SRM implements functions provided by the Node Manager on a conventional PlanetLab node, but must provide this functionality in the context of a system with a more complex internal structure, and a richer set of resources.
+
allows users to reserve and configure resources interactively or through a shell script.
 +
In brief, the SRM coordinates the use of various SPP resources while the RMP provides an interface through which user slices can request and configure resources.
 +
The RMP, in turn, may communicate with the Substrate Control Daemons (SCD) in the Line Card and the NPE to configure packet pathways.
  
−
The SRM polls SPP-PLC periodically to obtain new slice definitions.
 
−
When a new slice is detected, the SRM selects one of the two GPEs on which to instantiate the slice.
 
−
Slice instantiation involves creating a vServer on the selected slice, initializing it and configuring a login so that users can access their assigned vServer.
 
−
 
−
Once assigned to a vServer, a user can run programs that send and receive packets on the external interfaces.
 
−
Outgoing connections are subjected to port number translation at the Line Cards, as described in Section 4.
 
−
Users may also request the use of specific external port numbers in order to run servers that listen on specific ports.
 
 
User requests are made through an interface provided by the RMP on the user’s assigned GPE.
 
User requests are made through an interface provided by the RMP on the user’s assigned GPE.
 
The RMP forwards these requests to the SRM which manages all system level resources, including external port numbers, physical interface bandwidth and NPE resources.
 
The RMP forwards these requests to the SRM which manages all system level resources, including external port numbers, physical interface bandwidth and NPE resources.
 +
For example, a user can request the use of a specific external port number in order to run a server that listens on that port at the user's GPE.
 +
Or a user may want to reserve resources in advance (e.g., port bandwidth and NPE fastpath).
  
−
== Resource Manager Proxy (RMP) ==
+
That command line interface is currently provided through three programs:  ''scfg'' (slice configuration), ''ip_fpc'' (IPv4 filter configuration) and ''ip_fpd'' (IPv4 fastpath daemon).
 +
(<font color=red>'''Note:''' These names are likely to change in the near future.</font>
 +
The ''scfg'' program applies to any fastpath while ''ip_fpc'' and ''ip_fpd'' are specific to the IPv4 code option.
 +
 
 +
{| align=center border=1 cellspacing=0 cellpadding=3
 +
! Program            || Functionality
 +
|-
 +
| ''scfg''        || Interface information, endpoint management, <br>fastpath management, queue management, <br>reservation management
 +
|-
 +
| ''ip_fpc''      || IPv4 filter management
 +
|-
 +
| ''ip_fpd'' || IPv4 fastpath configuration, LD and EX traffic handling
 +
|}
  
−
The RMP provides an API used by applications running in vServers.
+
There will be other fastpath-specific programs as other code options get developed.
−
The API allows users to reserve resources in advance (such as external port bandwidth and NPE fastpaths), to acquire those resources when a reservation period starts and configure the resources as needed.
+
It is worth noting that all of these programs implement their functionallity by calling RMP library functions.
−
The RMP is implemented as a daemon that runs in the root context and is accessed through a
+
The fact that these programs exist as separate programs is historic.
−
set of library routines.
+
For example, one could have a single ''ipv4_control'' program that included all of the functionality of these programs.
−
A command line interface is also provided so that users can reserve and configure resources interactively, or through a shell script.
+
Furthermore, a new code option such as ''vnet'' would have a ''vnet_control'' program that included the functionality in the ''scfg''  program above and vnet-specific functionality for filters and handling LD and EX traffic.
−
The command line interface converts the given commands to API calls.
+
(Note:  The programs ''ip_fpc'' and ''ip_fpd'' were called ''fltr'' and ''create_fp'' respectively in [[The GEC4 Demo]].)
  
−
The main API calls are listed below in topical sub-sections, along with a brief description of how each call is used.
+
== Command Interface Syntax ==
−
We use a representation that attempts to informally describe the interface semantics.
+
 
−
More precise descriptions are given in the reference manual.
+
The ''scfg'' command uses GNU-style ''long options'' (e.g., ''--cmd get_endpoint'', ''--ipaddr 10.1.2.3'').
−
We use an abstract interface syntax that has the form
+
And although they both have a few arguments that control their configuration (e.g., message log noise volume), this page only describes the user commands.
−
“''R &larr; F(A1,…,An)''” where F is the function name, ''Ai'' is the i-th argument, and ''R'' is the return value.
+
In the descriptions below, the following metasymbols are used in the syntactic descriptions:
−
Mnemonic names are used to convey usage while data type modifiers have been omitted.
 
−
The following abbreviations and mnemonics are used in argument names and descriptions:
 
  
 
{| align=center border=0 cellspacing=5 cellpadding=2
 
{| align=center border=0 cellspacing=5 cellpadding=2
 
|-
 
|-
−
| FP   || FastPath
+
| '''A'''   || IP address (e.g., 192.168.2.32)      || || '''K'''    || The number of Kilobits per second
 
|-
 
|-
−
|EP    || EndPoint (a logical interface used by a slice and <br> mapped to a physical interface)
+
| '''D'''    || Date in YYYMMDDHHMMSS format          || ||            ||  
 
|-
 
|-
−
|LC    || LineCard
+
| '''F'''    || Filter identifier                    || || '''N'''    || Number (non-negative)
 
|-
 
|-
−
|BW    || BandWidth
+
| '''FN'''  || Filename                              || || '''P'''    || Protocol number (e.g., 6 (TCP), 17 (UDP))
 
|-
 
|-
−
|DB    || DataBase
+
| '''I'''    || An identifier or index                || || '''Q'''    || Queue identifier
 
|-
 
|-
−
|Xdescr || X description where X is Q, EP or FP for Queue, <br> EndPoint, or FastPath (e.g., FPdescr)
+
| '''['''x''']'''|| Optional argument x              || || '''S'''    || Statistics index
 
|-
 
|-
−
|Xid   || X identifier where X is F, FP, MI, Q or S for Filter, <br> FastPath, MetaInterface, Queue, or Slice (e.g., FPid)
+
| '''{''' x '''xxx''' y '''}''' || Alternative arguments x and y || || '''X'''   || Hexadecimal mask or 0
 
|}
 
|}
  
−
=== Interfaces ===
+
When a command description needs to use multiple instances of a metasymbol, distinction between instances is indicated by following the metasymbol insances by one or more distinguishing symbols (e.g., quote, integer, lowercase letters) (e.g., N and N' or N1, N2 and N3).
  
−
<ul>
+
== Command Interface ==
−
ifList &larr; get_ifaces(ifList)
 
−
        <ul>
 
−
          Return a list of all physical interfaces of the SPP.
 
−
        Slices configure MIs using the information from this list.
 
−
        The returned list indicates for each physical interface the
 
−
        attributes of the interface; i.e., interface number,
 
−
        the interface type (Internet or peering), the IP address,
 
−
        the total bandwidth and the available bandwidth.
 
−
        </ul>
 
−
ifNum &larr; get_ifn(EPaddr)
 
−
        <ul>
 
−
          Return the physical interface number of the EP.
 
−
        </ul>
 
−
ifAttributes &larr; get_ifattrs(ifNum,ifAttributes)
 
−
        <ul>
 
−
          Return the attributes of the physical interface.
 
−
        </ul>
 
−
IPaddr &larr; get_ifpeer(ifNum)
 
−
        <ul>
 
−
          Return the IP address of the physical interface.
 
−
        </ul>
 
−
</ul>
 
  
−
=== GPE Interface Bandwidth ===
+
The following description of the command-line interface is meant as a quick reference.
 +
If you are looking for a tutorial format, consider [[The SPP Tutorial]] page which will contain links to [[The Hello GPE World Tutorial]] and [[Using the IPv4 Code Option]].
  
−
<ul>
+
The command-line interface is provided through three programs:  ''scfg'' (slice configuration), ''ip_fpc'' (IPv4 fastpath filter configuration) and ''ip_fpd'' (IPv4 fastpath daemon).
−
rmpCode &larr; resrv_pl_ifbw(ifNum,BWkbps)
+
The ''scfg'' program applies to any fastpath while ''ip_fpc'' and ''ip_fpd'' are specific to the IPv4 code option.
−
        <ul>
+
The primary function of the IPv4 daemon ''ip_fpd'' is to intialize the IPv4
−
          Reserve bandwidth (Kbps) on the physical interface.
+
code option and handle local delivery and error packets for that code option.
−
        </ul>
+
The program ''ip_fpc'' manages filters for the IPv4 code option.
−
rmpCode &larr; free_pl_ifbw(ifNum,BWkbps)
+
All other functions that are generic to any fastpath (e.g., getting information about physical interfaces, allocating resources, making resource reservations, managing fastpath queues) are done by the ''scfg'' program.
−
        <ul>
 
−
          Release bandwidth (Kbps) from the physical interface.
 
−
        </ul>
 
−
</ul>
 
  
−
=== GPE Endpoints ===
+
=== Getting Interface Information ===
  
−
<ul>
+
* scfg --cmd get_ifaces
−
EPdescr &larr; alloc_endpoint(EPdescr)
+
         <ul style="padding-left: 0.5in;">
−
         <ul>
+
Display all interfaces
−
          Given an EP description, allocate a new EP, and return
 
−
        a reference to the EP.
 
−
        A filter is installed in the LC to direct matching traffic to the GPE.
 
−
        For TCP or UDP, you can select the port number or have the
 
−
        system give you one.
 
 
         </ul>
 
         </ul>
−
RMPcode &larr; free_endpoint(EPdescr)
+
* scfg --cmd get_ifpeer --ifn <font color=red>N</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
          Free the endpoint, de-install the LC filter for the EP,
+
Display the peer of interface num <font color=red>N</font>
−
        and return the status.
 
 
         </ul>
 
         </ul>
−
</ul>
+
* scfg --cmd get_ifattrs --ifn <font color=red>N</font>
−
 
+
         <ul style="padding-left: 0.5in;">
−
=== FastPaths ===
+
Display the attributes of interface num <font color=red>N</font>
−
 
 
−
<ul>
 
−
FPdescr &larr; alloc_fastpath(codeOpt,bwSpec,resSpec,memSpec,FPdescr)
 
−
         <ul>
 
−
          Given specifications for the aggregate bandwidth,
 
−
        other resource (filters, queues, buffers and stats) and
 
−
        memory, allocate a new FP for the code option, and
 
−
        return a reference to the FP description.
 
 
         </ul>
 
         </ul>
−
free_fastpath(FPid)
+
* scfg --cmd get_ifn --ipaddr <font color=red>A</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Free the resources of the FP.
+
Display the interface associated with IP address <font color=red>A</font>
 
         </ul>
 
         </ul>
−
</ul>
 
  
−
=== FastPath Bandwidth ===
+
=== Reservation Management ===
  
−
<ul>
+
* scfg --cmd get_resrvs
−
RMPcode &larr; resrv_fpath_ifbw(FPid,ifNum,BWkbps)
+
         <ul style="padding-left: 0.5in;">
−
         <ul>
+
Display all slice reservations
−
        Reserve bandwidth (Kbps) on a physical interface for a FP.
 
 
         </ul>
 
         </ul>
−
RMPcode &larr; free_fpath_ifbw(FPid,ifNum,BWkbps)
+
* scfg --cmd get_resrv [--date <font color=red>D</font>]
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Free the bandwidth (Kbps) of a FP from a physical interface,
+
Display slice reservation for period covered by date <font color=red>D</font> (default: current).
−
        and return the status.
+
<br>Date format is YYYYMMDDHHMMSS.
 
         </ul>
 
         </ul>
−
</ul>
+
* scfg --cmd make_resrv --xfile <font color=red>FN</font>
−
 
+
         <ul style="padding-left: 0.5in;">
−
=== FastPath MetaInterfaces ===
+
Make a reservation described by XML file <font color=red>FN</font>
−
 
 
−
<ul>
 
−
MIid &larr; alloc_udp_tunnel(FPid,EPdescr )
 
−
         <ul>
 
−
        Given a UDP tunnel EP description allocate the EP for the FP,
 
−
        and return the MI identifier.
 
 
         </ul>
 
         </ul>
−
RMPcode &larr; free_udp_tunnel(FPid,MIid)
+
* scfg --cmd cancel_resrv [--date <font color=red>D</font>]
−
        <ul>
+
        <ul style="padding-left: 0.5in;">
−
        Free the MI of a FP, and return the status.
+
Cancel slice reservation for period covered by date <font color=red>D</font> (default: current).
 +
<br>Date format is YYYYMMDDHHMMSS.
 
         </ul>
 
         </ul>
−
EPdescr &larr; get_endpoint(FPid,MIid,EPdescr)
+
* scfg --cmd update_resrv --xfile <font color=red>FN</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Return the UDP tunnel EP description for a given MI of a FP.
+
Update slice reservation described by XML file <font color=red>FN</font>
 
         </ul>
 
         </ul>
−
</ul>
 
  
−
=== FastPath Queue Management ===
+
=== Setup/Free ===
−
 
 
−
<ul>
 
−
RMPcode &larr; bind_queue(FPid,MIid,qidListType,qidList)
 
−
        <ul>
 
−
        Associate the listed queues to the MI of the FP, and
 
−
        return the status.
 
−
        </ul>
 
−
Qdescr &larr; get_queue_params(FPid,Qid,Qdescr)
 
−
        <ul>
 
−
        Return the parameters (threshold, bandwidth) for the FP queue,
 
−
        and return a description of the queue.
 
−
        </ul>
 
−
BWkbps &larr; set_queue_params(FPid,Qid,Qdescr)
 
−
        <ul>
 
−
        Set the queue parameters (threshold, bandwidth) for the FP queue,
 
−
        and return the bandwidth of the queue.
 
−
        </ul>
 
−
Qlen &larr; get_queue_len(FPid,Qid,Qlen)
 
−
        <ul>
 
−
        Return the length of the FP queue.
 
−
        </ul>
 
−
</ul>
 
  
−
=== Fastpath Filter Management ===
 
  
−
<ul>
+
* scfg --cmd claim_resources
−
rmpCode &larr; write_fltr(FPid,Fid,Fltr)
+
        <ul style="padding-left: 0.5in;">
−
        <ul>
+
Allocate slowpath (GPE) slice resources
−
        Install a FP filter, and return the status.
 
 
         </ul>
 
         </ul>
−
rmpCode &larr; update_result(FPid,Fid,Fltr)
+
* scfg --cmd free_sp_resources
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Modify the FP filter, and return the status.
+
Free slowpath (GPE) slice resources
 
         </ul>
 
         </ul>
−
Fltr &larr; get_fltr_byfid(FPid,Fid,Fltr)
+
* scfg --cmd free_fp_resources --fp <font color=red>I</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Return the FP filter given the filter ID.
+
Free fastpath slice resources for fastpath <font color=red>I</font>
 
         </ul>
 
         </ul>
−
Fltr &larr; get_fltr_bykey(FPid,key,Fltr)
+
* scfg --cmd setup_sp_endpoint --bw <font color=red>K</font> --ipaddr <font color=red>A</font> --port <font color=red>N</font> --proto <font color=red>P</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Return the FP filter that matches the key.
+
Create the <font color=red>K</font> Kbps capacity slowpath LC endpoint (<font color=red>A</font>, <font color=red>N</font>, <font color=red>P</font>)
 
         </ul>
 
         </ul>
−
fltrResult &larr; lookup_fltr(FPid,key,Fltr)
+
* scfg --cmd free_sp_endpoint --ipaddr <font color=red>A</font> --port <font color=red>N</font> --proto <font color=red>P</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Return the result part of the FP filter that matches the key.
+
Remove the slowpath LC endpoint (<font color=red>A</font>, <font color=red>N</font>, <font color=red>P</font>)
 
         </ul>
 
         </ul>
−
rmpCode &larr; rem_fltr_byfid(FPid,Fid)
+
* scfg --cmd setup_fp_tunnel --fp <font color=red>I</font> --bw  <font color=red>K</font> --ipaddr <font color=red>A</font> --port <font color=red>N</font>
−
        <ul>
+
        <ul style="padding-left: 0.5in;">
−
        Remove the FP filter given the filter ID, and return the status.
+
Add (<font color=red>A</font>, <font color=red>N</font>) as a local end of a fastpath (UDP) tunnel with capacity <font color=red>K</font> Kbps to fastpath <font color=red>I</font>.  Return the fastpath interface identifier in ''scfg'''s exit code which can be captured in the shell variable ''$?''.
 
         </ul>
 
         </ul>
−
rmpCode &larr; rem_fltr_bykey(FPid,key)
+
* scfg --cmd free_fp_tunnel --fp <font color=red>I</font> --fpif <font color=red>I'</font>
−
        <ul>
+
        <ul style="padding-left: 0.5in;">
−
        Remove the highest priority FP filter that matches the key, and
+
Free fastpath interface (<font color=red>I</font>, <font color=red>I'</font>) where <font color=red>I</font> is the fastpath identifier and <font color=red>I'</font> is the fastpath interface identifier
−
        return the status.
 
 
         </ul>
 
         </ul>
−
</ul>
 
  
−
=== FastPath Stats Management ===
+
=== Queue Management ===
  
−
<ul>
+
* scfg --cmd get_queue_params --fp <font color=red>I</font> --qid <font color=red>Q</font>
−
statsRecord &larr; read_stats(FPid,statsId,flags,statsRecord)
+
         <ul style="padding-left: 0.5in;">
−
         <ul>
+
Get the threshold and bandwidth for (fastpath <font color=red>I</font>, queue <font color=red>Q</font>)
−
        Return the FP stats record (counter group) for the stats ID.
 
−
        The flags argument selects which counters to return.
 
−
        You can select the byte or packet counter and
 
−
        whether the preQ or postQ counter
 
−
        </ul>
 
−
rmpCode &larr; clear_stats(FPid,statsId,flags)
 
−
        <ul>
 
−
        Reset the FP stats counters for the stats ID.
 
−
        The flags argument selects which counters to return.
 
 
         </ul>
 
         </ul>
−
statsHandle &larr; create_periodic(FPid,statsId,period, historySize,flags)
+
* scfg --cmd get_queue_len --fp <font color=red>I</font> --qid <font color=red>Q</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Create a periodic stats read event for the stats ID with
+
Get the length (bytes) of (fastpath <font color=red>I</font>, queue <font color=red>Q</font>)
−
        the given period and history size, and
 
−
        return a handle for the operation.
 
−
        The flags argument indicates the retrieval method:
 
−
        either push the stats data to a registered port, or
 
−
        have the VM pull the data using the get_periodic command.
 
 
         </ul>
 
         </ul>
−
rmpCode &larr; delete_periodic(FPid,statsHandle)
+
* scfg --cmd bind_queue --fp <font color=red>I</font> --fpif <font color=red>I'</font> --qid <font color=red>Q1,Q2,...</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Remove the periodic event, remove the callback state, and
+
Bind queues <font color=red>Q1</font>, <font color=red>Q2</font>, ... to (fastpath <font color=red>I</font>, fastpath interface <font color=red>I'</font>)
−
        return the status.
 
 
         </ul>
 
         </ul>
−
rmpCode &larr; set_callback( FPid,statsHandle,ipPortNum)
+
* scfg --cmd bind_queue --fp <font color=red>I</font> --fpif <font color=red>I'</font> --qid <font color=red>Q1-Q2</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Setup the callback for a periodic stats push model that
+
Bind queues in the range <font color=red>Q1</font> to <font color=red>Q2</font> inclusive to (fastpath <font color=red>I</font>, fastpath interface <font color=red>I'</font>)
−
        sends stats records to the IP port number, and return the status.
 
 
         </ul>
 
         </ul>
−
statsRecord &larr; get_periodic(FPid,statsHandle,statsRecord)
+
* scfg --cmd set_queue_params --fp <font color=red>I</font> --qid <font color=red>Q</font> --threshold <font color=red>B</font> --bw <font color=red>K</font>
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        Return the stats record associated with the stats handle.
+
Set the threshold and bandwidth of (fastpath <font color=red>I</font>, queue <font color=red>Q</font>) to <font color=red>B</font> bytes and <font color=red>K</font> Kbps
 
         </ul>
 
         </ul>
−
</ul>
 
−
=== FastPath Memory ===
 
  
−
Each code option is provided with a block of SRAM.
+
=== IPv4 Filter Management ===
−
A slice can read/write to any location in this block.
 
−
A code option may elect to provide library functions to manipulate control structures within this block.
 
−
The valBuf argument to the read/write functions is a structure that includes the number of bytes in the buffer and the buffer itself.
 
  
−
<ul>
+
Filter management for the IPv4 code option is done through the ''ip_fpc'' program.
−
rmpCode &larr; mem_write(FPid,offset,valBuf)
+
The stylized syntax of the ''ip_fpc'' command is:
−
        <ul>
 
−
          Write data to the SRAM starting at offset within the FP block, and
 
−
        return the status.
 
−
        The valBuf argument is a structure that includes the
 
−
        number of bytes and the data.
 
−
        </ul>
 
−
valBuf &larr; mem_read(FPid,offset,nbytes,valBuf)
 
−
        <ul>
 
−
        Read bytes into the value buffer, and return a reference to
 
−
        the value buffer.
 
−
        </ul>
 
−
</ul>
 
−
=== Reservation Management ===
 
  
−
<ul>
+
  ip_fpc --cmd write_fltr --fpid I --fid I'                                      # ID
−
rmpCode &larr; make_reservation(rsvRecord)
+
    [--res_ld]                                                                  # SPECIAL
−
        <ul>
+
   
−
        Make a reservation, and return the status.
+
    --key_type {0|1}  --key_rxmi M                                              # KEY
−
        </ul>
+
    --key_saddr Asrc  --key_sport Nsrc  --key_daddr Adst  --key_dport Ndst    # KEY
−
rmpCode &larr; update_reservation(rsvRecord)
+
    --key_proto P                                                                # KEY
−
        <ul>
+
   
−
        Update a reservation.
+
    --mask_saddr Asrc' --mask_sport Nsrc' --mask_daddr Adst' --mask_dport Ndst'  # MASK
−
        </ul>
+
    --mask_flags P'XXX                                                          # MASK
−
rmpCode &larr; cancel_reservation(date)
+
   
−
        <ul>
+
    --txdaddr Anxt    --txdport Nnxt                                            # RESULT
−
        Cancel the reservation that includes the specified date and time.
+
    --qid Q            --sindx S                                                # RESULT
−
        </ul>
 
−
</ul>
 
  
−
== Substrate Control Daemons (SCD) ==
+
(CAVEAT: XXXXX)
  
−
The SCDs run on the xScale processors of the Line Card and NPE.
+
* ID
−
They provide a messaging interface, through which other control software components can exercise control.
+
** x
−
These include messages to access traffic counters, add/remove TCAM packet filters, configure queue parameters (including WDRR weights and discard thresholds), read/write specific memory locations used for control and status registers, etc.
+
* SPECIAL
−
These are described in more detail below.
+
** x
−
All functions have a context ID (''contextID'') as an argument. A context ID of 0 indicates a privileged operation performed by the substrate.
+
* KEY
−
Any other context ID indicates a user context and is either a fastpath ID or internal slice ID.
+
** x
−
Many of the functions (e.g. ''write_fltr'') appear to be similar to ones in the RMP.
+
* MASK
−
This is expected because the evaluation of an RMP operation must often be relayed to an SCD for evaluation but with one important difference.
+
** x
−
The SCD has a substrate view of objects whereas the RMP provides a higher-level of abstraction.
+
* RESULT
−
The Line Card SCD allows the SRM to control various elements of the Line Card data path.
+
** x
−
This includes the TCAM-resident packet filters (on both input and output), interface addressing and bandwidth, NAT filter table configuration and queueing parameters.
 
−
The NPE SCD allows the SRM and the RMP to control various elements of the NPE data path.
 
−
This includes fast path configuration data, per-slice packet filters resident in the TCAM and queueing parameters.
 
  
−
=== Control Table Initialization ===
+
XXXXX
−
 
+
        <ul style="padding-left: 0.5in;">
−
There are several tables and control blocks used by the control software.
+
        Install a filter for fastpath I with filter identifier F.
−
 
+
        <br>
−
<ul>
+
        match (key-mask), peer, qid, stats
−
set_sched_params(contextId,Sid,ifNum,BWkbpsMax,BWkbpsMin,valBuf)
+
        <br>
−
         <ul>
+
        key_type
−
          Set the interface number and bandwidth characteristics
+
         <br>
−
        for a Scheduler in the Per Scheduler Parameters table.
+
        F is from the range 0 to n-3 where n is the number of filter resources requested for the fastpath.
 
         </ul>
 
         </ul>
−
set_encap_cb(contextId,Sid,srcIPaddr,dstMACaddr,valBuf)
+
  ip_fpc --cmd update_result --fpid I --fid I' RESULT
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
          On the NPE, set the source IP Address and destination
+
        Update the result part of filter (I, I') where I is the fastpath identifier and I' is the filter identifier.
−
        MAC Address associated with the specified scheduler.
 
 
         </ul>
 
         </ul>
−
set_sched_mac(contextId,Sid,dstMACaddr,srcMACaddr,valBuf)
+
  ip_fpc --cmd get_fltr_bykey --fpid I KEY
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
          On the LC, set the destination and source MAC Addresses for
+
        Display the entire filter (key, mask, result) from fastpath I matching KEY.
−
        the specified scheduler.
 
 
         </ul>
 
         </ul>
−
set_encap_gpe(contextId,FPid,GPEipAddr,NPEipAddr,valBuf)
+
  ip_fpc --cmd get_fltr_byfid --fpid I --fid I'
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
          On the NPE, for a fast path, set the GPE IP Address and
+
         Display the filter from fastpath I with filter identifier I'.
−
        NPE IP Address to be used for communication between
 
−
         the GPE and NPE for local delivery and exceptions.
 
 
         </ul>
 
         </ul>
−
set_fpmi_bw(contextId,FPid,Sid,MIid,BWkbps,valBuf)
+
  ip_fpc --cmd lookup_fltr --fpid I KEY
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
          On the NPE, for a particular fast path, set the bandwidth
+
        Display the result part of the filter from fastpath I matching KEY.
−
        for a MI using a particular scheduler.
 
 
         </ul>
 
         </ul>
−
SCDcode &larr; set_src_hwaddr(contextId,MACaddr)
+
  ip_fpc --cmd rem_fltr_bykey --fpid I KEY
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
        On the NPE, set the NPE’s source MAC Address.
 
 
         </ul>
 
         </ul>
−
SCDcode &larr; set_iface_table(contextId,ifTable)
+
  ip_fpc --cmd rem_fltr_byfid --fpid I --fid I'
−
         <ul>
+
         <ul style="padding-left: 0.5in;">
−
          On the NPE, initialize the RX Interface ID table.
+
        Remove the filter (I, I') where I is the fastpath identifier and I' is the filter identifier.
−
        This table translates the receive destination address
 
−
        on a packet to a 4 bit index which will be used in the Lookup key.
 
 
         </ul>
 
         </ul>
−
</ul>
 
  
−
=== FastPath (NPE SCD Only) ===
+
=== IPv4 Fastpath Creation ===
  
−
set_fast_path(contextId,FPid,codeOpt,vlanID, num_queues,num_filters,num_buffers,num_stats, SRAM_offset,SRAM_size,DRAM_offset,DRAM_size,valBuf)
+
  ip_fpd --fpName FN --myIP A --myPort N >> ip_fpd.log &
−
        <ul>
 
−
        On the NPE, create a new fast path.
 
−
        </ul>
 
−
rem_fast_path(contextId,FPid,valBuf)
 
−
        <ul>
 
−
        On the NPE, remove a fast path.
 
−
        </ul>
 
−
SCDcode &larr; set_gpe_info(contextId,EXport,LDport,EXqid,LDqid)
 
−
        <ul>
 
−
        On the NPE, for a particular fast path, set the
 
−
        Local Delivery and Exception traffic port numbers and QIDs.
 
−
        </ul>
 
  
−
=== Memory ===
 
  
−
<ul>
+
xxx
−
write_sram(contextId,offset,valBuf)
 
−
        <ul>
 
−
        On the NPE, write to the SRAM block for a particular fast path.
 
−
        </ul>
 
−
read_sram(contextId,offset,valBuf,count)
 
−
        <ul>
 
−
        On the NPE, read from the SRAM block for a particular fast path.
 
−
        </ul>
 
−
</ul>
 
  
−
=== Queue Management ===
+
 
  
−
<ul>
+
== Examples ==
−
SCDcode &larr; bind_queue(contextId,MIid,qidListType,qidVector)
 
−
        <ul>
 
−
        Associate the listed queues to the context’s MI, and return the status.
 
−
        </ul>
 
−
BWkbps &larr; set_queue_params(contextId,Qid,threshhold,BWkbps)
 
−
        <ul>
 
−
          Set the context’s queue parameters (threshold, bandwidth)
 
−
        for the queue, and return the bandwidth of the queue.
 
−
        </ul>
 
−
get_queue_params(contextId,Qid,threshhold,BWkbps)
 
−
        <ul>
 
−
          Return the context’s parameters (threshold, bandwidth)
 
−
        for the queue through the threshold and BWkbps parameters,
 
−
        and return a description of the queue.
 
−
        </ul>
 
−
get_queue_len(contextId,Qid,pktCnt,byteCnt)
 
−
        <ul>
 
−
          Return the length of the context’s queue through the pktCnt
 
−
        and byteCnt parameters.
 
−
        </ul>
 
−
set_queue_sched(contextId,Qid,Sid,valBuf)
 
−
        <ul>
 
−
        Associate a specified queue with the specified scheduler.
 
−
        </ul>
 
−
</ul>
 
−
 
 
−
=== NPE Filter Management ===
 
  
−
<ul>
+
xxx
−
SCDcode &larr; npe_write_fltr(contextId,Fid,substrateFltr)
 
−
        <ul>
 
−
        Install a context’s substrate (generic) filter with filter ID.
 
−
        </ul>
 
−
SCDcode &larr; npe_update_result(contextId,Fid,result)
 
−
        <ul>
 
−
          Modify the result part of a context’s substrate (generic)
 
−
          filter with filter ID.
 
−
        </ul>
 
−
substrateFltr &larr; npe_get_fltr_by_key(contextId,key,substrateFltr)
 
−
        <ul>
 
−
        Return the context’s substrate (generic) filter that matches the key.
 
−
        </ul>
 
−
substrateFltr &larr; npe_get_fltr_by_fid(contextId,Fid,substrateFltr)
 
−
        <ul>
 
−
        Return the context’s substrate filter given the filter ID.
 
−
        </ul>
 
−
substrateResult &larr; npe_lookup_fltr(contextId,key,substrateResult)
 
−
        <ul>
 
−
          Return the result part of the context’s substrate (generic) filter
 
−
        that matches the key.
 
−
        </ul>
 
−
SCDcode &larr; npe_rem_fltr_by_key(contextId,substrateKey)
 
−
        <ul>
 
−
          Remove the context’s highest priority substrate filter
 
−
        that matches the key, and return the status.
 
−
        </ul>
 
−
SCDcode &larr; npe_rem_fltr_by_fid(contextId,Fid)
 
−
        <ul>
 
−
          Remove the context’s substrate filter given the filter ID,
 
−
        and return the status.
 
−
        </ul>
 
−
</ul>
 
−
=== Line Card Filter Management ===
 
−
 
 
−
There are two Line Card filter databases: ingress and egress.
 
−
Ingress filters are used to determine which SPP component (e.g., NPE, GPE) should handle incoming packets.
 
−
Egress filters are used to determine which output interface to send outgoing packets.
 
−
The database ID (DBid) indicates the database to be used.
 
−
 
 
−
<ul>
 
−
write_fltr( contextId,DBid,Fid,key,mask,result,valBuf)
 
−
        <ul>
 
−
        Install a context’s LC filter (key, mask, result) in the given database.
 
−
        </ul>
 
−
update_result(contextId,DBid,Fid,result)
 
−
        <ul>
 
−
        Update a context’s LC filter result in the specified database.
 
−
        </ul>
 
−
get_fltr_by_key(contextId,DBid,key,mask,result,keyLen,resultLen)
 
−
        <ul>
 
−
        Given the key, retrieve a filter from the specified database.
 
−
        </ul>
 
−
get_fltr_by_fid(contextId,DBid,Fid,key,mask,result,keyLen, resultLen)
 
−
        <ul>
 
−
        Given the filter id, retrieve a filter from the specified database.
 
−
        </ul>
 
−
lookup_fltr(contextId,DBid,key,result,resultLen)
 
−
        <ul>
 
−
        Given the key, retrieve the filter result from the specified database.
 
−
        </ul>
 
−
rem_fltr_by_key(contextId,DBid,key,valBuf)
 
−
        <ul>
 
−
        Given the key, remove the filter from the specified database.
 
−
        </ul>
 
−
rem_fltr_by_fid(contextId,DBid,Fid,valBuf)
 
−
        <ul>
 
−
        Given the filter id, remove the filter from a specified database.
 
−
        </ul>
 
−
</ul>
 
−
 
 
−
=== Statistics Management ===
 
−
 
 
−
<ul>
 
−
statsRecord &larr; read_stats(contextId,statsId,flags,statsRecord)
 
−
        <ul>
 
−
          Return the context’s stats record (counter group) for the stats ID.
 
−
        The flags argument selects which counters to return.
 
−
        You can select the byte or packet counter and
 
−
        whether the preQ or postQ counter.
 
−
        </ul>
 
−
SCDcode &larr; clear_stats(contextId,statsId,flags)
 
−
        <ul>
 
−
          Reset the context’s stats counters for the stats ID, and
 
−
        return the status.
 
−
        The flags argument selects which counters to return.
 
−
        </ul>
 
−
statsHandle &larr; create_periodic(contextId,statsId,period,count, flags)
 
−
        <ul>
 
−
          Create a periodic stats read event for the stats ID of
 
−
        the context with the given period and history size, and
 
−
        return a handle for the operation.
 
−
        The flags argument indicates the retrieval method:
 
−
        either push the stats data to a registered port, or have
 
−
        the VM pull the data using the get_periodic command.
 
−
        </ul>
 
−
SCDcode &larr; del_periodic(contextId,statsHandle)
 
−
        <ul>
 
−
          Remove the context’s periodic event, remove the callback state,
 
−
        and return the status.
 
−
        </ul>
 
−
SCDcode &larr; set_callback(contextId,statsHandle,UDPport)
 
−
        <ul>
 
−
          Setup the context’s callback for a periodic stats push
 
−
        model that sends stats records to the UDP port number, and
 
−
        return the status.
 
−
        </ul>
 
−
statsRecordVector &larr; get_periodic(contextId,statsHandle, statsRecordVector)
 
−
        <ul>
 
−
        Return the context’s stats record associated with the stats handle.
 
−
        </ul>
 
−
</ul>
 
−
 
 
−
=== MicroEngine Management ===
 
−
 
 
−
<ul>
 
−
start_mes(contextId,valBuf)
 
−
        <ul>
 
−
        Start the MicroEngines on an NPU.
 
−
        </ul>
 
−
stop_mes(contextId,valBuf)
 
−
        <ul>
 
−
        Stop the MicroEngines on an NPU.
 
−
        </ul>
 
−
</ul>
 
−
 
 
−
=== NAT ===
 
−
 
 
−
<ul>
 
−
nat_filters(contextId,ingressStartFid,ingressEndFid, egressStartFid, egressEndFid)
 
−
        <ul>
 
−
          On the LC, initialize the NAT filter tables.
 
−
        This sets aside a block of the TCAM for the Ingress NAT filters
 
−
        and a block of the TCAM for the Egress NAT filters.
 
−
        </ul>
 
−
</ul>
 
−
=== MetaInterface Management ===
 
−
 
 
−
<ul>
 
−
SCDcode &larr; create_mi(contextId,FPid,MIid,Sid)
 
−
        <ul>
 
−
        On the NPE,cCreate a new meta-interface for a fast path.
 
−
        </ul>
 
−
SCDcode &larr; delete_mi(contextId,FPid,MIid)
 
−
        <ul>
 
−
          On the NPE, delete the specified meta-interface for
 
−
        the specified fast path.
 
−
        </ul>
 
−
SCDcode &larr; set_mi_bw(contextId,FPid,MIid,BWkbps)
 
−
        <ul>
 
−
          On the NPE, for the specified fast path, set the bandwidth
 
−
        for a meta-interface.
 
−
        </ul>
 
−
SCDcode &larr; bind_queue_sched(contextId,Qid,Sid)
 
−
        <ul>
 
−
        On the NPE, bind a queue to a scheduler.
 
−
        </ul>
 
−
SCDcode &larr; unbind_queue_sched(contextId,Qid)
 
−
        <ul>
 
−
          On the NPE, unbind a queue from a scheduler and
 
−
        release its bandwidth on that scheduler.
 
−
        </ul>
 
−
SCDcode &larr; unbind_queue(contextId,Qid)
 
−
        <ul>
 
−
          On the NPE, unbind a queue from a meta-interface and
 
−
        release its bandwidth on that meta-interface.
 
−
        </ul>
 
−
</ul>
 

Latest revision as of 16:02, 2 April 2010


System Resource Manager (SRM) and Resource Manager Proxy (RMP)

Major Control Software Modules

The page SPP Control Software describes the System Resource Manager (SRM) that runs on the SPP's Control Processor and the Resource Manager Proxy (RMP) that runs on each SPP's GPE, and it describes the programming API provided by the RMP. This page describes the accompanying command line interface which allows users to reserve and configure resources interactively or through a shell script. In brief, the SRM coordinates the use of various SPP resources while the RMP provides an interface through which user slices can request and configure resources. The RMP, in turn, may communicate with the Substrate Control Daemons (SCD) in the Line Card and the NPE to configure packet pathways.

User requests are made through an interface provided by the RMP on the user’s assigned GPE. The RMP forwards these requests to the SRM which manages all system level resources, including external port numbers, physical interface bandwidth and NPE resources. For example, a user can request the use of a specific external port number in order to run a server that listens on that port at the user's GPE. Or a user may want to reserve resources in advance (e.g., port bandwidth and NPE fastpath).

That command line interface is currently provided through three programs: scfg (slice configuration), ip_fpc (IPv4 filter configuration) and ip_fpd (IPv4 fastpath daemon). (Note: These names are likely to change in the near future. The scfg program applies to any fastpath while ip_fpc and ip_fpd are specific to the IPv4 code option.

Program Functionality
scfg Interface information, endpoint management,
fastpath management, queue management,
reservation management
ip_fpc IPv4 filter management
ip_fpd IPv4 fastpath configuration, LD and EX traffic handling

There will be other fastpath-specific programs as other code options get developed. It is worth noting that all of these programs implement their functionallity by calling RMP library functions. The fact that these programs exist as separate programs is historic. For example, one could have a single ipv4_control program that included all of the functionality of these programs. Furthermore, a new code option such as vnet would have a vnet_control program that included the functionality in the scfg program above and vnet-specific functionality for filters and handling LD and EX traffic. (Note: The programs ip_fpc and ip_fpd were called fltr and create_fp respectively in The GEC4 Demo.)

Command Interface Syntax

The scfg command uses GNU-style long options (e.g., --cmd get_endpoint, --ipaddr 10.1.2.3). And although they both have a few arguments that control their configuration (e.g., message log noise volume), this page only describes the user commands. In the descriptions below, the following metasymbols are used in the syntactic descriptions:

A IP address (e.g., 192.168.2.32) K The number of Kilobits per second
D Date in YYYMMDDHHMMSS format
F Filter identifier N Number (non-negative)
FN Filename P Protocol number (e.g., 6 (TCP), 17 (UDP))
I An identifier or index Q Queue identifier
[x] Optional argument x S Statistics index
{ x xxx y } Alternative arguments x and y X Hexadecimal mask or 0

When a command description needs to use multiple instances of a metasymbol, distinction between instances is indicated by following the metasymbol insances by one or more distinguishing symbols (e.g., quote, integer, lowercase letters) (e.g., N and N' or N1, N2 and N3).

Command Interface

The following description of the command-line interface is meant as a quick reference. If you are looking for a tutorial format, consider The SPP Tutorial page which will contain links to The Hello GPE World Tutorial and Using the IPv4 Code Option.

The command-line interface is provided through three programs: scfg (slice configuration), ip_fpc (IPv4 fastpath filter configuration) and ip_fpd (IPv4 fastpath daemon). The scfg program applies to any fastpath while ip_fpc and ip_fpd are specific to the IPv4 code option. The primary function of the IPv4 daemon ip_fpd is to intialize the IPv4 code option and handle local delivery and error packets for that code option. The program ip_fpc manages filters for the IPv4 code option. All other functions that are generic to any fastpath (e.g., getting information about physical interfaces, allocating resources, making resource reservations, managing fastpath queues) are done by the scfg program.

Getting Interface Information

  • scfg --cmd get_ifaces
    Display all interfaces
  • scfg --cmd get_ifpeer --ifn N
    Display the peer of interface num N
  • scfg --cmd get_ifattrs --ifn N
    Display the attributes of interface num N
  • scfg --cmd get_ifn --ipaddr A
    Display the interface associated with IP address A

Reservation Management

  • scfg --cmd get_resrvs
    Display all slice reservations
  • scfg --cmd get_resrv [--date D]
    Display slice reservation for period covered by date D (default: current).
    Date format is YYYYMMDDHHMMSS.
  • scfg --cmd make_resrv --xfile FN
    Make a reservation described by XML file FN
  • scfg --cmd cancel_resrv [--date D]
    Cancel slice reservation for period covered by date D (default: current).
    Date format is YYYYMMDDHHMMSS.
  • scfg --cmd update_resrv --xfile FN
    Update slice reservation described by XML file FN

Setup/Free

  • scfg --cmd claim_resources
    Allocate slowpath (GPE) slice resources
  • scfg --cmd free_sp_resources
    Free slowpath (GPE) slice resources
  • scfg --cmd free_fp_resources --fp I
    Free fastpath slice resources for fastpath I
  • scfg --cmd setup_sp_endpoint --bw K --ipaddr A --port N --proto P
    Create the K Kbps capacity slowpath LC endpoint (A, N, P)
  • scfg --cmd free_sp_endpoint --ipaddr A --port N --proto P
    Remove the slowpath LC endpoint (A, N, P)
  • scfg --cmd setup_fp_tunnel --fp I --bw K --ipaddr A --port N
    Add (A, N) as a local end of a fastpath (UDP) tunnel with capacity K Kbps to fastpath I. Return the fastpath interface identifier in scfg's exit code which can be captured in the shell variable $?.
  • scfg --cmd free_fp_tunnel --fp I --fpif I'
    Free fastpath interface (I, I') where I is the fastpath identifier and I' is the fastpath interface identifier

Queue Management

  • scfg --cmd get_queue_params --fp I --qid Q
    Get the threshold and bandwidth for (fastpath I, queue Q)
  • scfg --cmd get_queue_len --fp I --qid Q
    Get the length (bytes) of (fastpath I, queue Q)
  • scfg --cmd bind_queue --fp I --fpif I' --qid Q1,Q2,...
    Bind queues Q1, Q2, ... to (fastpath I, fastpath interface I')
  • scfg --cmd bind_queue --fp I --fpif I' --qid Q1-Q2
    Bind queues in the range Q1 to Q2 inclusive to (fastpath I, fastpath interface I')
  • scfg --cmd set_queue_params --fp I --qid Q --threshold B --bw K
    Set the threshold and bandwidth of (fastpath I, queue Q) to B bytes and K Kbps

IPv4 Filter Management

Filter management for the IPv4 code option is done through the ip_fpc program. The stylized syntax of the ip_fpc command is: 

 ip_fpc --cmd write_fltr --fpid I --fid I'                                      # ID
   [--res_ld]                                                                   # SPECIAL
   
   --key_type {0|1}   --key_rxmi M                                              # KEY
   --key_saddr Asrc   --key_sport Nsrc   --key_daddr Adst   --key_dport Ndst    # KEY
   --key_proto P                                                                # KEY
   
   --mask_saddr Asrc' --mask_sport Nsrc' --mask_daddr Adst' --mask_dport Ndst'  # MASK
   --mask_flags P'XXX                                                           # MASK
   
   --txdaddr Anxt     --txdport Nnxt                                            # RESULT
   --qid Q            --sindx S                                                 # RESULT

(CAVEAT: XXXXX)

  • ID
    • x
  • SPECIAL
    • x
  • KEY
    • x
  • MASK
    • x
  • RESULT
    • x

XXXXX

    Install a filter for fastpath I with filter identifier F.
    match (key-mask), peer, qid, stats
    key_type
    F is from the range 0 to n-3 where n is the number of filter resources requested for the fastpath. 
 ip_fpc --cmd update_result --fpid I --fid I' RESULT
    Update the result part of filter (I, I') where I is the fastpath identifier and I' is the filter identifier. 
 ip_fpc --cmd get_fltr_bykey --fpid I KEY
    Display the entire filter (key, mask, result) from fastpath I matching KEY. 
 ip_fpc --cmd get_fltr_byfid --fpid I --fid I'
    Display the filter from fastpath I with filter identifier I'. 
 ip_fpc --cmd lookup_fltr --fpid I KEY
    Display the result part of the filter from fastpath I matching KEY. 
 ip_fpc --cmd rem_fltr_bykey --fpid I KEY

 ip_fpc --cmd rem_fltr_byfid --fpid I --fid I'
    Remove the filter (I, I') where I is the fastpath identifier and I' is the filter identifier.

IPv4 Fastpath Creation

 ip_fpd --fpName FN --myIP A --myPort N >> ip_fpd.log &


xxx


Examples

xxx

Retrieved from "http://wiki.arl.wustl.edu/index.php?title=SPP_Command_Interface&oldid=1080"