Difference between revisions of "The SPP Tutorial"

From ARL Wiki
Jump to navigationJump to search
 
(14 intermediate revisions by one other user not shown)
Line 1: Line 1:
{{Template: Under Construction}}
 
 
[[Category: The SPP]]
 
[[Category: The SPP]]
 
[[Category: The SPP Tutorial]]
 
[[Category: The SPP Tutorial]]
Line 5: Line 4:
 
== Introduction ==
 
== Introduction ==
  
This tutorial is designed to give a new SPP user some simple, step-by-step usage examples.
+
This tutorial page and [[The Hello GPE World Tutorial]] page are designed to give you simple, step-by-step usage examples that will allow you to get started quickly.
This page and three others illustrate the main concepts required to get you started quickly and contain links to a number of auxilliary pages that will allow you to try more complex examples.
+
They also contain links to a number of auxilliary pages that will allow you to try more complex examples.
 +
Two other wiki pages complete the tutorial:  [[Using the IPv4 Code Option]] and [[Developing a New Code Option]].
  
[[ Image:SPP-components.png | thumb | right | 400px | SPP Components (LC, CP, GPE, NPE) ]]
+
You don't have to read all four pages before using the SPPs.
 +
But you can't do much unless you read this page and [[The Hello GPE World Tutorial]] page.
 +
But these two pages (like the other two) are designed so that you can get hands-on experience by following the prescribed steps.
  
Recall from [[Internet Scale Overlay Hosting]] that an advanced SPP developer writes code for a ''General-purpose Processing Engine'' (GPE) and a ''Network Processor Engine'' (NPE).
+
('''Note:''' Scan the [[Internet Scale Overlay Hosting]] page if you have not already done so paying particular attention to the parts describing the role of the GPE (''General-purpose Processing Engine'') and the NPE (''Network Processor Engine'').
Typically, a developer configures the SPP so that the NPE handles most data packets (the ''fastpath'') while the GPE handles control and exception packets (the ''slowpath'').
+
Keep in mind that in a high-speed implementation, the SPP would be configured so that the NPE handled most data packets (the ''fastpath'') leaving the GPE to handle control and exception packets (the ''slowpath'').)
The configuration process loads code into the GPE and NPE and installs filters in the ''Line Card'' (LC) to direct packets to the GPE or NPE.
 
  
In practice, we anticipate that most developers will write a prototype version of their router software using only the GPE and work with a SPP developer to transform the performance-critical parts of their GPE code to run on the NPE because programming an NPE is quite different than programming a general-purpose processor.
+
This page will walk you through the preliminary steps required to get setup to use an SPP; e.g., getting an SPP account and adding SPP nodes to your slice.
That is, most developers will write a ''GPE-only prototype'' which captures the logic of their router software and pass this design to an advanced SPP developer to create a high-speed NPE-GPE version.
 
  
There are four main wiki pages in [[The SPP Tutorial]]:
+
[[The Hello GPE World Tutorial]] will show you how to use the GPE to create a GPE-only prototype.
 +
It will show you how to get information about ''interfaces''; how to create ''slowpath endpoints'' for communicating with a process running on a GPE; and how to make a resource reservation.
 +
It will show you how to write a simple packet-processing program that will run on a GPE.
 +
This program can be extended to create a GPE-only prototype of a simple slice.
  
* [[The SPP Tutorial]]: is this page which gives an overview of the other tutorial pages and covers administrative issues such as creating an account and getting a slice.
+
By this point, you should be able to do simple SPP experiments that will help you develop your own GPE-only slice software.
* [[The Hello GPE World Tutorial]]: describes how to use the GPE to create a GPE-only prototype.
+
Although this GPE-only software can not take full advantage of the SPP because it doesn't use an NPE, most users will not be writing code for NPEs because programming an NPE requires knowledge about IXP network processor programming and the SPP substrate layer.
* [[The IPv4 Code Option Tutorial]]: describes how to use the IPv4 code option which uses an NPE to forward IPv4 packets and a GPE process packets terminating at the router (e.g., local delivery and exceptions).
+
In practice, we anticipate that most developers will write a GPE-only version of their slice and work with an advanced SPP developer to transform the performance-critical parts of their GPE-only code to run on the NPE.
* [[Writing A Simple Code Option Tutorial]]: discusses how to develop a new code option.
+
That is, most developers will write a GPE-only program which captures the logic of their slice and pass this design to an advanced SPP developer who will create a high-speed fastpath (NPE-GPE) version.
  
[[The Hello GPE World Tutorial]] describes how to get information about ''interfaces''; how to create ''slowpath endpoints'' for communicating with a process running on a GPE; and how to make a resource reservation.
+
[[Using the IPv4 Code Option]] describes how to use the IPv4 code option which uses an NPE to forward IPv4 packets and a GPE to process packets terminating at the router (e.g., local delivery and exceptions).
This tutorial will help you understand how to write a simple packet-processing program that will run on a GPE; i.e., a GPE-only prototype.
+
This page will walk you through an example of how to use the IPv4 code option.
[[The IPv4 Code Option Tutorial]] describes a simple example of using the IPv4 code option.
+
It explains how to create ''fastpath endpoints'', ''meta-interfaces'', ''packet queues'' and ''IPv4 filters''.
This tutorial explains how to create ''fastpath endpoints'', ''meta-interfaces'', ''packet queues'' and ''IPv4 filters''.
+
You can directly use all of these concepts in developing your own slice except for IPv4 filters.
Your code option can directly use all of these concepts except for IPv4 filters.
 
 
However, your code option will need to use filters to direct incoming packets to packet queues.
 
However, your code option will need to use filters to direct incoming packets to packet queues.
Seeing how it is done for the IPv4 code option will give you an understanding of how to design your filters.
+
Seeing how it is done for the IPv4 code option will give you a better understanding of how to design filters for your own slice.
[[Writing A Simple Code Option Tutorial]] goes through an example of a simple code option in which we first build a GPE-only prototype and how it would be transformed to use both an NPE and a GPE.
+
 
 +
[[Developing a New Code Option]] walks you through an example of how to develop a simple code option.
 +
It shows you how to build a GPE-only slice for a simple protocol and discusses how that code might be transformed to use both an NPE and a GPE.
 +
 
 +
== Tutorial Conventions ==
 +
 
 +
We show blocks of Unix commands and their output in monospaced typeface indented by four spaces.
 +
When a command and its output are shown together, the command is distinguished from its output by preceding the command with the prompt characters "<tt>&gt; </tt>" (greater than, space).
 +
In the example,
 +
 
 +
<pre>
 +
    > ls
 +
    foo    myfile    yourfile
 +
</pre>
 +
 
 +
<tt>ls</tt> is the command, and its output is shown without the preceding prompt characters.
 +
 
 +
User-specific command arguments will be shown as fully capitalized words.
 +
For example, in the command:
 +
 
 +
<pre>
 +
    ssh YOUR_SLICE@64.57.23.210
 +
</pre>
 +
 +
<tt>YOUR_SLICE</tt> should be replaced by the user's slice name.
 +
 
 +
The following command block shows two forms of commentary:
 +
 
 +
<pre>
 +
    ssh-add -v        # -v outputs debug info and is optional
 +
    ... You will be asked for your passphrase ...
 +
</pre>
 +
 
 +
that only provides guidance.
 +
You are not suppose to key in these comments.
 +
 
 +
When a command sequence involves more than one host, the hosts are distinguished by the command prompts.
 +
For example,
 +
 
 +
<pre>
 +
    host> ssh YOUR_SLICE@64.57.23.210    # command entered at myhost
 +
    SLC>  ls                            # command entered at SLC GPE
 +
</pre>
 +
 
 +
shows the first command entered while logged into ''myhost'' and the second command entered after logging into a Salt Lake City GPE.
  
 
== Requirements ==
 
== Requirements ==
  
This tutorial assumes that you are using a Linux operating system and have access to standard Unix tools.
+
This tutorial assumes that you will be using a Linux operating system and have access to standard Unix tools.
 
In particular, you will need:
 
In particular, you will need:
  
Line 41: Line 87:
 
# An OpenSSH executable
 
# An OpenSSH executable
 
# The latest version of Java to run some monitoring software and traffic generators
 
# The latest version of Java to run some monitoring software and traffic generators
 +
 +
=== Compiling ===
 +
 +
You can use almost any Intel host running a recent version of Linux to compile code for the GPE.
 +
But if you think this is an issue, you can always try to match the version of Linux and glibc on a GPE.
 +
You can get this information after you ''ssh'' to a GPE (described later in [[The Hello GPE World Tutorial]]) by entering:
 +
 +
<pre>
 +
    uname -sr          # get version of Linux
 +
    ldd --version      # get version of glibc
 +
</pre>
 +
 +
=== OpenSSH Executable ===
 +
 +
''OpenSSH'' is a suite of open source utilities using the ''ssh'' protocol.
 +
Although you don't have to specifically use the OpenSSH suite, the tutorial examples assume that the ''ssh'' tools are from OpenSSH.
  
 
=== The Java Run-time Environment (JRE) ===
 
=== The Java Run-time Environment (JRE) ===
Line 56: Line 118:
 
==== MAC Users ====
 
==== MAC Users ====
  
Follow this link [[Java Run-time Environment for the MAC OS]] for instructions.
+
Follow the instructions at [[Java Run-time Environment for the MAC OS]].
  
 
== Getting Started ==
 
== Getting Started ==
Line 77: Line 139:
 
The configuration process is fairly straightforward since a user typically customizes an existing configuration script to do the configuration.
 
The configuration process is fairly straightforward since a user typically customizes an existing configuration script to do the configuration.
  
=== Getting an SPP Account ===
+
=== Creating an SPP Account ===
  
* Create RSA key-pair
+
To create an SPP account, begin by using your browser to go to the SPP PlanetLab web site http:drn06.arl.wustl.edu.
** already have one if you have a PlanetLab account
+
Select the ''Create an account'' link, and a registration form will appear.
* Fill out registration form
+
Fill out the registration form and select the ''Register'' button at the bottom of the page.
** email address important
+
Your email address will be used as your SPP user name.
** site
+
So, make sure that you enter an address where you actually read email.
** roles
+
After your account is created (which can take 24 hours or longer if you try to get an account on a weekend or holiday), you will be notified by email.
* When registration has been accepted and you are assigned to a slice
 
** login
 
** upload RSA key (sidebar "Users => My Account")
 
  
What if you are a PI???
+
You will need to use ''ssh'' to log into to SPP nodes.
 +
So, while you are waiting for this notification, you can create your RSA key pair if you haven't already done so.
 +
You shouldn't need to do this if you already have a PlanetLab account because you can just use your PlanetLab RSA key pair.
 +
We assume below that you will be storing your RSA key pair in the standard place ''~/.ssh/id_rsa'' (private key) and ''~/.ssh/id_rsa.pub'' (public key).
 +
Use the ''ssh-keygen'' command to generate the RSA key pair:
  
== Adding SPP Nodes to Your SPP Slice ==
+
<pre>
 +
    $ ssh-keygen -t rsa -f ~/.ssh/id_rsa
 +
    Generating public/private rsa key pair.
 +
    Enter passphrase (empty for no passphrase): ENTER SOME PASSPHRASE
 +
    Enter same passphrase again: ENTER THE PASSPHRASE AGAIN
 +
</pre>
  
 +
You will need to upload your public key to the SPP PlanetLab database after you receive email notification that your SPP account has been created.
 +
After you receive the notification:
  
== Getting a PlanetLab Account ==
+
* Go to the SPP PlanetLab Central web site http:drn06.arl.wustl.edu.
 +
* Log into the web site.
 +
* In the navigation box on the left, click “My Account.”
 +
* In the “Keys” section, choose “Manage Keys”, and select “Browse” and navigate to your public key file (i.e. ''~/.ssh/id_rsa.pub'').
 +
* Check ''Upload''.  (sidebar "Users => My Account")
  
 +
You should see XXXXX.
  
 +
>>>>> ABOVE IS INCOMPLETE.  WHAT ABOUT SITE AND ROLES? <<<<<
  
== Adding PlanetLab Nodes to Your PlanetLab Slice ==
 
  
 +
>>> What should PIs and Technical Leads do??? <<<
  
 +
== Adding SPP Nodes to Your SPP Slice ==
  
== Deploying Software to the SPPs ==
+
>>> SOMETHING GOES HERE <<<

Latest revision as of 13:48, 29 June 2010


Introduction

This tutorial page and The Hello GPE World Tutorial page are designed to give you simple, step-by-step usage examples that will allow you to get started quickly. They also contain links to a number of auxilliary pages that will allow you to try more complex examples. Two other wiki pages complete the tutorial: Using the IPv4 Code Option and Developing a New Code Option.

You don't have to read all four pages before using the SPPs. But you can't do much unless you read this page and The Hello GPE World Tutorial page. But these two pages (like the other two) are designed so that you can get hands-on experience by following the prescribed steps.

(Note: Scan the Internet Scale Overlay Hosting page if you have not already done so paying particular attention to the parts describing the role of the GPE (General-purpose Processing Engine) and the NPE (Network Processor Engine). Keep in mind that in a high-speed implementation, the SPP would be configured so that the NPE handled most data packets (the fastpath) leaving the GPE to handle control and exception packets (the slowpath).)

This page will walk you through the preliminary steps required to get setup to use an SPP; e.g., getting an SPP account and adding SPP nodes to your slice.

The Hello GPE World Tutorial will show you how to use the GPE to create a GPE-only prototype. It will show you how to get information about interfaces; how to create slowpath endpoints for communicating with a process running on a GPE; and how to make a resource reservation. It will show you how to write a simple packet-processing program that will run on a GPE. This program can be extended to create a GPE-only prototype of a simple slice.

By this point, you should be able to do simple SPP experiments that will help you develop your own GPE-only slice software. Although this GPE-only software can not take full advantage of the SPP because it doesn't use an NPE, most users will not be writing code for NPEs because programming an NPE requires knowledge about IXP network processor programming and the SPP substrate layer. In practice, we anticipate that most developers will write a GPE-only version of their slice and work with an advanced SPP developer to transform the performance-critical parts of their GPE-only code to run on the NPE. That is, most developers will write a GPE-only program which captures the logic of their slice and pass this design to an advanced SPP developer who will create a high-speed fastpath (NPE-GPE) version.

Using the IPv4 Code Option describes how to use the IPv4 code option which uses an NPE to forward IPv4 packets and a GPE to process packets terminating at the router (e.g., local delivery and exceptions). This page will walk you through an example of how to use the IPv4 code option. It explains how to create fastpath endpoints, meta-interfaces, packet queues and IPv4 filters. You can directly use all of these concepts in developing your own slice except for IPv4 filters. However, your code option will need to use filters to direct incoming packets to packet queues. Seeing how it is done for the IPv4 code option will give you a better understanding of how to design filters for your own slice.

Developing a New Code Option walks you through an example of how to develop a simple code option. It shows you how to build a GPE-only slice for a simple protocol and discusses how that code might be transformed to use both an NPE and a GPE.

Tutorial Conventions

We show blocks of Unix commands and their output in monospaced typeface indented by four spaces. When a command and its output are shown together, the command is distinguished from its output by preceding the command with the prompt characters "> " (greater than, space). In the example,

    > ls
    foo    myfile    yourfile

ls is the command, and its output is shown without the preceding prompt characters.

User-specific command arguments will be shown as fully capitalized words. For example, in the command:

    ssh YOUR_SLICE@64.57.23.210

YOUR_SLICE should be replaced by the user's slice name.

The following command block shows two forms of commentary:

    ssh-add -v        # -v outputs debug info and is optional
    ... You will be asked for your passphrase ...

that only provides guidance. You are not suppose to key in these comments.

When a command sequence involves more than one host, the hosts are distinguished by the command prompts. For example,

    host> ssh YOUR_SLICE@64.57.23.210    # command entered at myhost
    SLC>  ls                             # command entered at SLC GPE

shows the first command entered while logged into myhost and the second command entered after logging into a Salt Lake City GPE.

Requirements

This tutorial assumes that you will be using a Linux operating system and have access to standard Unix tools. In particular, you will need:

  1. An environment where you can compile code for a Linux operating system
  2. An OpenSSH executable
  3. The latest version of Java to run some monitoring software and traffic generators

Compiling

You can use almost any Intel host running a recent version of Linux to compile code for the GPE. But if you think this is an issue, you can always try to match the version of Linux and glibc on a GPE. You can get this information after you ssh to a GPE (described later in The Hello GPE World Tutorial) by entering:

     uname -sr          # get version of Linux
     ldd --version      # get version of glibc

OpenSSH Executable

OpenSSH is a suite of open source utilities using the ssh protocol. Although you don't have to specifically use the OpenSSH suite, the tutorial examples assume that the ssh tools are from OpenSSH.

The Java Run-time Environment (JRE)

Most of the tutorial can be done without the JRE. But it will be helpful in the IPv4 Code Option Tutorial.

First, determine if you already have JRE 1.6.0 or higher. Enter the following in a command-line window: java -version. If the response says that you are already running some version that is atleast 1.6.0, you can skip this step.

If you need to install the JRE, go to the Java download page http://java.com/en/download/manual.jsp and download the latest JRE for your platform by following the instructions posted at the Sun site. Note: You only need to download the JRE (less than 20 MB), not the entire JDK (Java Development Kit).

MAC Users

Follow the instructions at Java Run-time Environment for the MAC OS.

Getting Started

You must get an SPP account before you can use the SPPs. The procedure is similar to the one used in PlanetLab except that SPP slices are administered through a special PLC (PlanetLab Central) site at http://drn06.arl.wustl.edu/. But the following PlanetLab procedure still holds:

  • A Principle Investigator (PI) must create a slice for you.
  • The PI must assign you to that slice.
  • Then, you can add SPP nodes to your slice.

In a typical scenario, you (as an ordinary PlanetLab user) will use both normal PlanetLab nodes and SPP nodes. This requires that you get assigned both a normal PlanetLab slice and an SPP slice. Once you have been assigned to an SPP slice, you can log into the GPE of an SPP slice like any other PlanetLab node using ssh and develop a socket program as you would for any other PlanetLab node except for a few small differences:

  • An SPP has multiple network interfaces if you want to use more than one interface.
  • You must configure the SPP before you can use it.

The configuration process is fairly straightforward since a user typically customizes an existing configuration script to do the configuration.

Creating an SPP Account

To create an SPP account, begin by using your browser to go to the SPP PlanetLab web site http:drn06.arl.wustl.edu. Select the Create an account link, and a registration form will appear. Fill out the registration form and select the Register button at the bottom of the page. Your email address will be used as your SPP user name. So, make sure that you enter an address where you actually read email. After your account is created (which can take 24 hours or longer if you try to get an account on a weekend or holiday), you will be notified by email.

You will need to use ssh to log into to SPP nodes. So, while you are waiting for this notification, you can create your RSA key pair if you haven't already done so. You shouldn't need to do this if you already have a PlanetLab account because you can just use your PlanetLab RSA key pair. We assume below that you will be storing your RSA key pair in the standard place ~/.ssh/id_rsa (private key) and ~/.ssh/id_rsa.pub (public key). Use the ssh-keygen command to generate the RSA key pair:

    $ ssh-keygen -t rsa -f ~/.ssh/id_rsa
    Generating public/private rsa key pair.
    Enter passphrase (empty for no passphrase): ENTER SOME PASSPHRASE
    Enter same passphrase again: ENTER THE PASSPHRASE AGAIN

You will need to upload your public key to the SPP PlanetLab database after you receive email notification that your SPP account has been created. After you receive the notification:

  • Go to the SPP PlanetLab Central web site http:drn06.arl.wustl.edu.
  • Log into the web site.
  • In the navigation box on the left, click “My Account.”
  • In the “Keys” section, choose “Manage Keys”, and select “Browse” and navigate to your public key file (i.e. ~/.ssh/id_rsa.pub).
  • Check Upload. (sidebar "Users => My Account")

You should see XXXXX.

>>>>> ABOVE IS INCOMPLETE. WHAT ABOUT SITE AND ROLES? <<<<<


>>> What should PIs and Technical Leads do??? <<<

Adding SPP Nodes to Your SPP Slice

>>> SOMETHING GOES HERE <<<