I am currently a senior at UCSB majoring in Computer
Science and I also have been working part-time as a developer in the local
computer industry for three years. During that time I have worked
within the datacommunications and telecommunications fields, first at Stratacom,
then cisco Systems, and now for Cadence
Design Systems. I have been with Cadence for a little over a
year. Within Cadence I work for the Broadband Services Business Unit,
a group that specializes in design services of data/tele-communications
applications and products.
The Trout project got started because the customer
hired us to port Mrouted so they would be able to have a multicast capable
router. When Trout started I was not actively on a project so I was
assigned as one of the developers to Trout. After Trout, I suggested
a port of the Trout software be added to the Dorado project and since it
was my suggestion, it became my assignment.
For the past several weeks I have worked on a project code named Dorado. The project is a continuation of a customer project and involves porting DVMRP, a IP Multicast Routing Protocol, onto a single board computer that is based on a Motorola MPC860. The original source code used was UNIX Mrouted 3.8 which was downloaded from the Xerox PARC internet site. There were several issues to consider in moving the original sources from a UNIX based host to a router platform that contained no file system. After removing the UNIX dependencies, the software had to be grafted into our customer's router. Instrumentation had to be added to conform with the new SNMP MIBs that the customer required for remote management, which are not supported by Mrouted. The final stage of this project (Dorado) involved porting Mrouted from the customer's router to the internal develop system, using what was learned in the original port.
BACKGROUND: This Dorado project grew out of another project that I worked on called Trout. The goal of Trout was to port the UNIX Mrouted to a customer router based on the Intel i960 using VxWorks 5.1.
When the project was started, it was discovered that
the customer was just using VxWorks as a scheduler and that they had their
own implementation of the various protocol stacks. This created two
main issues that had to be overcome in the port. Other issues which
arose were the need for additional configuration and management functionality,
and the removal of UNIX-centric programming practices in the Mrouted code.
First, since the customer's router had no file system,
the way that Mrouted expected to be started and get its configuration had
to be changed. Instead of retrieving its' configuration from a file,
Mrouted was modified to use a set of function calls that could query the
customer's router to get its configuration.
The new platform had its own networking code, which
did not include socket calls, get and set socket options, or ioctl functionality.
Mrouted had to be changed to call other existing library functions,
or ones that were implemented along with the port, to perform kernel related
functions.
There has been substantial work done on MIB's for
DVMRP. The customer insisted that the port complied with these new
MIBs. This meant that a new chunk of instrumentation in the way of
SNMP sets and gets had to be added. There was additional instrumentation
that had to be added to allow for remote administration of Mrouted, to
include being able to dynamically turn on and off the IP Multicast routing
on a per port basis.
One of the biggest problems in porting the code
turned out to be the way memory was allocated. The original Mrouted
allocated all static data structures on the stack and only used heap memory
for dynamic memory. While this is perfectly valid for software running
on a workstation it creates problems inside of an embedded world.
Inside of the of embedded OS that the customer had each task had to declare
what it stack space would be. Knowing that there was significant
processing going on inside of Mrouted, a stack space of 4KB was allocated
to ensure that the stack would not be overrun. This would normally
be plenty of room except for the fact that all these static data structures
were allocated. The two that really stand out was the static transmit
and receive buffers. Each of these buffers were 8KB long and in global
space. Just these two buffers comprised four times our allocated
stack space. This was not noticed until some random behavior was
observed, which is a tell-tale sign of a stack overrun, and the resulting
investigation uncovered these data structures. After moving the data
structures off of the stack and onto the heap, the software no longer behaved
randomly. This just illustrates one of the complication of programming
in an embedded environment.
DORADO: After the completion of the Trout project, it was decided by management that IP Multicast functionality would be something that customers in the future would want and that the new Mrouted code would be a good starting point
It was decided that the Trout software would be ported
onto our development platform. and in this effort an abstraction layer
would be added to allow this next port to be as independent of the operating
system as possible. There is also an internal standard that the code
had to comply with. This internal standard included using SystemV
style mblk's as the buffer abstraction and that all protocol stacks would
use a stream interface (for a discussion on streams, see http://cm.bell-labs.com/cm/cs/who/dmr/st.html).
The internal standard came in handy during this
project because these were two places that were very dependent on the customer's
code during the Trout project. It is impossible to make functionality
like Mrouted completely independent of the underlying operating system,
but since the Dorado port was moving onto a system with VxWorks 5.3 running,
it was handy because VxWorks also uses mblks as the abstraction for its
buffers. This allowed the software to fit nicely into VxWorks and
also allow it to conform to the internal standard and thus be mostly OS
independent.
The other piece of the Trout code that had to be
changed was the configuration management. There were several places
in which the code expected to have access to data structures that our customer
had, and that do not exists in the scope of Dorado. Most of these
data structures included Task Control Blocks, Queue Control Blocks, and
Port Control Blocks. Luckily this turns out not to be much of a problem
because most of the configuration routines defined an API that the customer's
software used to tell the code what to do. After replacing the customer's
data structures with VxWorks data structures, the configuration routines
have most of the data that they need and now can be accessed through the
VxWorks shell. This will have to be the extent of the interface to
the Dorado Mrouted until Dorado has a user interface made for it.
DOCUMENTATION: The documentation for Dorado consists of a design
specification and a test plan. Both of these documents were based
on the Trout documentation. The design specification for Trout was
mostly a reiteration of the RFCs that covered DVMRP, IGMP, and the required
MIB support. However, there was a major portion of the design specification
that related how Mrouted was going to be integrated into the customers
router product.
There were three issues in the design specification
that had to be changed when the code was moved from Trout to Dorado.
First was how the code was going to get packets from the network.
Second was how the code was going to send packets onto the network, and
lastly was how the code was going to fit into the operating system.
Of course these three items where also the only places in the Trout design
specification were it was not just a reiteration of the relevant RFC's.
These three areas of the Dorado design specification now read that Mrouted
will interface with the network via a streams interface, as required, and
that the code will get its configuration from someone calling functions
from the VxWorks shell prompt. The test plan was much easier to write
since the basic Mrouted functionality has not changed and we were able
to use the original Trout test plan.
DEVELOPMENT ENVIRONMENT: The development environment during the Trout project is worth noting. In accordance with our customer's wishes, we choose Linux as the development environment. This mirrors how the customer does development and had a known good tool chain for Linux.
The Trout team commendeered a PC from one of the labs and replaced Windows with Linux. This PC became the file server, compile server, and the NFS server. The Linux PC kept all of the customer's supporting source code along with all of the code that was written/ported for Mrouted with RCS being used for source code control management. Since there was only a Linux tool chain, all compiling during the development cycle was handled by this PC. Eventually a Solaris version of the tool chain was delivered by the customer, but it was never installed or tested. The customer's router was able to download a runable image in RAM to execute via NFS. Once again the Linux PC came through by pulling triple duty as the NFS server for the three routers that the customer had provided. At one point in time there was five developers working full time on this PC and it kept a sustained load average about 0.2 - 0.5 and at its maximum was up for 40 days before a power outage forced it to shutdown. Since Dorado is using a MPC860 and not an i960, the development environment has moved back to Solaris since that is where the tool chain for the MPC860 lives. The source code control has also migrated from RCS to Clearcase, which is made by Atria Software.
CONCLUSION: Being involved with Trout and Dorado, I learned several things about being a software engineer in the current computer industry. On the technical side I learned how the MBONE works and how DVMRP is used to route IP multicast packets across the internet.
In conjunction with DVMRP, I received good exposure to IGMP version 1 and 2. This was my first experience writing a software design specification which was at first a daunting task but in retrospect it was not that difficult. Also this was my first experience in porting software of any notable size. While it was tempting for me to rewrite parts of Mrouted that I felt were not done correctly there simply was not enough time in the schedule to allow that to happen. In the end this probably was a better approach because the less that was changed, the greater the chance that it would correctly inter operate with other Mrouted running on UNIX workstations. After spending the last 18 weeks working with this technology, I have come to the realization that IP multicast has a big part to play in the evolution of the internet and I am glad I have gotten this exposure so early in my career.