Welcome to CS 293N!
Table of contents
Objective
In recent years, the application of Machine Learning (ML) tools has extended beyond traditional domains like image classification and Natural Language Processing (NLP) to tackle a variety of classification, detection, and control problems. More notably, the adoption of ML in addressing networking issues has emerged as a significant area of interest. However, implementing ML solutions for networked systems presents unique challenges that go beyond merely adapting tools designed for other fields. It necessitates a more thorough investigation to create ML-based networked systems that are flexible, scalable, and generalizable.
This course aims to delve into recent research efforts that have been successful in developing new ML tools and techniques specifically tailored for networked systems. We will explore studies published in prestigious networking, security, and ML conferences, such as USENIX NSDI, ACM SIGCOMM, NeurIPS, and ICML. Through this, students will learn to identify problems within networking that are amenable to ML solutions, select the appropriate tools or algorithms, and engage in interdisciplinary research that spans networking, ML, and systems.
The curriculum will also concentrate on understanding the decision-making processes involved in networking. This includes how network protocols or operators make decisions across various levels (network, TCP, application, etc.) to ensure network safety and performance. The course will examine the current heuristic-based decision-making frameworks and discuss how they can be replaced with ML-driven models.
Additionally, we will address the specific challenges of applying ML to networking problems and outline strategies to overcome them. Students will gain insights into the various stages of the ML pipeline, including data collection, pre-processing, representation, feature extraction, model selection, training, evaluation, and analysis. Emphasis will be placed on modifying these stages to develop models that are not just high-performing but also reliable, incorporating qualities such as trustworthiness, explainability, and robustness.
Background Expections
There are no official prerequisites; however, a basic familiarity with networking, ML, and distributed systems concepts will be very helpful.
You can refer to these slides from CS 190 N to catch up on the basics.
Course Expectations
In this course, the students are expected to do the following:
Before class: For most classes, we will be reading 1-2 papers. These papers will be assigned at least one week before the class. The students are expected to read the assigned research papers, and submit their reviews using the Google form (more details soon).
In-class: Students are expected to participate actively in the class. The instructor will lead the discussions, but the students are expected to elevate the discussions by explaining the key ideas, techniques used, experiments, and summarizing the key takeaways from the paper. Constructively contributing to the discussions, will require students to be prepared by doing the assigned preparation and thinking about the related problems deeply.
After class: Students will work in a team of 3-5 students to summarize the in-class discussions as blogs. For each class, a group will be assigned to produce scribe blogs for the discussed topics. For each topic, a team will be assigned to write-up a blog about the topic, which will be posted on the course site.
Term project: Work in a team (3-5 students) to work on projects that will contribute to the area of ML4Nets. Projects may contribute to open source implementations of data collection and analysis tools, ML algorithms, end-2-end systems for data-driven network management, etc. For each project, the students are expected to prepare a project report, which describes the research plan in detail. More details on this later.
Teams
You can work in a team for the term project. I will need the name of teams (and its members) by the end of second week (April 12, 2024). Please keep an eye out for potential team members during the introductions.
Evaluation
| Reviews | 30% |
| Class participation | 10% |
| Project (proposal) | 10% |
| Project (demo/results) | 20% |
| Project (report) | 20% |
| Project (presentation) | 10% |
Reviews
I expect students to read around 15 research papers as part of this course. The reading assignment would be made at least one week in advance.
Before the start of a lecture, you will submit Google form(s) (typically shared a week in advance). This form will require you to answer some of the questions related to the reading assignments. These questions will test how well you understood the assigned readings. For example, while reviewing a system’s paper, you might have to answer what specific techniques the paper used for evaluation, how the system can be improved, etc. Points will be awarded based on the level of depth demonstrated by your answers.
Writing an insightful review will require effort, so don’t plan to do the reading assignments at the last minute. The quality and depth matter the most, which takes time.
Class Participation
Active in-class participation is desired for this course. Points will be awarded based on your constructive contributions to the topics.
Term Project
One key aspect of this class is to have students to gain hands-on experiences in open research problems. To do this, each student will need to propose a research project. The teaching staff will provide the feedback on the proposal, and track the progress of each student. We will provide all the required computing resources to each team for their project.
Proposal
Your research proposal will report the problem statement, motivation, and give a rough idea of the related work and critical insights. More concretely, the project proposal must clearly mention the following aspects of their project:
- What is the motivation of the problem?
- What is the exact specification of the problem?
- What are some existing approaches to this problem?
- What are some existing datasets that you can work on?
- What is the novelty in your project? New problem? New approach? New dataset?
- How are you going to implement your approach, what are your metrics of success, and what’s your evaluation plan?
This document should be at most two pages of content; you may use extra pages for bibliographic references. Your proposal must follow the formatting requirements mentioned above. Your instructor will read your plan in detail and provide feedback, especially on whether the proposed idea has the potential to advance the state-of-the-art. Based on your instructor’s feedback, you will refine your proposal.
Note: The plans that don’t promise to advance the state-of-the-art will receive zero points, and ones that show promise of turning into a publication will receive extra credit. Note that a thorough reproduction or replication of an existing research artifact/paper could still be considered a novel contribution. In general, any well-motivated new methodology, algorithm, system/module, and analysis would be considered a novel contribution. Given the subjective nature of novelty, please consult with your instructor before committing yourself to any project.
FAQ Can I use my existing research projects/thesis research project as the project in this class? I would prefer students to get out of their comfort zone and try something new in this class. If you are using existing techniques from your existing project, it is unlikely that you will be able to learn anything new during the course project. However, you may still draw inspiration from your research problem to formulate your class project.
Results
In this part, you will need to quantify the success of your proposed approach. It is recommended that you show your key results and get the course instructor’s feedback before integrating them into the project report. Additionally, you are expected to create a public GitHub repository, providing sufficient details to facilitate the reproducibility of your efforts. Points will be deducted if the instructions are not self-sufficient and/or if the results are not reproducible.
Report
You must use this template for writing the report. The final report must be 6-12 pages long including references. It is encouraged to include the following components in your reports (not necessarily this order): abstract, introduction (motivation, task definition, your novel contributions), related work, your technical approach, such as math formulation of the problem, algorithms, theorems (if any), experiments, discussion, and conclusion.
Presentation
Coming soon.
Note: Use of LLMs
You can use Large Language Models (LLMs) such as GPT, Claude, etc., to enhance the quality of your text, whether for reviews, discussion summaries, or project reports. However, I would strongly advise against using them to answer the original questions in the review form. It is crucial for you to become extremely familiar with the papers you read, to the extent that you have a very good sense of how you would reproduce their key results. Over-relying on LLMs might save you some time, but it could hamper your learning experience.
Communication
Piazza
We will use Piazza for class communications. I have already sent an invitation to registered students. If you haven’t received it, please use this link to join the workspace. You can join using any @ucsb.edu email addres
s.
Etiquettes
To make sure that this is a helpful resource for everyone, please follow these guidelines when posting on Piazza:
- Before posting a question check/search through existing questions to see if someone already asked a similar question.
- Use the filters/folders (e.g., hw1, hw2, etc.) to see what questions were asked regarding specific assignments.
- As much as possible, keep related questions in the follow-up discussions, instead of creating new posts.
- Conversely, when posting your question/note, use the appropriate folders/labels.
- Include “Lab/HW## Q##” in the title of your post, e.g., Lab01 Q2.3.
Whenever possible, copy/paste the error as text instead of attaching a picture. Use Piazza’s code formatting if you post sample code. Follow the guidelines below to compose the content of your post.
If other students will benefit from the answer to your question (e.g., you are asking about a deadline or a policy), make the question public. We encourage you to make more of your posts public, making private posts only if something that’s too specific to your case or you need to share your code snipped with the instructors.
Do not post answers to homework/lab problems. Do not publicly share your code/solution.
Use Piazza instead of emailing the course instructor or mentors. (When composing a post, select “Individual students/instructors” (instead of “Entire Class”) and type “Instructors” to include all instructors). Use the “Instructor” posts instead of the individual messages (but follow #3 above).
Make sure that all your communication is professional, considerate, and productively contribute to the course. We expect you to abide by the University of California, Santa Barbara general code of conduct for all students. The UCSB policy applies to conduct over Zoom, Piazza, and any other digital tools being used for this course
- Guidelines for creating a well-structured post:
- You are going to get a faster and more helpful answer if you tell us:
- this is the specific question/function that I am working on
- this is the issue that I am running into,
- this is my code (make sure the message is private if you are directly sharing your code),
- here’s the input that I used in calling my functions,
- here’s the output/error I got,
- here’s what/where/how the output doesn’t match what I would have expected.
Course Policy
Cheating
Cheating is not tolerated. Please read the UCSB Academic Code of Conduct if you are not familiar with the definition of cheating. If you are caught cheating on an assignment, you will get a zero for that assignment. Other repercussions are also possible, such as being reported to the UCSB Office of Judicial Affairs.
Diversity and Inclusiveness
We (the instructor and the mentors) strive to create an environment in which students from diverse backgrounds and perspectives can be well-served in this course, where students’ learning needs can be addressed both in and out of class, and where the diversity that the students bring to this class is viewed as a resource, strength, and benefit. It is our intent to present materials and activities that are respectful of diversity: gender identity, sexuality, disability, age, socioeconomic status, ethnicity, race, nationality, culture, or religion (or lack thereof). Your suggestions are encouraged and appreciated: help us improve the effectiveness of the course for you personally, or for other students or student groups. If at any point you are made to feel uncomfortable, disrespected, or excluded by the course staff or fellow students, please, don’t hesitate to talk to us about it so that we may address the issue and maintain a supportive and inclusive learning environment. If you are uncomfortable bringing up an issue with the course staff directly, submit anonymous feedback or contact the Office of the Ombud. Help us create a welcoming, inclusive atmosphere that supports a diversity of thoughts, perspectives and experiences, and honors your identities.
These notes are inspired by and adopted from Mine Çetinkaya-Rundel, Hacker Hours, and Kevin Lin.
Disabled Students Program
If you are a registered DSP student and you intend to request a specific accommodation, you should inform me within the first two weeks of the quarter or of enrolling in the DSP. You should also submit your request no less than one week before the scheduled exam so that we can plan for any special arrangements.