From e8fbbf58ab4470bdc923c64525ad6ad232e67294 Mon Sep 17 00:00:00 2001 From: Duncan Date: Sun, 19 Jan 2020 19:17:13 -0500 Subject: [PATCH] Edits to add sources and writing changes --- TracingPaper.tex | 358 ----------------------------------------------- sigproc.bib | 16 +++ 2 files changed, 16 insertions(+), 358 deletions(-) delete mode 100644 TracingPaper.tex diff --git a/TracingPaper.tex b/TracingPaper.tex deleted file mode 100644 index 5df5dbb..0000000 --- a/TracingPaper.tex +++ /dev/null @@ -1,358 +0,0 @@ -% template-v1.tex: LaTeX2e template for Usenix papers. -% Version: usetex-v1, 31-Oct-2002 -% Revision history at end. - -\documentclass[XXX,endnotes]{usetex-v1} -% Choose the appropriate option: -% -% 1. workingdraft: -% -% For initial submission and shepherding. Features prominent -% date, notice of draft status, page numbers, and annotation -% facilities. The three supported annotation macros are: -% \edannote{text} -- anonymous annotation note -% \begin{ednote}{who} -- annotation note attributed -% text to ``who'' -% \end{ednote} -% \HERE -- a marker that can be left -% in the text and easily -% searched for later -% 2. proof: -% -% A galley proof identical to the final copy except for page -% numbering and proof date on the bottom. Annotations are -% removed. -% -% 3. webversion: -% -% A web-publishable version, uses \docstatus{} to indicate -% publication information (where and when paper was published), -% and page numbers. -% -% 4. finalversion: -% -% The final camera-ready-copy (CRC) version of the paper. -% Published in conference proceedings. This doesn't include -% page numbers, annotations, or draft status (Usenix adds -% headers, footers, and page numbers onto the CRC). -% -% If several are used, the last one in this list wins -% - -% -% In addition, the option "endnotes" permits the use of the -% otherwise-disabled, Usenix-deprecated footnote{} command in -% documents. In this case, be sure to include a -% \makeendnotes command at the end of your document or -% the endnotes will not actually appear. -% - -% These packages are optional, but useful -\usepackage{epsfig} % postscript figures -\usepackage{url} % \url{} command with good linebreaks -%\usepackage[showframe]{geometry} % http://ctan.org/pkg/geometry -\usepackage{graphicx} % http://catn.org/pkg/graphicx - -\begin{document} - -\title{Tracing SMB: Searching for Unknowns} - -% document status: submitted to foo, published in bar, etc. -\docstatus{Submitted to Cool Stuff Conference 2002} - -% authors. separate groupings with \and. -\author{ -\authname{Paul A.\ Wortman} -\authaddr{ECE} -\authaddr{University of Connecticut} -\authaddr{ Storrs, CT, 06279} -\authurl{\url{paul.wortman@engr.uconn.edu}} -\authurl{\url{http://host.dom/yoururl}} -%\and -%\authname{Name Two} -%\authaddr{Two's Institution} -%\authurl{\url{two@host.dom}} -% -} % end author - -\maketitle - -\begin{abstract} -Traces are an important and necessary part of systems research because this works leads to a better understanding of the behavior of protocols, architectures, or even entire networks. Further more the lessons learned through examining these traces can lead to the development of better benchmarks which will in turn allow for more accurate testing and advancement of technologies and their performance. Some key findings found from the examination of CIFS traces include \textbf{ADD ONCE KEY FINDINGS HAVE BEEN FOUND}. -\\ -%NOTE: Perhaps bring up at the end of the paper when mentioning why trace work is important in a grander scheme -%With any sort of benchmark, there are inherent oversimplifications that are taken into account when first designing these watermarks for advancing technology. In the case of networking benchmarks, many of these simplifications occur when dealing with the low level operations of the system; spatial/temporal scaling, timestamping, IO and system behavior. While these simplifications were acceptable for past systems being tested, this facile outlook is no longer acceptable for supplying worthwhile information. Without taking into account the intricacies of current day machines, technology will only be able to progress in the avenues that we know of, while never being able to tackle the bottlenecks that are made apparent through more accurate benchmarking. -\end{abstract} - -\section{Introduction} -\label{Introduction} -Traces are important for the purpose of developing and taking accurate metrics of current technologies. One must determine which aspects of the trace are most representative of what occurred during the tracing of the system, while figuring out which are represntative of the habits and patterns of said system. This discovered information is used to produce a benchmark, either by running a repeat of the captured traces or by using synthetic benchmark created from the trends detailed within the captured tracing data~\cite{Anderson2004}. - -As seen in previous trace work done [Leung et al, Ellard et al, Roselli et al], the general perceptions of how computer systems are being used versus their initial purpose have allowed for great strides in eliminating actual bottlenecks rather than spending unnecessary time working on imagined bottlenecks. Leung's \textit{et. al.} work led to a series of obervations, from the fact that files are rarely re-opened to finding that read-write access patterns are more frequent ~\cite{Leung2008}. Without illumination of these underlying actions (e.g. read-write ratios, file death rates, file access rates) these issues can not be readily \& effectively tackled. -\\ -%\textbf{NOT SURE IF KEEP OR NEEDED} I/O benchmarking, the process of comparing I/O systems by subjecting them to known workloads, is a widespread pratice in the storage industry and serves as the basis for purchasing decisions, performance tuning studies, and marketing campaigns ~\cite{Anderson2004}. - -The purpose of my work is to tackle this gap and hopefully bring insight to the complexity of network communication through the examination of CIFS network traffic. - -\subsection{Issues with Tracing} -\label{Issues with Tracing} -\textbf{REWORD TO REMOVE MENTION OF BENCHMARKS}\\ -The majority of benchmarks are attempts to represent a known system and structure on which some “original” design/system was tested. While this is all well and good, there are many issues with this sort of approach; temporal \& spatial scaling concerns, timestamping and buffer copying, as well as driver operation for capturing packets~\cite{Orosz2013,Dabir2008,Skopko2012}. Each of these aspects contribute to the inital problems with dissection and analysis of the captured information. For example, inaccuracies in scheduling I/Os may result in as much as a factor of 3.5 differences in measured response time and factor of 26 in measured queue sizes; differences that are too large to ignore~\cite{Anderson2004}. - -Temporal scaling refers to the need to account for the nuances of timing with respect to the run time of commands; consiting of computation, communication \& service. A temporally scalable benchmarking system would take these subtleties into account when expanding its operation across multiple machines in a network. While these temporal issues have been tackled for a single processor (and even somewhat for cases of multi-processor), these same timing issues are not properly handles when dealing with inter-network communication. Inaccuracies in packet timestamping can be caused due to overhead in generic kernel-time based solutions, as well as use of the kernel data structures ~\cite{Orosz2013,PFRINGMan}. \\ -Spatial scaling refers to the need to account for the nuances of expanding a benchmark to incorporate a number of (\textbf{n}) machines over a network. A system that properly incorporates spatial scaling is one that would be able to incorporate communication (even in varying intensities) between all the machines on a system, thus stress testing all communicative actions and aspects (e.g. resource locks, queueing) on the network. - -\subsection{Previous Advances Due to Testing} -\label{Previous Advances Due to Testing} -Tracing collection and analysis has proved its worth in time from previous studies where one can see important lessons pulled from the research; change in behavior of read/write events, overhead concerns originating in system implementation, bottlenecks in communication, and other revelations found in the traces. \\ -Previous tracing work has shown that one of the largest \& broadest hurdles to tackle is that traces (and benchmarks) must be tailored (to every extent) to the system being tested. There are always some generalizations taken into account but these generalizations can also be a major source of error~\cite{Ellard2003,EllardLedlie2003,Anderson2004,Orosz2013,Dabir2008,Skopko2012,Vogels1999,Traeger2008,Ruemmler1993}. To produce a benchmark with high fidelity one needs to understand not only the technology being used but how it is being implemented within the system being traced \& benchmarked~\cite{Roselli2000,Traeger2008,Ruemmler1993}. All of these aspects will lend to the behavior of the system; from timing \& resource elements to how the managing software governs actions~\cite{Ellard2003,EllardLedlie2003,Douceur1999}. Further more, in pursuing this work one may find unexpected results and learn new things through examination~\cite{Leung2008,Ellard2003,Roselli2000}. \\ -These studies are required in order to evaluate the development of technologies and methodologies along with furthering knowledge of different system aspects and capabilities. As has been pointed out by past work, the design of systems is usually guided by an understanding of the file system workloads and user behavior~\cite{Leung2008}. It is for that reason that new studies are constantly performed by the science community, from large scale studies to individual protocol studies~\cite{Leung2008,Ellard2003,Anderson2004,Roselli2000,Vogels1999}. Even within these studies, the information gleaned is only as meaningful as the considerations of how the data is handled. \\ - -A detailed overview of the tracings and analysis system can be seen in section ~\ref{Trace Collection}. - -\subsection{Contributions} -\label{Contributions} -Out of all the elements that make up the tracing system used for this research, there are a few key aspects that are worth covering due to their uniqueness within the system. These key components of the tracing system are the use of PF\_RING to mitigate timing and resource concerns, the use of proper hardware and software to handle incoming data, along with the tweaking of DataSeries code to create analysis tools for the captured data. \\ -% PF\_RING section -The addition of PF\_RING lends to the tracing system by minimizing the copying of packets which, in turn, allows for more accurate timestamping of incoming traffic packets being captured ~\cite{Orosz2013,Skopko2012,PFRING,PFRINGMan}. PF\_RING acts as a kernel module which allows for kernel-based capture and sampling that limits packet loss and timestamping overhead leading to faster packet capture while efficiently preserving CPU cycles ~\cite{PFRING}. This aids in minimizing packet loss/timestamping issues by not passing packets through the kernel data structures~\cite{PFRINGMan}. The other reason PF\_RING is instrumental is that it functions with the 10Gb/s hardware that was installed into the Trace1 server; allowing for full throughput from the network tap on the UITS system. \\ -% DataSeries + Code section -The tweaks and code additions to the existing DataSeries work are filtering for specific CIFS/SMB protocol fields along with the writing of analysis tools to parse and dissect the captured packets. Interesting fields were chosen to be kept for analysis. It should be noted that this was done arbitrarily and changes/additions have been made as the value of certain fields are determined to be worth examining. The code written for analysis of the captured DataSeries format packets focuses on read/write events, ID tracking (PID/MID/TID/UID), and OpLock information. The future vision for this information is to combine ID tracking with the OpLock information in order to track resource sharing of the different clients on the network. - -%\subsection{My Work} -%\label{My Work} -%Certain elements of this research are purposed to improve tracing knowledge along with general understanding of networked systems. One such element is the delevopment of tracking IO inter-arrival times along with processor times. This allows for more realistic replay of commands run due to more complete temporal considerations; time taken for computation and "travel" time. Another elements is the PID/MID/TID/UID tracking which allows for following command executions between a given client and server. This element paired with the previous development helps expand the understanding and replay-ability of temporal scaling. -%Things to make sure to say: Need trace of CIFS behavior. -%The following are issues that our work hopes to alleviate: there has been no large scale study done on networks for some time, there has been no study on CIFS(Common Internet File System)/SMB(Server Message Block) protocols for even longer, and most importantly these studies have not tackled lower level aspects of the trace, such as spacial \& temporal scaling idiosyncrasies of network communication. It is for these reasons that we have developed this tracing system and have developed new studies for lower level aspects of communication network. \\ - -\section{Trace Collection} -\label{Trace Collection} - -\subsection{Procedure} -\label{Procedure} - -\subsubsection{Capture} -\label{Capture} -The packet capturing aspect of the tracing system is fairly straight forward. On top of the previously mentioned alterations to the system (e.g. PF\_RING), the capture of packets is done through the use of \textit{tshark}, \textit{pcap2ds}, and \textit{inotify} programs. The broad strokes are that incoming SMB/CIFS information comes from the university's network. All packet and transaction information is passed through a duplicating switch that then allows for the tracing system to capture these packet transactions over a 10 Gb port. The reason for using 10Gb hardware is to help ensure that the system is able to capture and all information on the network. These packets are passed along to the \textit{tshark} packet collection program which records these packets into a cyclical capturing ring. A watchdog program (\textit{inotify}) watches the directory where all of these packet-capture (pcap) files are being stored. As a new pcap file is completed \textit{inotify} passes the file to \textit{pcap2ds} along with what protocol is being examined (i.e. SMB). The \textit{pcap2ds} program reads through the given pcap files, filters out any data fields deemed important or interesting for the passed protocol type, then the results are written in DataSeries format; these compressed files are then collected and stored. Due to the fundamental nature of this work, there is no need to track every piece of information that is exchanged, only that information which illuminates the behavior of the clients \& servers that function over the network (e.g. read \& write transactions). It should also be noted that all sensitive information being captured by the tracing system in encrypted to proect the users whose information is be examined by this tracing system. - -\subsubsection{Collection} -\label{Collection} -The collection of these files is rather straight forward. Once the DataSeries files have been collected to an arbitrary amount (in this case 100,000 files), these files are then moved off of the tracing system and are stored on a more secure \textit{/trace-store/} machine. This storage location is only accessable from the trace system along with RAIDing on its disk to protect against data loss of the collected DataSeries files. These files are then used in analysis to determine the behavior on the university network. - -\subsubsection{Dissection/Analysis} -\label{Dissection/Analysis} -The trace analysis in performed by an analysis module code that both processes the DataSeries files for extraction of information but also outputs meaningful information (such as IO patterns) to a file that can be used for further analysis. This section of the tracing system is always growing and changing as discoveries and limitations are found during the continuous execution of this code. Alterations range from edits to speed up the analysis process to adjustments to how communications are tracked and interpreted. This analysis will eventually incorporate oplocks and other aspects of resource sharing on the network to gain a more complete picture of the network's usage and bottlenecks. - -\subsection{ID Tracking} -\label{ID Tracking} -All comands sent over the network are coupled to an identifying MID/PID/TID/UID tuple. This identifying tuple is coupled with FID information for the purpose of tracking the different files that are exchanged/written over the network. The purpose for this tracking is for profiling the habits of individual users. While initially simplistic (drawing a connection between FIDs and tuple IDs), this aspect of the research is easily expandable to other facets of communication. -\\One could also track the amount of sharing that is occurring between users. The PID is the process identifier and the MID is the multiplex identifier, which is set by the client and is to be used for identifying groups of commands belonging to the same logical thread of operation on the client node. -\\The per client process ID can be used to map the activity of given programs, thus allowing for finer granularity in the produced benchmark (e.g. control down to process types ran by individual client levels). Other features of interest are the time between an open \& close, or how many opens/closes occurred in a window (e.g. a period of time). This information could be used as a gauge of current day trends in filesystem usage \& its consequent taxation on the surrounding network. It would also allow for greater insight on the r/w habits of users on a network along with a rough comparison between other registered events that occur on the network. Lastly, though no less important, it would allow us to look at how many occurrences there are of shared files between different users. One must note that the type of sharing may differ and there can be an issue of resource locking (e.g. shared files) that needs to be taken into account. This is preliminarily addressed by monitoring any oplock flags that are sent for read \& writes. This information also helps provide a preliminary mapping of how the network is used and what sort of traffic populates the communication. - -\begin{figure*} - \includegraphics[width=\textwidth,height=4cm]{./images/SMBHeader.jpg} - \caption{SMB Header Specification.} - \label{fig:SMBSpec} -\end{figure*} - -\subsection{SMB} -\label{SMB} -Server Message Block (SMB) is the modern dialect of Common Internet File System (CIFS). The most important aspect of SAMBA (e.g. SMB) is that it is a stateful protocol , i.e. one where the information being sent via SMB has identifying fields that allow for process ID tracking. -\\The structure for sending message payloads in SMB is as follows: each SMB message is split into three blocks. The first block is a fixed-length SMB header. The second block is made up of two variable-length blocks called the SMB parameters. The third block is made up of the SMB data. Depending on the transaction occurring these different blocks are used in different manners. The purpose of the SMB header is particularly important because the header identifies the message as an SMB message payload~\cite{MS-CIFS} (as shown in Figure~\ref{fig:SMBSpec}). The most important aspects of the SMB header, which the tracing system constantly examines, are the PID/MID tuple (for the purpose of identifying a client/server) and the commands value which is passed (notifying our tracing system of the actions taking place on the network). It is through this command field that the process ID tracking system is able to follow the different commands (read/write/general event) that occur and try to find patterns in these network communications. - -\subsection{System Limitations and Challenges} -\label{System Limitations and Challenges} -When initially designing the tracing system used in this paper, different aspects were taken into account, such as space limitations of the tracing system, packet capture limitations (e.g. file size), and speed limitations of the hardware. One limitation encountered in the packet capture system deals with the functional pcap (packet capture file) size. The concern being that the pcap files only need to be held until they have been filtered for specific protocol information and then compressed using the DataSeries format, but still allow for room for the DataSeries files being created to be stored. Other limitation concerns came from the software and packages used to collect the network traffic data~\cite{Orosz2013,Dabir2008,Skopko2012}. These ranged from timestamp resolution provided by the tracing system's kernel~\cite{Orosz2013} to how the packet capturing drivers and programs (such as dumpcap and tshark) operate along with how many copies are performed and how often. The speed limitations of the hardware are dictated by the hardware being used (e.g. GB capture interface) and the software that makes use of this hardware (e.g. PF\_RING). After all, our data can only be as accurate as the information being captured~\cite{Ellard2003,Anderson2004}. -An other concern was whether or not the system would be able to function optimally during periods of high network traffic. All apsects of the system, from the hardware to the software, have been altered to help combat these concerns and allow for the most accurate packet capturing possible. \\ - -%About Challenges of system -While the limitations of the system were concerns, there were other challenges that were tackled in the development of this research. -One glaring challenge with building this tracing system was using code written by others; tshark \& DataSeries. While these programs are used within the tracing structure there are some issues when working with them. These issues were unfortunately so inherent to the code that hacks and workarounds were developed to minimize their effect. Other challenges centralize around selection, interpretations and distribution scope of the data collected. A researcher must select which information is important or which information will give the greatest insight to the workings on the network. Too little information can lead to incorrect conclusions being drawn about the workings on the system, while too much information (and not knowing which information is pertinent) can lead to erroneous conclusions as well. There is a need to strike a balance between what information is important enough to capture (so as not to slow down the capturing process through needless processing) while still obtaining enough information to acquire the bigger picture of what is going on. Every step of the tracing process requires a degree of human input to decide what network information will end up providing the most complete picture of the network communication and how to interpret that data into meaningful graphs and tables. This can lead to either finds around the focus of the work being done, or even lead to discoveries of other phenomena that end up having far more impact on the overall performance of the system~\cite{Ellard2003}. - -%About interpretation of data -To some degree these interpretations are easy to make (e.g. file system behavior \& user behavior~\cite{Leung2008}) while others are more complicated (e.g. temporal scaling of occurances of read/write), but in all scenarios there is still the requirment for human interpretation of the data. While having humans do the interpretations can be adventageous, a lack of all the "background" information can also lead to incorrectly interpreting the information. The trends and habits of the network will change based on the time of year, time of day, and even depend on the exam schedule. Truly interesting examination of data requires looking at all different periods of time to see how all these factors play into the communications of the network. - -\section{Trace Analysis} -\label{Trace Analysis} -The trace analysis is performed by an AnalysisModule code that both processes the DataSeries(ds) files for extraction of information to an event\_data structure and also outputs meaningful information to a file that can be used for further analysis. - -\subsection{System Information and Predictions} -\label{System Information and Predictions} -It is important to detail out any benchmakring system so that when the results of one's research are being examined, they can be properly understood with the correct background information and understanding that lead the originating author to those results~\cite{Traeger2008}. The following is an explination the UITS system from which Trace1 pulls it's packet traffic information along with predicitions of how the data will look along with the reasoning behind the shape of the information. - -The UITS system consisnts of five Microsoft file server cluster nodes. These blade servers are used to host home directories for all UConn users within a list of 88 departments. These users include facultiy, staff and students, along with at lest one small group of users. Each server is capable of handling 1Gb/s of traffic in each direction (e.g. outbound and inbound traffic). All together the five blade server system can in theory handle 10Gb/s of recieving and tranmitting data, but only four of the servers should be active at any given time. Some of these blade servers have local storage but the majority do not have any. To the understanding of this paper, the blade servers are purposed purely for dealing with incoming traffic to the SAN storage nodes that sit behind them. This system does not currently implement load balancing, instead the servers are set up to spread the traffic load among four of the active cluster nodes while the fifth node is passive and purposed to take over in the case that any of the other nodes go down (e.g. become inoperable or crash). \\ - -From the information at hand I theorized the following behaviors and attributes that would be seen in the system. First are the predictions based on what was learned from talking to people within UITS, after that are my general predictions. - -From conversations with UITS, the understanding of the file server system behavior is that there are spikes of traffic that tend to happen during the night time. Our assumption is that the majority of this traffic will occur between 2am and 6am because this is when backups occur to the SAN system, while additional back-up traffic for application servers will occur between 12am and 4am; whose starting time can vary. These application server back-ups are performed incrementally, starting with a random 50 servers, and are performed with device-specific configurations. -It is important to note that, however, it is not expected that we would see any of this traffic as the traffic would be encrypted; therefore the protocol used is not the SMB/CIFS protocol that is being analyzed by this paper. However, while the SAN back-up may be encrypted, other application server back-ups may still appear in communications. -Furthermore, any traffic that does occur during the duration of "day time hours" (i.e. 9am to 5pm) would be solely due to the actions taken by the users of this system (e.g faculty, staff, students). If there is an automatic backup manager we would expect to see traffic caused by it pulling cached information from the machine of users across the network. \\ -One general assumption is that these blade servers should not ever fail, thus the greatest transfer rate observed should be 8Gb/s (i.e. 1Gb/s per server). If there is a greater rate than that this means that the fifth server is aiding rather than acting as backup should another server fail. \\ - -During the three weeks of data observation there was a moritorium on changes/alterations to the system, therefore all observed traffic is assumed to only be regualr esage and software managing network communications. - -\textit{\textbf{SMB Expectations}}: SMB will be heavily used by students to access their network accounts from any networked computer, along with network access to shared file systems and connected printers. Oplocks will be in heavy use and cause a slowdown of the system for multiuser shared storage space. Authentication of network computers could bottleneck during moments of high traffic (e.g. students all logging in for a class). - -Expectation would dicatate that there will be spikes of activity the week leading up the beginning of the semester, the first few days after the semester starts, and during any moments of maintenance that make use of the network; valleys/low points of activity right before (2 days leading up to) the beginning of the semester, holidays (9/1 - labor day), and general progression of less traffic as the semester progresses. Spikes would be expected to be moments either when new files are being added onto the system (mainly writes), or when a large number of users are attempting to access files (mainly reads). Ditches would most likely show periods of low to no activity, although both the IO \& Bytes graphs should be examined to confirm inactivity rather than something else. - -\begin{figure*} - \includegraphics[width=\textwidth,height=4cm]{./images/totalBufSeen.png} - \caption{Summation of Buffer Size Occurrences.} - \label{fig:totalBuf} -\end{figure*} - -\subsection{Run Patterns} -\label{Run Patterns} -Interpreting the data collected is done by producing three separate graphs showing three important areas of interest for being able to understand the traffic being examined. These areas are: \textbf{1)} the total number of read/write IO events, \textbf{2)} the occurrrences of different file sizes being read/written, \textbf{3)} the total number of bytes (combined read \& write) that are communicated over the traffic captured by the trace1 system. The reason for needing these three areas of information is that combined together one is able to better interpret all the collected information. By comparing the byte traffic to the IO information this is how we are able to tell not only when the times of greatest traffic are but which types of IO interactions dominate these periods. It should be noted that unfortunately the analysis program does not include a granularity to allow knowledge on whether the read, or write, events are responsible for the most data transfer (via communication) but that is one of the many future additions to this work. From early examinations the data leads us to interpret that the majority of reads are due to client exploring the shared drives \& queries about directories and directory contents, along with users copying large directories of data (most likely for their own purposes). The file size information allows for interpretation of the size of the information that is being passed between the UITS servers and clients (e.g. buffer sizes). Although the granualrity for this information is far corser (24 hours versus the tracked 15 minute time window) it still shows which variations of data lengths were most encountered over the period of the given day. This information coupled with the byte and IO information reflects a priliminary protrait of how the UITS file server system is used, which can be compared to the internal network information that the UITS department keeps for their own maintenance and troubleshooting purposes. The hope is to essentially replicate the data that UITS keeps for their own records. - -\subsection{Locating Performance Bottlenecks} -\label{Locating Performance Bottlenecks} -When examinging the data produced from this research, one has to look for a limitation or "caping" of the data captured. These limiting points represent times when the total amount of data throughput, IO events managed, number of clients, etc. reaches some cap (or high point) where the system is unable to push harder (e.g. more throughput, more IO events, greater number of clients). Traditionally these bottlenecks are due to physical limitations of a system, but can just as easily come from human-based sources (e.g. implemnentation, bad code writing, unexpected raceconditions or user activity). - -\section{Intuition Confirm/Change} -\label{Intuition Confirm/Change} -In order to interpret the data being analyzed and dissected, the first step was to understand how to pair the byte throughput and IO event frequency into an understanding of the system. This was achieved by including examination of the data relative to the surrounding behavior. Pairing the information in this manner shows not only the bytes \& IO behavior but preliminary understanding of how much throughput is being generated by each IO event; giving an outline of client behvaior on the system. This information compared with the buffer sizes allows for improved interpretations of the network communication information.\\ - -\begin{figure*} - \includegraphics[width=\textwidth,height=4cm]{./images/IOWeek.png} - \caption{R/W IO over three weeks.} - \label{fig:IOWeek} -\end{figure*} - -\begin{figure*} - \includegraphics[width=\textwidth,height=4cm]{./images/BytesWeek.png} - \caption{Btyes throughput over three weeks.} - \label{fig:BytesWeek} -\end{figure*} - -\begin{figure*} - \includegraphics[width=\textwidth,height=4cm]{./images/IOWeekZoom.png} - \caption{Zoom in on IOs seen.} - \label{fig:IOWeek} -\end{figure*} - -\begin{figure*} - \includegraphics[width=\textwidth,height=4cm]{./images/BytesWeekZoom.png} - \caption{Zoom in on bytes seen.} - \label{fig:BytesWeekZoom} -\end{figure*} - -\subsection{Characterizations of Different Behaviors} -\label{Characterizations of Different Behaviors} -Different behavioral situations (seen using pairing of bytes and IO graphs/behavior) are imaged as follows: -\textit{Assumptions:} -\begin{itemize} -\item Bytes - Amount of data being passed -\item IOs - Number of interactions occurring on network -\end{itemize} -Three scenarios observed were (1) a large number of IOs and small number of bytes, (2) a small number of IOs and large number of bytes, and (3) similar number of IOs and bytes transferred. This data is interpreted as follows, the number of bytes directly correlates to the data being transferred over the wire and the number of IOs correlates to the number of clients interacting over the network. Scenario (1) has high client interaction where expected bottlenecks would be due to management software dealing with the large volume of client interactions. Race conditions and age of requests could be an issue, but ideally OpLocks and SMB’s internal management tackle these issues. Scenario (2) has high traffic throughput, where any bottleneck would most likely be due to physical limitations of the system; wires, switches, etc. The last scenario (3) has equal interactions of each, where if both are high the system will taxed and bottlenecks could occur in multiple aspects or both are low and the system will be in a relaxed state. -It was discovered that the median buffer size exchanged over the network was 4096 bytes (i.e. 4K). This leads to the understanding that the majority of any network-based bottlenecks will be due to a large number of small communications. Paired with the knowledge that the interacting systems are individually configured supports that a good deal of "back-and-forth" traffic is being created during back-up processes. - -\section{Conclusion} -\label{Conclusion} -On the outset of this work it was believed that the data collected and analyzed would follow similar behavior patterns seen in previous papers ~\cite{Douceur1999, RuemmlerWilkes1993, Bolosky2007, EllardLedlie2003}. The expectation is that certain aspects of the data, such as transfer/buffer sizes, will produce a bell-shape and be centralized around a larger size than previous papers' findings. The number of I/O operations was expected to peak during nocturnal hours and fall during day time hours. On top of that the expectation is that a greater number of reads will be seen over the course of a day, where the majority of writes will occur near the expected times of UITS' backup (e.g. 2am to 6am). Granted, one must recall that the expectation is that any backup traffic that is seen will be due to a fetching of user's caches in order to preserve fidelity of any shared data (e.g. traffic produced by incremental back-ups).\\ -One oddity was that during the day one would see a greater increase in writes instead of reads. The first assumption was that this is due to the system and how users interact with everything. -I believe that the greater number of writes comes from students doing intro work for different classes in which students are constantly saving their work while reading instructions from a single source. The early traffic is most likely due to professors preparing for classes. A majority of the late night/early morning write traffic is due to the back-ups occurring, though these can occur through-out the day. One must also recall that this data itself has limited interpretation because only a small three week windows of information is being examined. A better, and far more complete, image can be constructed using data captured from the following months, or more ideally, from an entire year's worth of data. Another limitation of the results is the scope of the analysis is curbed and does not yet fully dissect all of the fields being passed in network communication. -The future work of this project would be to incorporate more fields from the SMB/CIFS specifications in order to depict a fuller picture of the network system. Additional alterations would be a work-around allowing for DataSeries files that incorporate an entire day's worth of trafafic, for the purpose of distribution of the code, modulation of the captruing software to incorporate additional protocols, better automation of code elements in order to reduce the potential for human error causing loss of data, and incorporation of DataSeries version changes to tackle data primitive translation issues. - -The hope is to further the progress made with benchmarks \& tracing in the hope that it too will lend to improving and deepening the knowledge and understanding of these systems so that as a result the technology and methodology is bettered as a whole. - -%references section -%\bibliographystyle{plain} -%\bibliography{body} - -\begin{thebibliography}{99} - -\bibitem{Leung2008} Andrew W.~Leung and Shankur Pasupathy and Garth Goodson and Ethan L.~Miller, -\emph{Measurement and Analysis of Large-Scale Network File System Workloads}, USENIX Annual Technical Conference (June 2008) - -\bibitem{Ellard2003} Daniel Ellard and Margo Seltzer, \emph{NFS Tricks and Benchmarking Traps}, -USENIX Annual Technical Conference (2003) - -\bibitem{EllardLedlie2003} Daniel Ellard and Jonathan Ledlie and Pia Malkani and Margo Seltzer, \emph{ -Passive NFS Tracking of Email and Research Workloads}, 2nd USENIX Conference on File and Storage Technologies (2003) - -\bibitem{Anderson2004} Eric Anderson and Mahesh Kallahalla and Mustafa Uysal and Ram Swaiminnathan, \emph{ -Buttress: A Toolkit for Flexible and High Fidelity I/O Benchmarking}, 3rd USENIX Conference of File and Storage Technologies (April 2004) - -\bibitem{Orosz2013} P\'{e}ter Orosz and Tam\'{a}s Skopk\'{o}, \emph{ -Multi-threaded Packet Timestamping for End-to-End QoS Evaluation}, The Eighth International Conference on Systems and Networks Communications (2013) - -\bibitem{Dabir2008} Abes Dabir and Ashraf Matrawy, \emph{ -Bottleneck Analysis of Traffic Monitoring using Wireshark}, IEEE (2008) - -\bibitem{Narayan2010} Sumit Narayan and John A. Chandy, \emph{ -I/O Characterization on a Parallel File System}, International Symposium on Performance Evaluation of Computer and Telecommunication Systems (2010) - -\bibitem{Skopko2012} Tam\'{a}s Skopk\'{o}, \emph{ -Loss Analysis of the Software-based Packet Capturing}, Carpathian Journal of Electronic and Computer Engineering 5 (2012) - -\bibitem{MS-CIFS} \emph{Common Internet File System (CIFS) Protocol}, url{http://msdn.microsoft.com/en-us/library/ee442092.aspx} - -\bibitem{MS-SMB} \emph{Server Message Block (SMB) Protocol}, url{http://msdn.microsoft.com/en-us/library/cc246231.aspx} - -\bibitem{MS-SMB2} \emph{Server Message Block (SMB) Protocol Versions 2 and 3}, url{http://msdn.microsoft.com/en-us/library/cc246482.aspx} - -\bibitem{Roselli2000} Drew Roselli and Jacob R. Lorch and Thomas E. Anderson, \emph{ -A Comparison of File System Workloads}, Proceedings of 2000 USENIX Annual Technical Conference (June 2000) - -\bibitem{Vogels1999} Werner Vogels, \emph{ -File system usage in Windows NT 4.0}, Proceedings of the seventeenth ACM symposium on Operating systems principles (December 1999) - -\bibitem{Meyer2012} Dutch T. Meyer and William J. Bolosky, \emph{ -A Study of Practical Deduplication}, ACM Transactions on Storage (January 2012) - -\bibitem{PFRING} \emph{PF\_RING High-speed packet capture, filtering and analysis}, url{http://www.ntop.org/products/pf\_ring/} - -\bibitem{PFRINGMan} \emph{PF\_RING User Guide}, -url{https://svn.ntop.org/svn/ntop/trunk/PF\_RING/doc/UsersGuide.pdf} - -\bibitem{Traeger2008} Avishay Traeger and Erez Zadok and Nikolai Joukov and Charles P.~Wright, \emph{ -A Nine Year Study of File System and Storage Benchmarking}, ACM Transactions on Storage (May 2008) - -\bibitem{Kavalanekar2009} Swaroop Kavalanekar and Dushyanth Narayanan and Sriram Sankar and Eno Thereska and Kushagra Vaid and Bruce Worthington, \emph{ -Measuring Database Performance in Online Services: A Trace-Based Approach}, Performance Evaluation and Benchmarking (2009) - -\bibitem{Douceur1999} John R.~Douceur and William J.~Bolosky, \emph{ -A Large-Scale Study of File-System Contents}, Proceedings of the 1999 ACM SIFMETRICS international conference on Measurement and modeling of computer systems (June 1999) - -\bibitem{Ruemmler1993} Chris Ruemmler and John Wilkes, \emph{ -UNIX disk access patterns}, Winter USENIX 1993 (January 1993) - -\bibitem{RuemmlerWilkes1993} Chris Ruemmler and John Wilkes, \emph{ -A trace-driven analysis of working set sizes}, Hewlett-Packard Laboratories (5 April 1993) - -\bibitem{Bolosky2007} Nitin Agrawal and William J.~Bolosky and John R.~Douceur and Jacob R.~Lorch, \emph{ -A Five-Year Study of File-System Metadata}, ACM Transactions on Storage (TOS) Volume 3 Issue 3 (October 2007) - -\end{thebibliography} - -\end{document} - -% Revision History: -% designed specifically to meet requirements of -% TCL97 committee. -% originally a template for producing IEEE-format articles using LaTeX. -% written by Matthew Ward, CS Department, Worcester Polytechnic Institute. -% adapted by David Beazley for his excellent SWIG paper in Proceedings, -% Tcl 96 -% turned into a smartass generic template by De Clarke, with thanks to -% both the above pioneers -% use at your own risk. Complaints to /dev/null. -% make it two column with no page numbering, default is 10 point - -% Munged by Fred Douglis 10/97 to separate -% the .sty file from the LaTeX source template, so that people can -% more easily include the .sty file into an existing document. Also -% changed to more closely follow the style guidelines as represented -% by the Word sample file. -% This version uses the latex2e styles, not the very ancient 2.09 stuff. -% - -% Revised July--October 2002 by Bart Massey, Chuck Cranor, Erez -% Zadok and the FREENIX Track folks to ``be easier to use and work -% better''. Hah. Major changes include transformation into a -% latex2e class file, better support for drafts, and some -% layout improvements. -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -% for Ispell: -% LocalWords: workingdraft BCM ednote SubSections xfig SubSection joe diff --git a/sigproc.bib b/sigproc.bib index fbbd6b8..95501e9 100644 --- a/sigproc.bib +++ b/sigproc.bib @@ -741,4 +741,20 @@ year={2016} pages={1802--1813}, year={2012}, publisher={VLDB Endowment} +} + +@misc{SMB1Spec, + author="Microsoft Corporation", + title="[{MS-SMB}]Server Message Block ({SMB})Protocol", + url="https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb/f210069c-7086-4dc2-885e-861d837df688", + month=sep, + year="2018" +} + +@misc{SMB2Spec, + author="Microsoft Corporation", + title="[{MS-SMB2}]Server Message Block ({SMB})Protocol Versions 2 and 3", + url="https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/5606ad47-5ee0-437a-817e-70c366052962", + month=sep, + year="2019" } \ No newline at end of file