A Trace-Based Study of SMB Network File System Workloads in an Academic Enterprise

Abstract—Storage system traces are important for examining real-world applications, studying potential bottlenecks, as well as driving benchmarks in the evaluation of new system designs. While file system traces have been well-studied in earlier work, it has been some time since the last examination of the SMB network file system. The purpose of this work is to continue previous SMB studies to better understand the use of the protocol in a real-world production system in use at the University of Connecticut. The main contribution of our work is the exploration of I/O behavior in modern file system workloads as well as new examinations of the inter-arrival times and run times for I/O events. We further investigate if the recent standard models for traffic remain accurate. Our findings reveal interesting data relating to the number of read and write events. We notice that the number of read events far exceeds writes and that the average of bytes transferred over the wire is greater for reads as well. Furthermore we find an increase in the use of metadata for overall network communication that can be taken advantage of through the use of smart storage devices.

I. Introduction

Over the last twenty years, data storage provisioning has been centralized through the use of network file systems. The architectures of these storage systems can vary from storage area networks (SAN), network attached storage (NAS), clustered file systems, hybrid storage, amongst others. However, the front-end client-facing network file system protocol in most enterprise IT settings tends to be, for the most part, solely SMB (Server Message Block) because of the preponderance of MS Windows clients. While there are other network file systems such as Network File System (NFS) and clustered file systems such as Ceph, PanFS, and OrangeFS, they tend to be used less extensively in most non-research networks.

In spite of the prevalence of SMB usage within most enterprise networks, there has been very little analysis of SMB workloads in prior academic research. The last major study of SMB was more than a decade ago [?], and the nature of storage usage has changed dramatically over the last decade. It is always important to revisit commonly used protocols to examine their use in comparison to the expected use case(s). This is doubly so for network communications because the nuances of networked data exchange can greatly influence the effectiveness and efficiency of a chosen protocol. Since an SMB-based trace study has not been undertaken recently, we took a look at its current implementation and use in a large university network.

Our study is based on network packet traces collected on the University of Connecticut’s centralized storage facility over a period of three weeks in May 2019. This trace-driven analysis can help in the design of future storage products as well as providing data for future performance benchmarks. Benchmarks allow for the stress testing of various aspects of a system (e.g. network, single system). Aggregate data analysis collected from traces can lead to the development of synthetic benchmarks. Traces can also expose systems patterns that can also be reflected in synthetic benchmarks. Finally, the traces themselves can drive system simulations that can be used to evaluate prospective storage architectures.

We created a new tracing system to collect data from the UConn storage network system. The tracing system was built around the high-speed PF RING packet capture system and required the use of proper hardware and software to handle incoming data. We also created a new trace capture format derived on the DataSeries structured data format developed by HP [?]. PF RING acts as a kernel module that aids in minimizing packet loss/timestamping issues by not passing packets through the kernel data structures [?]. DataSeries was modified to filter specific SMB protocol fields along with the writing of analysis tools to parse and dissect the captured packets. Specific fields were chosen to be the interesting fields kept for analysis. The DataSeries data format allowed us to create data analysis code that focuses on I/O events and ID tracking (TID/UID). The future vision for this information is to combine ID tracking with the OpLock information in order to track resource sharing of the different clients on the network. As well as using IP information to recreate communication in a larger network trace to establish a better benchmark.

The contributions of this work are the new traces of SMB traffic over a larger university network as well as new analysis of this traffic. Our new examination of the captured data reveals that despite the streamlining of the CIFS/SMB protocol to be less ”chatty”, the majority of SMB communication is still metadata based I/O rather than actual data I/O. We found that read operations occur in greater numbers and cause a larger overall number of bytes to pass over the network. However, the average number of bytes transferred for each write I/O is greater than that of the average read operation. We also find that the current standard for modeling network I/O holds for the majority of operations, while a more representative model needs to be developed for reads.

A. Related Work

In this section we discuss previous studies examining traces and testing that has advanced benchmark development. We summarize major works in trace study in Table I. In addition we examine issues that occur with traces and the assumptions in their study. Tracing collection and analysis has proved

Study Date of Traces  FS/Protocol  Network FS Trace Approach  Workload
Ousterhout, et al. [?]  1985  BSD  Dynamic  Engineering
Ramakrishnan, et al. [?]  1988-89  VAX/VMS  x  Dynamic  Engineering, HPC, Corporate
Baker, et al. [?]  1991  Sprite  x  Dynamic  Engineering
Gribble, et al. [?]  1991-97  Sprite, NFS, VxFS  x  Both  Engineering, Backup
Douceur and Bolosky [?]  1998  FAT, FAT32, NTFS  Snapshots  Engineering
Vogels [?]  1998  FAT, NTFS  Both  Engineering, HPC
Zhou and Smith [?]  1999  VFAT  Dynamic  PC
Roselli, et al. [?]  1997-00  VxFS, NTFS  Dynamic  Engineering, Server
Malkani, et al. [?]  2001  NFS  x  Dynamic  Engineering, Email
Agrawal, et al. [?]  2000-2004  FAT, FAT32, NTFS  Snapshots  Engineering
Leung, et al. [?]  2007  CIFS  x  Dynamic  Corporate, Engineering
Vrable, et al. [?]  2009  FUSE  x  Snapshots  Backup
Benson, et al. [?]  2010  AFS, MapReduce, NCP, SMB  x  Dynamic  Academic, Corporate
Chen, et al. [?]  2012  MapReduce  x  Dynamic  Corporate
This paper  2020  SMB  x  Dynamic  Academic, Engineering, Backup

TABLE I

Summary of major file system studies over the past decades. For each study the tables shows the dates of the trace data, the file system or protocol studied, whether it involved network file systems, the trace methodology used, and the workloads

studied. Dynamic trace studies are those that involve traces of live requests. Snapshot studies involve snapshots of file system contents.

its worth in time from previous studies where one can see important lessons pulled from the research; change in behavior of read/write events, overhead concerns originating in system implementation, bottlenecks in communication, and other revelations found in the traces. Previous tracing work has shown that one of the largest & broadest hurdles to tackle is that traces (and benchmarks) must be tailored to the system being tested. There are always some generalizations taken into account but these generalizations can also be a major source of error [?], [?], [?], [?], [?], [?], [?], [?], [?]. To produce a benchmark with high fidelity one needs to understand not only the technology being used but how it is being implemented within the system [?], [?], [?]. All of these aspects will lend to the behavior of the system; from timing & resource elements to how the managing software governs actions [?], [?], [?]. Furthermore, in pursuing this work one may find unexpected results and learn new things through examination [?], [?], [?]. These studies are required in order to evaluate the development of technologies and methodologies along with furthering knowledge of different system aspects and capabilities. As has been pointed out by past work, the design of systems is usually guided by an understanding of the file system workloads and user behavior [?]. It is for that reason that new studies are constantly performed by the science community, from large scale studies to individual protocol studies [?], [?], [?], [?], [?]. Even within these studies, the information gleaned is only as meaningful as the considerations of how the data is handled.

The work done by Leung et. al. [?] found observations related to the infrequency of files to be shared by more than one client. Over 67% of files were never open by more than one client. Leung’s et. al. work led to a series of observations, from the fact that files are rarely re-opened to finding that readwrite access patterns are more frequent [?]. The 2004 paper by Anderson et. al. [?] has the following observations. A source of decreased precision came from the Kernel overhead for providing timestamp resolution. This would introduce substantial errors in the observed system metrics due to the use inaccurate tools when benchmarking I/O systems. These errors in perceived I/O response times can range from +350% to -15%. Issues of inaccuracies in scheduling I/O can result in as much as a factor 3.5 difference in measured response time and factor of 26 in measured queue sizes. These inaccuracies pose too much of an issue to ignore.

Orosz and Skopko examined the effect of the kernel on packet loss in their 2013 paper [?]. Their work showed that when taking network measurements the precision of the timestamping of packets is a more important criterion than low clock offset, especially when measuring packet inter-arrival times and round-trip delays at a single point of the network. One concern is that Dumpcap is a single threaded application and was suspected to be unable to handle new arriving packets due to a small size of the kernel buffer. Work by Dabir and Matrawy, in 2008 [?], attempted to overcome this limitation by using two semaphores to buffer incoming strings and improve the writing of packet information to disk.

Narayan and Chandy examined the concerns of distributed I/O and the different models of parallel application I/O. Due to the striping of files across multiple nodes, this can cause any read or write to access all the nodes; which does not decrease the inter-arrival times (IATs) seen. As the number of I/O operations increase and the number of nodes increase, the IAT times decreased. Observations from Skopko in a 2012 paper [?] examined the nuance concerns of software based capture solutions. The main observation was software solutions relied heavily on OS packet processing mechanisms. Further more, depending on the mode of operation (e.g. interrupt or polling), the timestamping of packets would change. As seen in previous trace work [?], [?], [?], the general perceptions of how computer systems are being used versus their initial purpose have allowed for great strides in eliminating actual bottlenecks rather than spending unnecessary time working on imagined bottlenecks. Without illumination of these underlying actions (e.g. read-write ratios, file death rates, file access rates) these issues can not be readily tackled.

II. Background

A. Server Message Block

The Server Message Block (SMB) is an application-layer network protocol mainly used for providing shared access to files, shared access to printers, shared access to serial ports, miscellaneous communications between nodes on the network, as well as providing an authenticated inter-process communication mechanism. The SMB 1.0 protocol [?] has been found to have high/significant impact on performance due to latency issues. Monitoring revealed a high degree of “chattiness” and disregard of network latency between hosts. Solutions to this problem were included in the updated SMB 2.0 protocol which decreases “chattiness” by reducing commands and sub-commands from over a hundred to nineteen. Additional changes, most significantly being increased security, were implemented in SMB 3.0 protocol (previously named SMB 2.2) [?].

The rough order of communication for SMB session file interaction contains about five steps. First is a negotiation where a Microsoft SMB Protocol dialect is determined. Next a session is established to determine the share-level security. After this the Tree ID (TID) is determined for the share to be connected to as well as a file ID (FID) for a file requested by the client. From this establishment, I/O operations are performed using the FID given in the previous step.

The only data that needs to be tracked from the SMB traces are the UID (User ID) and TID for each session. The SMB commands also include a MID (Multiplex ID) value that is used for tracking individual packets in each established session, and a PID (Process ID) that tracks the process running the command or series of commands on a host. For the purposes of our tracing, we do not track the MID or PID information.

Some nuances of SMB protocol I/O to note are:

• SMB/SMB2 write request is the command that pushes bytes over the wire. Note: the response packet only confirms their arrival and use (e.g. writing).

• SMB/SMB2 read response is the command that pushes bytes over the wire. Note: The request packet only asks for the data.

B. Issues with Tracing

There are three general approaches to creating a benchmark based on a trade-off between experimental complexity and resemblance to the original application. (1) Connect the system to a production test environment, run the application, and measure the application metrics. (2) Collect traces from running the application and replay them (after possible modification) back on the test I/O system. (3) Generate a synthetic workload and measure the system performance.

The majority of benchmarks attempt to represent a known system and structure on which some “original” design/system was tested. While this is all well and good, there are many issues with this sort of approach; temporal & spatial scaling concerns, timestamping and buffer copying, as well as driver

Fig. 1. Visualization of SMB Packet

operation for capturing packets [?], [?], [?]. Each of these aspects contribute to the initial problems with dissection and analysis of the captured information. For example, inaccuracies in scheduling I/Os may result in as much as a factor of 3.5 differences in measured response time and factor of 26 in measured queue sizes; differences that are too large to ignore [?]. Dealing with timing accuracy and high throughput involves three challenges. (1) Designing for dealing with peak performance requirements. (2) Coping with OS timing inaccuracies. (3) Working around unpredictable OS behavior; e.g. mechanisms to keep time and issue I/Os or performance effects due to interrupts.

Temporal scaling refers to the need to account for the nuances of timing with respect to the run time of commands; consisting of computation, communication & service. A temporally scalable benchmarking system would take these subtleties into account when expanding its operation across multiple machines in a network. While these temporal issues have been tackled for a single processor (and even somewhat for cases of multi-processor), these same timing issues are not properly handled when dealing with inter-network communication. Inaccuracies in packet timestamping can be caused due to overhead in generic kernel-time based solutions, as well as use of the kernel data structures [?], [?].

Spatial scaling refers to the need to account for the nuances of expanding a benchmark to incorporate a number of machines over a network. A system that properly incorporates spatial scaling is one that would be able to incorporate communication (even in varying intensities) between all the machines on a system, thus stress testing all communicative actions and aspects (e.g. resource locks, queueing) on the network.

III. Packet Capturing System

In this section, we describe the packet capturing system as well as decisions made that influence its capabilities. We illustrate the existing university network filesystem as well as our methods for ensuring high-speed packet capture. Then, we discuss the analysis code we developed for examining the captured data.

A. UITS System Overview

We collected traces from the University of Connecticut University Information Technology Services (UITS) centralized storage server. The UITS system consists of five Microsoft file server cluster nodes. These blade servers are used to host SMB file shares for various departments at UConn as well as personal drive share space for faculty, staff and students, along with at least one small group of users. Each server is capable of handling 1 Gb/s of traffic in each direction (e.g. outbound and inbound traffic). Altogether, the five-blade server system can in theory handle 5 Gb/s of data traffic in each direction. The blade servers serve as SMB heads, but the actual storage is served by SAN storage nodes that sit behind them. This system does not currently implement load balancing. Instead, the servers are set up to spread the traffic load with a static distribution among four of the active cluster nodes while the fifth node is passive and purposed to take over in the case that any of the other nodes go down (e.g. become inoperable or crash).

The actual tracing was performed with a tracing server connected to a switch outfitted with a packet duplicating element as shown in the topology diagram in Figure 2. A 10 Gbps network tap was installed in the file server switch, allowing our storage server to obtain a copy of all network traffic going to the 5 file servers. The reason for using 10 Gbps hardware is to help ensure that the system is able to capture and all information on the network at peak theoretical throughput.

B. High-speed Packet Capture

In order to maximize our faithful capture of the constant rate of traffic, we implement on the tracing server an ntop [?] solution called PF RING [?] to dramatically improve the storage server’s packet capture speed. We had to tune an implementation of tshark (wireshark’s terminal pcap implementation) to maximize the packet capture rate. tshark outputs .pcap files which captures all of the data present in packets on the network. We configure tshark so that it only captures SMB packets. Furthermore, to optimize this step, a capture ring buffer flag is used to minimize the amount of space used to write .pcap files, while optimizing the amount of time to filter data from the .pcap files. The filesize used was in a ring buffer where each file captured was 64000 kB.

The .pcap files from tshark do not lend themselves to easy data analysis, so we translate these files into the DataSeries [?] format. HP developed DataSeries, an XMLbased structured data format, that was designed to be selfdescriptive, storage and access efficient, and highly flexible.

Total Days  21
Total Sessions  2413589
Number of SMB Operations  281419686
Number of Read I/Os  8355557
Number of Write I/Os  7872219
R:W I/O Ratio  1.06
Number of Creates  54486043
Number of General SMB Operations 210705867
Total Data Read (GB)  0.97
Total Data Written (GB)  0.6
Average Read Size (B)  144
Average Write Size (B)  63

TABLE II

Summary of Trace I/O Statistics for the time of April 30th, 2019 to May 20th, 2019

The system for taking captured .pcap files and writing them into the DataSeries format (i.e. .ds) does so by first creating a structure (based on a pre-written determination of the data desired to capture). Once the code builds this structure, it then reads through the capture traffic packets while dissecting and filling in the prepared structure with the desired information and format. Due to the fundamental nature of this work, there is no need to track every piece of information that is exchanged, only that information which illuminates the behavior of the clients and servers that interact over the network (i.e. I/O transactions). It should also be noted that all sensitive information being captured by the tracing system is hashed to protect the users whose information is examined by the tracing system. Furthermore, the DataSeries file retains only the first 512 bytes of the SMB packet - enough to capture the SMB header information that contains the I/O information we seek, while the body of the SMB traffic is not retained in order to better ensure security of the university’s network communications. It is worth noting that in the case of larger SMB headers, some information is lost, but this is a trade-off by the university to provide, on average, the correct sized SMB header but does lead to scenarios where some information may be captured incompletely.

C. DataSeries Analysis

Building upon existing code for the interpretation and dissection of the captured .ds files, we developed C/C++ code for examining the captured traffic information. From this analysis, we are able to capture read, write, create and general I/O information at both a global scale and individual tracking ID (UID/TID) level. In addition, read and write buffer size information is tracked, as well as the inter-arrival and response times. Also included in this data is oplock information and IP addresses. The main contribution of this step is to aggregate seen information for later interpretation of the results. This step also creates an easily digestible output that can be used to re-create all tuple information for SMB/SMB2 sessions that are witnessed over the entire time period. Sessions are any communication where a valid UID and TID is used.

IV. Data Analysis

Table IV shows a summary of the I/O operations, response times, and inter arrival times observed for the network filesystem. This table illustrates that the majority of I/O operations

Fig. 2. Visualization of Packet Capturing System

are general (74.87%). As shown in the bottom part of Table IV general includes metadata commands such as connect, close, query info, etc.

Our examination of the collected network filesystem data revealed interesting patterns for the current use of CIFS/SMB in a large engineering academic setting. The first is that there is a major shift away from read and write operations towards more metadata-based ones. This matches the last CIFS observations made by Leung et. al. that files were being generated and accessed infrequently. The change in operations are due to a movement of use activity from reading and writing data to simply checking file and directory metadata. However, since the earlier study, SMB has transitioned to the SMB2 protocol which was supposed to be less ”chatty” and thus we would expect fewer general SMB operations. Table III shows a breakdown of SMB and SMB2 usage over the time period of May. From this table, one can see that the SMB2 protocol makes up 99.14% of total network operations compared to just 0.86% for SMB, indicating that most clients have upgraded to SMB2. However, 74.66% of SMB2 I/O are still general operations. Contrary to the purpose of implementing the SMB2 protocol, there is still a large amount of general I/O.

General operations happen at very high frequency with inter arrival times that were found to be relatively short (1317μs on average).

Taking a deeper look at the SMB2 operations, shown in the bottom half of Table III, we see that 9.06% of the general operations are negotiate commands. These are commands sent by the client to notify the server which dialects of the SMB2 protocol the client can understand. The three most common commands are close, tree connect, and query info. The latter two relate to metadata information of shares and files accessed, however the close operation relates to the create operations relayed over the network. Note that the create command is also used as an open file. The first thing one will notice is that the number of closes is greater than the total number of create operations; by 9.35%. These extra close operations are most likely due to applications doing multiple closes that do not need to be done.

A. I/O Data Request Sizes

Each SMB Read and Write command is associated with a data request size that indicates how many bytes are to be read or written as part of that command. Figures 3 and 5 show the probability density function (PDF) of the different sizes of bytes transferred for read and write I/O operations respectively. The most noticeable aspect of these graphs are that the majority of bytes transferred for read and write operations is around 64 bytes. It is worth noting that write I/O also have a larger number of very small transfer amounts. This is unexpected in terms of the amount of data passed in a frame. Our belief is that this is due to a large number of long term calculations/scripts being run that only require small but frequent updates. This assumption was later validated in part when examining the files transferred, as some were related to running scripts creating a large volume of files.

I/O Operation  SMB  SMB2  Both
Read Operations  1931  8353626  8355557
Read %  0.08%  2.99%  2.97%
Write Operations  303  7871916  7872219
Write %  0.01%  2.82%  2.80%
Create Operations  0  54486043  54486043
Create %  0.00%  19.53%  19.36%
General Operations  2418980 208286887  210705867
General %  99.91%  74.66%  74.87%
Combine Protocol Operations 2421214 278998472  281419686
Combined Protocols %  0.86%  99.14%  100%
SMB2 General Operation  Occurrences  Percentage of Total
Negotiate  25276447  9.06%
Session Setup  2041208  0.73%
Logoff  143592  0.05%
Tree Connect  48414491  17.35%
Tree Disconnect  9773361  3.5%
Close  80114256  28.71%
Flush  972790  0.35%
Lock  1389250  0.5%
IOCtl  4475494  1.6%
Cancel  0  0.00%
Echo  4715  0.002%
Query Directory  3443491  1.23%
Change Notify  612850  0.22%
Query Info  27155528  9.73%
Set Info  4447218  1.59%
Oplock Break  22397  0.008%

TABLE III

Percentage of SMB and SMB2 Protocol Commands from April 30th, 2019 to May 20th, 2019. Breakdown of General

Operations for SMB2

Fig. 3. PDF of Bytes Transferred for Read I/O

Figures 4 and 6 show cumulative distribution functions (CDF) for bytes read and bytes written. As can be seen, almost no read transfer sizes are less than 32 bytes, whereas 20% writes below 32 bytes. Table IV shows a tabular view of this data. For reads, 34.97% are between 64 and 512 bytes, with another 28.86% at 64 byte request sizes. There are a negligible

Fig. 4. CDF of Bytes Transferred for Read I/O

Fig. 5. PDF of Bytes Transferred for Write I/O

percentage of read requests larger than 512. This read data differs from the size of reads observed by Leung et al. by a factor of 4 smaller. Writes observed also differ from previous inspection of the protocol’s usage. Leung et al. showed that 60-70% of writes were less than 4K in size and 90% less than 64K in size. In our data, however, we see that only 11.16%

Fig. 6. CDF of Bytes Transferred for Write I/O

Transfer size  Reads  Writes
< 4  0.098%  11.16%
= 4  1.16%  4.13%
> 4, < 64  34.89%  28.14%
= 64  28.86%  52.41%
> 64, < 512 34.97%  4.15%
= 512 0.002% 2.54e-5%
= 1024  1.22e-5% 3.81e-5%

TABLE IV

Percentage of transfer sizes for reads and writes

of writes are less than 4 bytes, 52.41% are 64 byte requests, and only 43.63% of requests are less than 64 byte writes. In the ten years since the last study, it is clear that writes have become significantly larger. This may be explained by the fact that large files, and multiple files, are being written as standardized blocks more fitting to the larger data-sets and disk space available. This could be as an effort to improve the fidelity of data across the network, allow for better realtime data consistency between client and backup locations, or could just be due to a large number of scripts being run that create and update a series of relatively smaller documents.

In comparison of the read, write, and create operations we found that the vast majority of these type of I/O belong to creates. By the fact that there are so many creates, it seems apparent that many applications create new files rather than updating existing files when files are modified. Furthermore, read operations account for the largest aggregate of bytes transferred over the network. However, the amount of bytes transferred by write commands is not far behind, although, non-intuitively, including a larger number of standardized relatively smaller writes. The most unexpected finding of the data is that all the the read and writes are performed using much smaller buffers than expected; about an order of

Fig. 7. CDF of Inter Arrival Time for General I/O

Reads  Writes Creates General
I/O %  2.97  2.80  19.36  74.87
Avg RT (μs) 59819.7  519.7  698.1 7013.4
Avg IAT (μs) 33220.8 35260.4 5094.5 1317.4

TABLE V

Summary of Trace Statistics: Average Response Time (RT) and Inter Arrival Time (IAT)

magnitude smaller (e.g. bytes instead of kilobytes).

B. I/O Response Times

Most previous tracing work has not reported I/O response times or command latency which is generally proportional to data request size, but under load, the response times give an indication of server load. In Table V we show a summary of the response times for read, write, create, and general commands. We note that most general (metadata) operations occur fairly frequently, run relatively slowly, and happen at high frequency. Other observations of the data show that the number of writes is very close to the number of reads, although the write response time for their operations is very small - most likely because the storage server caches the write without actually committing to disk. Reads on the other hand are in most cases probably not going to hit in the cache and require an actual read from the storage media. Although read operations are only a few percentage of the total operations they have a the greatest average response time; more than general I/O. As noted above, creates happen more frequently, but have a slightly slower response time, because of the extra metadata operations required for a create as opposed to a simple write.

Figure 7 shows the inter arrival times CDF for general I/O. As can be seen, SMB commands happen very frequently -

Fig. 8. PDF of Inter Arrival Time for General I/O

Fig. 9. CDF of Response Time for General I/O

85% of commands are issued less than 1024 μs apart. As was mentioned above, the SMB protocol is known to be very chatty, and it is clear that servers must spend a lot of time dealing with these commands. For the most part, most of these commands are also serviced fairly quickly as seen in Figure 9. Interestingly, the response/return time (RT) for the general metadata operations follows a similar curve to the inter-arrival times.

Fig. 10. PDF of Response Time for General I/O

Next we examine the response time (RT) of the read, write, and create I/O operations that occur over the SMB network filesystem. The response time for write operations (shown in Figure 17) does not follow the step function similar to the bytes written CDF in Figure 6. This is understandable as the response time for a write would be expected to be a more standardized action and not necessarily proportional to the number of bytes written. However, the read response time (Figure 13) is smoother than the bytes read CDF (Figure 6). This is most likely due to the fact that some of the reads are satisfied by server caches, thus eliminating some long access times to persistent storage. However, one should notice that the response time on read operations grows at a rate similar to that of write operations. This, again, shows a form of standardization in the communication patterns although some read I/O take a far greater period of time; due to larger amounts of read data sent over several standardized size packets.

C. File Extensions

Tables VI and VII show a summary of the various file extensions that were seen within the three-week capture period. The easier to understand is Table VII, which illustrates the number of common file extensions (e.g. doc, ppt, xls, pdf) that were part of the data. Originally we expected that these common file extensions would be a much larger total of traffic. However, as seen in Table VII, these common file extensions were less than 2% of total files seen. The top ten extensions that we saw (Table VI) comprised approximately 84% of the total seen. Furthermore, the majority of extensions are not readily identified. Upon closer examination of the tracing system it was determined that many files simply do not have a valid extension. These range from linux-based library files, manual pages, odd naming schemes as part of scripts

Fig. 11. CDF of Inter Arrival Time for Read I/O

Fig. 12. PDF of Inter Arrival Time for Read I/O Fig. 13. CDF of Response Time for Read I/O

Fig. 14. PDF of Response Time for Read I/O

or back-up files, as well as date-times and IPs as file names. There are undoubtedly a larger number more, but exhaustive determination of all variations is seen as out of scope for this work.

D. Distribution Models

For simulations and analytic modeling, it is often useful to have models that describe storage systems I/O behavior. In this section, we attempt to map traditional probabilistic distributions to the data that we have observed. Specifically, taking the

Fig. 15. CDF of Inter Arrival Time for Write I/O

Fig. 16. PDF of Inter Arrival Time for Write I/O

developed CDF graphs, we perform curve fitting to determine the applicability of Gaussian and Weibull distributions to the the network filesystem I/O behavior. Note that an exponential distribution, typically used to model interarrival times and response times, is a special case of a Weibull distribution where k = 1. Table VIII shows best-fit parametrized distributions for

Fig. 17. CDF of Return Time for Write IO

Fig. 18. PDF of Return Time for Write IO

the measured.

Our comparison of the existing standard use of a exponential distribution to model network interarrival and response times is still valid. One should notice that the Gaussian distributions for write and create operations are similar, while those for read operations are not. Further more there is less similarity between the modeled behavior of general operation inter arrival times and their response times, showing the need for a more refined model for each aspect of the network filesystem interactions. One should also notice that the general operation

Fig. 19. CDF of Inter Arrival Time for Create I/O

Fig. 20. PDF of Inter Arrival Time for Create I/O

model is more closely similar to that of the creates. This makes sense since the influence of create operations are found to dominate the I/O behavior of the network filesystem, which aligns well with the number of existing close operations. Due to the large number of metadata operations, the use of smart storage solutions could be used to minimize the impact of these

Fig. 21. CDF of Response Time for Create I/O

Fig. 22. PDF of Response Time for Create I/O

I/O. Smart storage elements can aid by performing metadata operations without the need to access persistent storage, thus causing shorter response times. In this manner, the use of smart storage can also help reduce bottlenecks with larger network filesystems and minimize the effect of traffic on overall network performance.

E. System Limitations and Challenges

When initially designing the tracing system used in this paper, different aspects were taken into account, such as space

SMB2 Filename Extension Occurrences Percentage of Total
-Travel  33396147  15.26
o  28670784  13.1
e  28606421  13.07
N  27639457  12.63
one  27615505  12.62
<No Extension>  27613845  12.62
d  2799799  1.28
l  2321338  1.06
x  2108279  0.96
h  2019714  0.92

TABLE VI

Top 10 File Extensions Seen Over Three Week Period

SMB2 Filename Extension Occurrences Percentage of Total
doc  352958  0.16
docx  291047  0.13
ppt  46706  0.02
pptx  38604  0.02
xls  218031  0.1
xlsx  180676  0.08
odt  28  0.000013
pdf  375601  0.17
xml  1192840  0.54
txt  167827  0.08

TABLE VII

Common File Extensions Seen Over Three Week Period

limitations of the tracing system, packet capture limitations (e.g. file size), and speed limitations of the hardware. One limitation encountered in the packet capture system deals with the functional pcap (packet capture file) size. The concern being that the pcap files only need to be held until they have been filtered for specific protocol information and then compressed using the DataSeries format, but still allow for room for the DataSeries files being created to be stored. Other limitation concerns came from the software and packages used to collect the network traffic data [?], [?], [?]. These ranged from timestamp resolution provided by the tracing system’s kernel [?] to how the packet capturing drivers and programs (such as dumpcap and tshark) operate along with how many copies are performed and how often. The speed limitations of the hardware are dictated by the hardware being used (e.g. GB capture interface) and the software that makes use of this hardware (e.g. PF RING). After all, our data can only be as accurate as the information being captured [?], [?]. An other concern was whether or not the system would be able

Model  Gaussian  Weibull
CDF  \frac{1}{\sqrt{2π}} ∫−∞^{\frac{x−μ}{σ}} e^{\frac{−t2}{2}} dt  1 − e^{(−x/λ)^{k}}
I/O Operation  μ  σ  k  λ
General IAT  786.72  10329.6  0.9031 743.2075
General RT  3606.66 2.74931e+06 0.5652 980.9721
Read RT  44718.5 1.72776e+07 0.0004  1.5517
Read IAT  24146  1.189e+07  0.0005  3.8134
Write RT  379.823  4021.72  0.8569 325.2856
Write IAT  25785.7 1.22491e+07 0.0004  3.1287
Create RT  502.084  21678.4  0.9840 496.9497
Create IAT  3694.82 4.65553e+06 0.0008  2.3504
Read Buff Transfer 82.9179  1117.9  1.0548 85.2525
Write Buff Transfer 46.2507  640.621  1.0325 46.8707

TABLE VIII

Comparison of μ, σ, k, and λ Values for Curve Fitting Equations on CDF Graphs

to function optimally during periods of high network traffic. All aspects of the system, from the hardware to the software, have been altered to help combat these concerns and allow for the most accurate packet capturing possible.

While the limitations of the system were concerns, there were other challenges that were tackled in the development of this research. One glaring challenge with building this tracing system was using code written by others; tshark & DataSeries. While these programs are used within the tracing structure there are some issues when working with them. These issues ranged from data type limitations of the code to hash value and checksum miscalculations due to encryption of specific fields/data. Attempt was made to dig and correct these issues, but they were so inherent to the code being worked with that hacks and workarounds were developed to minimize their effect. Other challenges centralize around selection, interpretations and distribution scope of the data collected. Which fields should be filtered out from the original packet capture? What data is most prophetic to the form and function of the network being traced? What should be the scope, with respect to time, of the data being examined? Where will the most interesting information appear? As each obstacle was tackled, new information and ways of examining the data reveal themselves and with each development different alterations & corrections are made.

Even when all the information is collected and the most important data has been selected, there is still the issue of what lens should be used to view this information. Because the data being collected is from an active network, there will be differing activity depending on the time of day, week, and scholastic year. For example, although the first week or so of the year may contain a lot of traffic, this does not mean that trends of that period of time will occur for every week of the year (except perhaps the final week of the semester). The trends and habits of the network will change based on the time of year, time of day, and even depend on the exam schedule. Truly interesting examination of data requires looking at all different periods of time to see how all these factors play into the communications of the network. A complication of this process is that the DataSeries code makes use of a pushpop stack for iterating through packet information. This means that if information can not be re-read then errors occur. This can manifest in the scenario where a produced .ds file is corrupted or incomplete, despite the fact that the original .pcap being fine. Normally, one could simply re-perform the conversion process to a DataSeries file, but due to the rate of the packets being captured and security concerns of the data being captured, we are unable to re-run any captured information.

V. Conclusions and Future Work

Our analysis of this university network filesystem illustrated the current implementation and use of the CIFS/SMB protocol in a large academic setting. We notice the effect of caches on the ability of the filesystem to limit the number of accesses to persistant storage. The effect of enterprise storage disks access time can seen in the response time for read and write I/O. The majority of network communication is dominated by metadata operation, which is of less surprise since SMB is a known chatty protocol. We do notice that the CIFS/SMB protocol continues to be chatty with metadata I/O operations regardless of the version of SMB being implemented; 74.66% of I/O being metadata operations for SMB2. We also find that read operations happen in greater number than write operations (at a ratio of 1.06) and the size of their transfers are is also greater by a factor of about 2. However, the average write operation includes a larger number of relatively smaller writes. Examination of the return times for these different I/O operations shows that exponential distribution curve fitting equation is most accurate at modeling the CDF of the various I/O operations. This shows that the current model is still effective for the majority of I/O, but that for read operations there needs to be further research in modeling their behavior. Our work finds that write and create response times can be modeled similarly, but that the read response times require the alteration of the general model. However, the general I/O can be modeled using the same standard; which has similar shape and scale to that of the write and create operations.

A. Future Work

The analysis work will eventually incorporate oplocks and other aspects of resource sharing on the network to gain a more complete picture of the network’s usage and bottlenecks. Network filesystem usage from an individual user scope has become simple and does not contain a greater deal of read, write, and create operations. Further analysis will be made in examining how the determined metrics change when examined at the scope of a per share (i.e. TID) or per user (i.e. UID). At this level of examination we will be able to obtain a better idea of how each share is interacted with, as well as how files and directories are shared and access control is implemented.