2.1 Phase-change Memory

PCM stores information by alternating the phase of Ge$_{2}$Sb$_{2}$Te$_{5}$ (GST). The phase of the material is an amorphous state or crystalline state and determines the resistance. It can be changed by applying different temperatures to a cell, so PCM is considered as a representative NVM. It can be highly integrated since it has high scalability without introducing a large number of capacitors. Furthermore, it has byte-granularity and latency similar to DRAM, so it is expected to replace DRAM in the future.

2.2 Direct Access (DAX)

Page cache is widely used in conventional block device storage. It is a buffer between memory and storage when a system reads or writes files. The latency of storage can be hidden by caching data in page cache (DRAM). However, the use of page cache in a memory-like block device results in degraded performance due to unnecessary copies. Furthermore, the page cache has no persistency, so a system crash hurts the integrity of data in the page cache even if the storage has a non-volatility feature.

DAX allows an application to access storage directly by load/store commands. Therefore, a file system with DAX capability prevents duplication in the page cache by skipping it. Moreover, DAX provides better performance when applications with a number of small writes or synchronization operations are executed on the target system.

2.3 Filebench

$\textit{Filebench}$ is a file system and storage benchmark that is widely used in academia and industry ^[17]. Workload Model Language (WML) is a high-level language defined by $\textit{Filebench}$ that makes it easy to change the configuration of workloads, observe the performance of each operation, or even create new workloads in a few lines of code. In this study, $\textit{Filebench}$ was used to test workloads with various characteristics and obtain detailed results easily. Four representative pre-defined workloads were evaluated: Fileserver, Webserver, Webproxy, and Varmail. The features of each workload are as follows:

· Fileserver: Fileserver is a workload that simulates the operation of a file server and performs various file management commands, such as creating, writing, opening, reading, appending, and deleting files. Other workloads only write by appending, but Fileserver has writewholefile command and appendfile command, so it has a write-intensive feature.

· Webserver: Webserver is a workload that receives an HTTP request, reads a large amount of files, passes it to the client, and stores access records in a log file. It has a read-intensive feature and no directory operation.

· Webproxy: Webproxy acts as a normal web proxy server that receives requests from clients and seeks resources from other servers. Reading files is its main feature, and it has many directory operations like createfile and deletefile.

· Varmail: Varmail is a workload that functions as a mail server that receives mail, reads it, and synchronizes it after marking them read. The difference from other workloads is the fsync command, which is the synchronization command of $\textit{Filebench}$.

4.1 Simulation Environment

Experiments were done using a virtual machine with Linux on a QEMU emulator ^[18], which corresponds to the hypervisor in Fig. 1. For easy installation of the library, Fedora 28 (kernel version 4.18) was used. The virtual environment system uses an Intel i7-7800X 3.5-GHz core and 16 GB of DRAM. 64-GB PCM (vPCM in Fig. 1) was emulated on a file (storage in Fig. 1) that is located in each DRAM, HDD (WD20EZRZ), and SSD (TS512GSSD230S) according to the experiments. It is recognized in the virtual system using LIBNVDIMM ^[19], a sub-system that manages NVDIMM devices in Linux, and NDCTL, a library for the user-interface control of LIBNVDIMM ^[20]. The PCM and conventional storage are mounted with ext4-DAX and ext4, respectively. The host page cache is turned off in all experiments.

4.2 Workload Configuration

The four $\textit{Filebench}$ workloads mentioned in Section 2 were evaluated. The basic configuration of the workloads modified from previous studies is presented in Table 1 ^[11,13]. The parameters are configured in a way that preserves the characteristics of each workload. It should be noted that the number of files of Varmail (50K) is half that of other workloads (100K) because Varmail originally has a small file set.

The experiments were performed by changing the amount of files, I/O size, and number of threads. The effect of the amount of files was tested by comparing the default workloads and the workloads with 10-times fewer files. The change in throughput and latency was observed while increasing the I/O size from 64 B to 16 KB and the number of threads from 1 to 32. Since there is a small variation in each run of a workload, all experiments were done five times.

4.3 Throughput of Default Configurations

Fig. 2(a) presents the $\textit{Filebench}$ throughput of the default setting with 5 different cases: PCM emulated on a file in an HDD (PCM-HDD), PCM emulated on a file in an SSD (PCM-SSD), and PCM emulated on a file in DRAM (PCM-DRAM), an HDD, and an SSD. The experimental results show that PCM-DRAM has the best performance in all workloads, so it can achieve the most similar performance to that of actual PCM devices.

When comparing the other two PCMs (PCM-HDD and PCM-SSD) with conventional storage (HDD and SSD), the PCMs outperform the conventional storage in the write-intensive environments (Fileserver and Varmail) thanks to the DAX feature. In particular, Varmail shows a significant performance gap because there are many fsync commands that do not have the benefit of the guest page cache. On the other hand, in read-intensive Webserver and Webproxy, the HDD and SSD outperform PCMs because the hypervisor layer overhead becomes notable and the gain of the guest page cache almost reaches the performance of DRAM. From these results, it can be concluded that the DAX feature does not give a performance improvement in read-intensive environments and is suitable for environments with many writes and synchronization commands.

Fig. 2. Throughputs on each Filebench with various memory devices and configurations.

4.4 Latency Breakdown of Each Workload

Fig. 3 shows a plot of the latency breakdown of $\textit{Filebench}$ workloads. Overall, the read-intensive workloads have shorter latency than the write-intensive workloads. Fig. 3(a) shows that in Fileserver, the write commands of HDD take 3 times longer than those of PCM-HDD and are responsible for the largest part of the latency. In Varmail, the fsync commands are responsible for a small portion of the total latency of PCM-HDD, but on HDDs, they cause a 200 times longer latency than that of PCM-HDD and are responsible for almost all the latency of the HDD.

Webserver and Webproxy show similar latency breakdown patterns on both the PCM-HDD and HDD, but the HDD shows better performance for append and readfile commands. When the storage is changed to the SSD, as shown in Fig. 3(b), the whole latency is remarkably reduced because the SSD latency of writefile in Fileserver and that of fsync in Varmail show a significant improvement. In the case of PCM-SSD, the latency of readfile in Webproxy is significantly reduced. From these results, it can be concluded that PCM shows high performance for the writefile and fsync commands, but not for the readfile commands, which can cause many transactions.

Fig. 3. Latency breakdown on each Filebench with various memory devices and configurations.

4.5 Experimental Results According to Various Operating Environments

Figs. 2(b), 3(c) and (d) show the experimental results with regard to the amount of files. Small amounts of files improve the performance on most workloads due to their high locality ^[11], as shown in Fig. 2(b). The change of HDD and SSD in Fileserver is especially notable. There is a significant improvement compared to the default configuration (Fig. 2(a)) because the small number of files can fit in the page cache, causing a decrease of latency from writefile commands.

On the other hand, as shown in Figs. 3(c) and (d), the latency of all commands is evenly reduced due to the high locality. This shows that PCM has no advantages of a small amount of files because conventional storage has a better throughput improvement in write-intensive environments due to small amounts of files and still shows higher throughput in read-intensive environments. Changing the directory width leads to similar results as changing the amount of files.

Fig. 4 shows the measured throughput with regard to the I/O size. The variation of latency can be inferred by taking the inverse of the throughput. In all devices, larger I/O size causes more pre-fetch effects and fewer transactions, resulting in better performance, but the throughput converges around 4K. Webserver, Webproxy, and Varmail present this tendency clearly. On Webserver, which has many read operations with high locality, convergence happens a bit late. This result shows that the I/O size of PCM should be set to 4K based on the convergence of the throughput.

Fig. 5 presents the average latency under a varying number of threads. It shows the latency on a log scale in order to show the correlation between the latency increase and the number of threads. Like in the previous experiments, the variation of throughput can be inferred by taking the inverse of the latency. For most workloads, the latency increases proportionally to the number of threads, which means increasing the number of threads does not help to improve the performance due to additional latency.

In Varmail, the slope of the graphs of HDD and SSD is very gradual because the guest page cache can be flushed simultaneously on HDDs or SSDs. Therefore, their latency is more tolerable to the growing number of threads compared to PCMs. However, even if this tolerance is present, it should be noted that the absolute value of the latency is significantly higher than that of PCM’s latency. Therefore, it can be concluded that PCM has no advantages over conventional storage, depending on the number of threads.

Fig. 4. Throughput of Filebench with varying I/O size.

Fig. 5. Latency of Filebench with varying the number of threads.

4.6 Application Proposal

The experiments demonstrate that PCM performs better than traditional storage on workloads with the following characteristics: many synchronization commands, write-intensive commands, low locality, and 4-KB I/O size. Hence, applications that write a small-size unit in a large amount of data and perform frequent metadata updates are proposed as a target for PCM.