The volume of unstructured data has been growing at a tremendous rate, and cloud services have become widespread. Accordingly, both private enterprises and public organizations have started to utilize the accumulated data for various purposes, including:
Retailers ingesting video surveillance streams, manufacturers collecting sensor data, media companies streaming movies
Multiple commodity servers for storage management will consume more power and are more complex to manage
Retain server performance
Increase storage capacity
Retailers analyzing video for consumer purchasing behavior
Financial services companies use automated, real-time processing to detect credit card fraud
Some customers want to deal with volume, variety and velocity of data simultaneously
Low-latency and high-capacity storage is very expensive similar to in-memory-computing (DRAM based)
There are two data storage architectures to increase storage capacity in order to handle data growth: scale-up and scale-out. Generally, it has been considered that a scale-out (distributed) model provides an easier means of adding more resources.
At the same time, it has been pointed out that, as the data volume increases, the conventional scale-out storage architecture will suffer from bottlenecks caused by the processing of metadata and consequently degrade system performance.
Higher hardware and infrastructure costs to manage stack inefficiencies
Rebalancing and replicating are expensive tasks impacting performance
Delicate compute and storage balance to optimize performance and costs
Besides an increase in the per-disk capacity, a new type of storage called Ethernet drives has emerged. An Ethernet drive is a storage node that incorporates a processing unit and a network interface and is directly connected to an IP network.
Unlike block and file storage, object storage is configured in a distributed architecture. Since object storage uses Key-Value interface commands, it eliminates the need for driver software comprising block devices and file systems. Furthermore, object storage uses a flat structure and has a wide namespace. For these reasons, object storage devices are better suited for the handling of unstructured data than block and file storage devices.
Regardless of the type of storage media, object storage devices manage data as objects as opposed to block storage devices that use logical block addressing (LBA) and file storage devices that use a file system (NFS, CIFS, etc.). Moreover, multiple low-cost disks can be configured as a large-capacity storage pool by using a RAID or other disk array controller.
In general data storage, data consisting of a series of fixed-length blocks of 512 bytes each are identified by their logical block address (LBA).In contrast, in a Key-Value drive, data are identified by a key rather than an LBA. Both keys and data (called values in this parlance) can be of variable length.
Whereas general drives are primarily connected to a server via the SATA, SAS, PCIe or other interface, Key-Value drives are implemented as a kind of Ethernet drive and thus directly connect to a network via Ethernet. Consequently, each Key-Value drive can act as a node and perform part of communications directly with clients without involving a server.
Toshiba is exploring the possibility of delivering various Key-Value drives to meet different market needs. This section introduces HDD-based drives and hybrid drives.