Unstructured data is growing at an astonishing rate with organisations estimating that 34% of their data resides in unstructured storage, with 40% of their object storage capacity being all-flash. The average amount of active/primary unstructured data capacity is reported to be 28 petabytes, with projections of 38% annual growth. This rapid expansion of unstructured data presents central data portability and protection challenges.
The Promise and Pitfalls of Public Cloud
Many organisations initially saw public cloud storage as the ideal solution for this flood of data. The public cloud provides easy scalability, no upfront infrastructure costs, and seemingly unlimited capacity. However, as unstructured data sets grow over time, significant pitfalls emerge:
- Data egress fees make moving large data sets out of the public cloud expensive. This creates vendor lock-in even when better solutions emerge.
- Latency across wide area networks (WANs) hampers application performance, especially for analytics and AI workloads that require frequent data access.
- Storage costs that seem modest can escalate rapidly as data sets expand. ESG found examples of organisations struggling with monthly public cloud bills.
The Rise of On-Premises Object Storage
Innovative IT leaders are embracing on-premises object storage solutions designed specifically for massive-scale unstructured data in response to the pitfalls of public cloud storage. Dell ECS is one such leading solution offering compelling benefits:
- Cost Efficiency Through Software-Defined Storage
Dell ECS utilises a software-defined architecture that allows customers to leverage lower-cost, industry-standard hardware rather than expensive proprietary storage arrays. The software-based approach enables seamless integration of new hardware innovations without disruptive data migrations.
- Built-in Data Resiliency and Access
Dell ECS automates data resilience through erasure coding, triple mirroring across nodes, and protection across geographically dispersed deployment sites. This eliminates the need for separate cloud gateways just for data protection. Dell ECS also provides a global namespace to enable simple access with strong consistency across the entire storage pool.
- High Performance and Low Latency
By locating storage physically close to applications rather than across a WAN, Dell ECS achieves much lower latency for data access. This performance advantage is critical for modern analytics, AI, and machine learning workloads needing frequent, fast access to enormous datasets.
- No Hidden Public Cloud Costs
With Dell ECS, there are no unpredictable cloud provider charges for data egress or API access requests. The economics are transparent upfront rather than escalating as workloads access growing data.
Validating the Cost Benefits
An ESG analysis compared the 5-year total cost of ownership (TCO) for storing 3PB of data in the public cloud versus on-premises Dell ECS object storage. The public cloud was 50-76% more expensive, even when comparing Dell ECS to discounted archive/cold storage tiers.
The analysis made conservative assumptions around asset life, data access patterns, and cloud provider pricing. Thus, real-world use cases will likely show even more significant TCO advantages from Dell ECS.
Critical Considerations for Object Storage
The ESG analysis highlights several essential factors that make on-premises object storage like Dell ECS more cost-effective than public cloud storage in many common scenarios:
Data Set Size
For data sets of 400TB or more, the economies of scale of on-prem object storage result in lower TCO than public cloud solutions. The TCO savings become even more substantial as data volumes scale into the petabyte range.
Asset Lifespan
For data with a lifespan of 3 years or more, amortising the upfront cost of on-prem infrastructure over that extended period reduces its effective cost. Public cloud services may appear less expensive for short-term storage but cost more over multi-year timeframes.
Access Patterns
If stored data needs to be accessed at least 1% of the time monthly, the egress fees charged by public cloud providers add up rapidly. On-prem solutions avoid such fees for internal data transfers and local computation.
Object Size
For use cases with small average object sizes, typically under 1MB, the overhead of public cloud storage requests and transactions adds noticeable expense. The optimised object management of solutions like Dell ECS handles small objects more efficiently.
Summing Up
On-premises object storage allows complete control and customisation of the infrastructure, often has lower egress costs, offers physically proximate data access that minimises latency, and may be mandated for compliance or data sovereignty reasons. However, it requires upfront capital expenditures, in-house management and upkeep, and scaling can be complex and costly.
Public cloud object storage provides easy scalability on demand, no hardware maintenance or personnel required, potential cost savings, built-in redundancy and disaster recovery, and access to advanced cloud services. However, there are ongoing operating expenses, limited control requiring the provider’s trust, egress costs can accumulate for frequent access, and latency may be higher than on-premises.
The choice often concerns a trade-off between control/customisation and lower maintenance (on-prem) versus flexibility and no hardware overhead (public cloud). If data sovereignty, low latency access, or regulatory compliance are critical requirements, on-prem object storage is the better choice for most businesses.
