By Dataversity, California
As unstructured data continues to grow exponentially, it is creating an inflection point for IT leaders. The average enterprise IT organization is managing petabytes of file and object data. This has resulted in high costs for data storage and protection, growing security risks from shadow IT and too many data silos, and the desire to leverage AI on this data for organizational gain. The question is: How to turn a liability into an opportunity?
We predict that IT leaders will demand more automation and intelligence from vendors and partners offering solutions to store, protect, manage, and exploit data. At the same time, the individuals overseeing this technology stack – the IT infrastructure and storage experts – will need to adopt a data services mindset rooted in metrics and cross-functional collaboration. This approach will help them better serve departments and stakeholders across the business. Here are our unstructured data management predictions for 2024.
Cloud-first strategies have come under the microscope in the past year or two as enterprises have created flexible, hybrid cloud environments using multiple vendor technologies. The idea that having most or all your workloads in the cloud is the most cost-effective has not panned out. What we’re seeing in the past 12 months is a flurry of new storage innovations on the market – from ever more efficient flash products including QLC flash to AI-ready storage developed by many of the large vendors, in concert with NVIDIA, to a growing array of cheaper storage tiers on the major clouds. Progressive IT organizations will choose from the many storage options on the market based on the performance, cost, and security needs of their data through its lifecycle. Homogenous storage infrastructure creates lock-in and is seldom optimal. Thus, the ability to easily move data becomes paramount as data’s classification changes and to keep up with the fast-moving innovations in storage. Unstructured data management tools that provide data mobility without vendor lock-in will be increasingly valuable.
Data storage technology will evolve based on the changing needs of the enterprise. This is great news for enterprise customers as choice will continue to grow next year. Major storage vendors are creating GPU and Flash products specifically for the extreme processing needs of AI and machine learning applications. Storage vendors are incorporating more security features to protect data at the source, along with energy-efficient features to help organizations cut energy bills and operate more sustainably. As AI becomes more prevalent, the need to create workflows to continuously feed AI and store the results from AI processing will become an inherent part of the IT landscape. However, AI and the distinct move away from one-size-fits-all storage means that enterprise IT complexity will grow. IT teams will need strategies, knowledge, and guidance for data lifecycle management to ensure that data continually moves to the best storage tier for current needs. By doing so, IT can optimize data storage across the board for maximum cost savings, data protection, and performance while also enabling AI.
The traditional approach of expensive data protection focused on mission-critical data no longer works as ransomware, global disasters, and security attacks are on the rise. Organizations need to protect all data but cannot afford to treat all data as if it’s mission-critical, which requires expensive disaster recovery (DR) solutions with fast recovery and stringent SLAs. The top unstructured data management capabilities identified in our 2023 State of Unstructured Data Management survey include data protection, such as policy-based copying of file data to object storage for ransomware protection. Unstructured data management solutions can play a prevalent role by classifying datasets and providing significantly less expensive DR schemes for non-mission-critical files. The reduced cost will make it possible to protect the much larger swath of file data from ransomware and other disasters. AI tools will also play a role in the equation as they can provide a more fine-grained level of classification by searching within the content of the files for sensitive PII and IP information.
In our survey mentioned above, 25% reported full self-service data management capabilities with the prevailing trend allowing employees to see analytics such as data growth and usage trends in their department, search for data across silos, and create custom workflows. Self-service capabilities are imperative across IT because of the limited bandwidth of most IT organizations and the urgency to bring new data services and easier data access to end users. Self-service tools will combine with automation to create policy-driven actions such as finding files with specific metadata (clinical images tagged with diagnostic codes) and sending to a cloud data lake for analysis. Storage and IT managers will be responsible for overseeing the tools and services to help departments and users efficiently find and analyze the data that they need.
