Feeds:
Posts
Comments

Posts Tagged ‘Data Management’

Some of you may remember the big buzz around Information Life-cycle Management or ILM.  EMC pushed the concept of ILM a few years back and many of their competitors followed them down this winding road.  You know that marketing campaigns are working when customers talk about ILM strategies using some of the same language as their vendors.  I witnessed this fairly extensively with ILM but the reality never matched the rhetoric.  

On some measure ILM has been successful.  A number of customers went to a multi-tier storage environment.  Some never moved data but actually became smarter about where they placed it to begin with.  Others would actually move data at either the volume level or if they were using file systems – at the file or file system level.  During this time a number of technologies and vendors came and went and when the dust settled there was modest levels of ILM but nowhere near the promise of the hype.  

The term ILM is rarely used these days and it is not going to open any doors for you.  However, just because we never reached the nirvana of ILM doesn’t mean that there wasn’t real value in the concept.  

In my view, the goal of ILM was to move data transparently to the appropriate storage tier balancing performance, protection and cost.  And the end result of implementing ILM included significant cost reductions and better utilization of your expensive IT infrastructure.  

But the mega-hype around ILM actually over-complicated it and created confusion.  There was and is no magic application or technology that could just make it all happen with a push of a button.  However, with the combination of people, process and technology there are great strides that can be made.  In fact, I know of IT professionals that have saved tons of money by implementing some form of ILM.  I submit that some level of ILM – regardless of what you call it – should be a requisite part of every data center.  In fact, it should be as fundamental a part of the data center as disaster recovery.

Read Full Post »

As many of you may know Microsoft acquired FAST Search and Transfer in 2008 for a sizable sum of money ($1.2 billion).   For those of you that aren’t familiar with FAST, there were a fairly successful search and indexing company that was used by Enterprise end user companies.  From the onset there seemed to be a real desire to marry FAST with Microsoft SharePoint and after some twists and turns that is still the strategy – see this Beyond Search blog entry for more insights.

Certainly it makes sense for Microsoft to integrate FAST with SharePoint but what about the rest of the Enterprise?  Yes, Microsoft wants to have all unstructured content be stored in SharePoint but since that hasn’t happened yet and it may never be a fully realized goal – it is important to have Enterprise search be heterogeneous. 

One thing that many readers may not know is that FAST did a great job partnering with storage vendors including EMC and HDS – just to name two of the biggest.  It is uncertain what is going to happen with these relationships based on the fact that Microsoft seems to be focused on making FAST work only with Microsoft products.

It appears that Microsoft still really doesn’t know what to do with its consumer or Enterprise search strategy.  They have a ton of smart people, lots of resources at their finger tips, etc, etc.  However, search software seems to be a perpetual stumbling block.

Read Full Post »

I was an industry analysts for many years. I focused heavily on storage systems and was convinced that search and storage would eventually be like peanut butter and jelly. Although we have not seen the realization of this yet – I am still convinced that it needs to and will ultimately occur. However, like all things in the data center, it just takes time.

There are practical reasons why storage and search aren’t more bounded together. For one, search solutions haven’t been scalable or intelligent enough to provide the value that IT professionals are looking for. Second, most search solutions have been associated with specific storage systems and not the entire storage complex. That is very limiting. We need Enterprise search solutions that can access all storage within an organization. The third big issue is that storage adminstrators haven’t figured out why they need it. There are some applications and use cases that are a priority – such as eDiscovery. But storage admins have not found the killer app that gives them that “aha” moment where they just need to have it and are willing to invest time and money.

What is the killer app for search and storage? I believe one killer app is using a universal search application as a tool to give Enterprise end users greater access to the company’s data. We create so much content, using any number of applications, and instead of looking for data via the various application interfaces, having a single pane of glass, to get to any and all content in the Enterprise, would provide huge increases in productivity and efficiency.

This concept should not be a leap for most people , but since no one is complaining about it or demanding it, it isn’t a priorty. However, if you think about the power of being able to easily access content – data – information – we all know that mountains can be moved when this ability is provided. This is where storage admins have to transcend their nuts-and-bolts view of the world and think about the business and how they can apply technology to elevate the companies they work for. It is “right brain” thinking (creative) versus the typical “left brain” logical and rational thinking that is typically needed in the data center. Only by combining the creative and the logical can real leaps forward be made.

Read Full Post »

Patternicity – defined by Michael Shermer – a writer for Scientific American – is the tendency to find meaningful patterns in meaningless noise.  When I read this article on Pattnernicity I immediately related it to the challenges we face with information access. 

Patternicity deals with false positives and we have a compartive with search tools – too many responses that may or may not be what we are looking for.  Human Patternicity is meant to err on the side of caution because as Shermer points out – “the cost of believing that the rustle in the grass is a dangerous predator when it is just the wind is relatively low compared with the opposite. Thus, there would have been a beneficial selection for believing that most patterns are real.”

Digital Patternicity is also meant to err on the side of caution because the cost of believing that the keyword matches your intentions is relatively low compared with returning a false negative.  Therefore returning a false positive is better than returning a false negative. 

The problem in both Human and Digital Patternicity is that the algorithms are limited and have stopped evolving because they don’t need to improve.   Human beings are very successful and don’t require more sophisticated methods for returning fewer false positives.  Likewise, search companies like Google are very successful and have built a huge business in spite of the number of false positives they return. 

However, increasingly within the world of business – where information equates to revenue, competitive advantage and market growth – there is a big price to pay with false positives and a shift in the evolution of Digital Patternicity must occur.  There will always be a place for acceptable false positives in the mass market – but when you get to specialization, when the stakes become too high, when survival is at risk – then evolution aggressively adapts.

Read Full Post »

I’m a senior consultant and founder for the INI Group and am working very closely with Digital Reef as a consultant, advisor and blogger. I’ve been in the high tech industry for over 23 years with a focus on the data management and storage arena and you can find out more about me at www.contemplatingIT.com. I believe Digital Reef has brought to the table an extremely impressive solution at a critical time when our unstructured data content is growing to massive levels.

In addition to being and advisor and consultant for Digital Reef – I’m going to be blogging for them on a regular basis discussing a wide range of topic areas from business issues, compelling technology, market dynamics and visions going forward.

Who and what is Digital Reef? They are a startup – an emerging vendor – that came to right conclusion that Enterprise search is woefully inadequate on multiple levels – the mechanics of making it work efficiently and intelligently in environments with massive amounts of content; and the ability to get relevant data to the user rapidly and without drowning them with irrelevant results.

I describe the Digital Reef solution as a data and content management platform leveraging intelligent and scalable search and indexing technologies. Digital Reef provides appliances with a grid architecture that ingests and indexes massive amounts of content spread across heterogeneous storage throughout the Enterprise creating a global federated index. Some of the biggest challenges with indexing include scalability, transparency and true global federation – and Digital Reef solves all three.

Once you have all of your unstructured data indexed – what are you going to do with it? Another big challenge with management of unstructured data is making order out of chaos. If you just use keyword searches there will be a large number of irrelevant returns that obscure what you really need.

The problem with keywords is that there is very little useful context. Digital Reef’s magic ingredient is its similarity engine – the ability to analyze content including documents, email threads and terms and return to you results based on a user defined similarity ratio. Digital Reef’s similarity engine is sophisticated technology that not only uses keywords but the associations of terms and the context in which they are used within unstructured data – providing relevant results.

Companies are frustrated because information is really three dimensional but we are using two dimensional tools to access and manage them. The first step is to implement solutions that provide us rapid access to relevant data for reactive purposes such as a discovery process, audits, research, customer support, projects, etc. However, think of the potential of really using information to also build revenue generating products and services leveraging existing intellectual property. The potential is compelling and landscape changing.

Read Full Post »