Author Archive

01: Forget “Search” — It Is All About The “Virtual Aggregation” Of Data

I mentioned in a post late last year that 2010 would be the year of federated search, but now I think we need to go even further …

The number of record and content management systems that are coming onto the market (and the investment that is being made by vendors and customers alike) is simply staggering.  In isolation, a large number of these systems are good applications; some are even great … with some tremendous functionality.  But having spent time at the HP TRIM User Forum (TUF23) in Sydney last week, I can say they all have one thing in common – they require end users to know where content is in order to find it.

Think about this for a minute … if you know your content is in HP TRIM, then the search in TRIM will allow you to find what you are looking for, same if you know your content is in SharePoint.  However, no matter what role you have in an organisation, chances are you will interact with more than one of these data repositories.  From an enterprise perspective we cannot continue to expect our users to know where information lives.

The issue today is that data needs to live in different places.  These different repositories are purpose-built for the type of information that they contain.  The term “federated search” does not really do the concept justice … more accurately, what we are doing with federated search is providing users with the ability to find information by “virtually aggregating data” or allowing users to ask for some information that is likely scattered across multiple repositories.  

We can even take this one step further, following on from the commentary about “physical” data aggregation projects that are being embarked upon in Europe and the US after some high-profile information failures.  If you have a physical data aggregation project going on in your organisation, stop right now and see if there is a better way to accomplish the end result you are looking for. 

We can wrestle with terminology all we want (CMSWatch’s Theresa Regli recently presented some great thoughts on the related topic of “enterprise search” vs “federated search”), as long as we all agree that the true benefit of this technology is to locate and leverage information where it resides.

02: Open Source Search – Ready For Prime Time?

Well… it really depends on who you are and where you are. In the US, vendors like Lucid Imagination are making a business out of the support of Lucene/Solr. There are even commercial search vendors developing products off a Lucene base. Open source search outside the US is not as prevalent. We see it to some degree in Europe, but not in Asia Pacific. When IT budgets are tight, doing some prototype development with open source to prove the merits of a business case certainly has value. Just make sure you are clear on the end goal of that prototype. Turning a prototype into a final commercial product only works if the technology was selected correctly in the first place. A lot of prototyping technology is designed specifically to generate a prototype, and do it quickly. Moving to a final product requires something different again.

But back to the original question…it really comes down to a question of your company’s appetite for risk. At a recent conference, the CIO of a large financial institution was talking about her attitude to open source in general. In her words – when something goes wrong you can’t hold the open source developer community accountable; sometimes you just need a commercial entity in there with you that you can put some pressure on, the “one throat to choke” so to speak.

Now you may be in a company that does embrace open source and is willing to tolerate the General Public License (GPL) aspects. If this is the case and you have enough expertise in house to deal with anything that goes wrong, then perhaps it is for you. If you do go down the Open Source Search route, be sure to come talk to us about the ISYS File Readers. The availability of document filters is clearly a key area of open source search that’s not ready for prime time.

03: Security for federated search

Thank you to all those who commented on the recent post, and as a number of you pointed out, yes, security is a challenge in an environment with multiple data repositories. Stay tuned for our upcoming whitepaper that will tell you everything you need to know about security in a federated search environment. We’ll do our best to have it out by the end of January!

04: 2010 – the year of Federated Enterprise Search?

How the landscape has changed…ten years ago we had many different file types, but not many places to store them. Today, we have fewer file types, but many more repositories – add the complexity of the different security models into the mix and clearly the focus of enterprise search is changing. It is good to see the issue of federated search finally being raised, witness recent articles in the likes of Information Week.

The question that enterprises need to ask is who is best placed to provide true federated search? My opinion is that it is not the owner of any one of the repositories —  it is those companies that have an independent view of data. We will be hearing more about this issue in 2010 as our customers demand better access to information, and don’t care what email system of document repository their data resides in.