Tech - Search: Curation versus Algorithm

For those who remember the early days of the internet, they will be familiar with the free web hosting sites like Geocities. Before Geocities was taken offline, it's content was archived and is now available for download as a 650 GB torrent. While this seems like a large amount of data, it is estimated that around 2,551,568,697 GB data per day was created in 2011! While the earlier days of the internet were largely image and text based content, data today is in the form of images, text, social media posts, videos, businesses, services and user generated content.

All this only increases our dependency on search engines. More than the challenge of indexing all the data, search engines are starting to grapple with relevancy of results. The added complication is that some of the social media and other sites are not publicly available to be indexed. The two main search engines - Google and Bing seem to be taking markedly different approaches. Fundamentally, the approach is relying on either algorithms (by Google) or curation (by Bing). Let's look at their recent updates.

Power of algorithm

Google Search
In the above example, search for 'mozart' on google. The first set of results are all centred around Mozart, the music composer. So, how does Google do it? The recent updates to the search include the 'knowledge graph'. According to Google, this graph contains all the real-world entities and their relationships. It is a bold aim and I wouldn't be surprised if there was an element of curation in building the graph.

Power of crowd

Bing Search

Bing has decided to take a different approach. Rather than building an context-aware graph, it has decided to use the existing real-world connections. In the example above (currently available only in the US), a search for 'movies' shows the results from the search engine. In addition, it also shows results from your social graph (for now twitter and facebook). Of course, you need to be signed into all these services. Currently, Bing is a distant second in the search engine market. It will be interesting to see what kind of impact this latest update has.

What Next

But, what has not been solved so far is the 'intent' of the search query. There are times when we want to be influenced by our real-world connections and there are times when we want a fresh perspective. It will be interesting how Google and Bing solve this problem.

Also, how far will we take curation? For example, in most cases, currently there are algorithms which power everything - right from the balancing of books in a finance software to recommending music to users. Will some of these be replaced by curation? 

While we will continue to have algorithms, there is definitely potential for curation too. I think we have barely begun to scratch the surface.
Location: Bangalore, Karnataka, India

Comments

Post a Comment

Popular posts from this blog

Meta Prompting for Success

Image
Meta Prompt  Regardless of the Generative AI model used, sometimes the model may refuse to provide a response for a valid prompt. Often the reasons cited are - "I don't know", "I am not trained for the task", "violation of safety/responsible/ethical guardrails". This is sometimes more acutely felt when using Generative AI models for creating images. Overcoming this behavior can be frustrating and time consuming. An approach is to ask the AI model itself to generate prompts so that the prose and nuances of prompting are aligned with how the AI model has been built. This is called meta-prompting . The theory is that the prompts created by the AI model would have better chance of working. Let's dive in to see if and how this meta-prompting works in Gemini .  How to Create a Meta Prompt As with all good prompts, start with a persona/situation that describes the overall context. Next, describe the task in detail. In this case, it is about creating some ...
Read more

Book Review - The Price of Being Fair

Image
Overview  As the FairPrice group in Singapore completes 50 years, "The Price of Being Fair" book is a timely story of it's birth and evolution. The book has been authored by Sue-Ann Chia and Shing Huei Peh. While being true to history, the book has nuggets of "behind the scenes" information, given the time the authors have spent with various leaders at FairPrice. The book is already a best seller according to Kinokuniya book store. At under 200 pages, it makes for a breezy read. The Usual Suspects Having worked in retail industry for over two decades, the travails of  NTUC FairPrice seem familiar to me. A retail CEO once remarked that retail is a pretty "public" industry. There is not much behind which a retail business can hide. As such, retailers often have to follow the aspirations and concerns of the communities and customers they serve. The journey of FairPrice from being an EDLP (everyday low price) retailer to one having multiple formats, busin...
Read more

How to create a big picture

Image
Typically, you need wide view or fisheye lenses to capture a panoramic picture. An alternate method (though may not give the same quality) is to use software to create a panoramic image. There are several software available which you can use to create a panoramic image. The concept is called - "stitching". Some of the software available are - PTGui, Panavue, PAX-it, AutoStitch and so on. I sometimes use AutoStitch to create the panoramas since I have a 50mm lens for now. Each of these software vary in price and features. The added advantage is that you can take the pictures using any kind of camera - digital, phone or SLR. Steps: 1. Visualize the picture you want to take. 2. Use your camera and "divide" your image into rows and columns. An example being - 2 rows and 5 columns. 3. Take 10 pictures to cover your entire image. 4. Use any of the software to "stitch" them together. Here is an example of a panorama created using AutoStitch. I to...
Read more