Tech - NOSQL Tutorial - Part 2 (Overview)


I started looking at various NOSQL databases like Hbase, CouchDB and MongoDB. I finally settled on MongoDB, mainly because it was easy to setup in Windows. The name Mongo is derived from 'humongous', interesting name! For more details, you can refer - http://www.mongodb.org. As it says on the website, it is very easy to setup and highly scalable. Here are my first set of statistics -
  • Time to download the software (64 bit edition) - 2 minutes
  • Time to setup the database - 2 minutes
  • Time to get the database up and running - 1 minute
And, this is where I found the database to be refreshing. It is quick, even to setup, leave alone processing of data! Overall, I found these to be key features of this database -
  • Light foorprint. Easy to setup either single or replicated instances and to administer.
  • Schema-free data storage. In other words, there is no need to define any data structure.
  • Incredibly quick for reads and writes.
  • Map Reduce (with Javascript).
  • Very easy to scale horizontally.
  • RESTful API's available to access the database.
I will cover some of these features in separate posts.


Here is the next set of statistics around some complimentary software.
  • RockMongo - Web (PHP) based administration console. Time to download - 1 minute. Time to configure - 1 minute.
  • MongoLive - Web (Chrome App) based monitoring tool. Time to download - 1 minute. Time to configure - 1 minute.
Of course there are many other tools available. The default mongo (default console client) is the best tool to access the database. And with these web based tools, it's such a change from using big and slow clients which are required for some of the RDBMS databases. You can refer the mongodb website for help and tutorials.


One of the key features I wanted to try out was the performance of the database. So, I had to get a 'meaty' set of data. I generated my own dummy data with about 20 million rows and 10 columns.


I used the default mongoimport tool to import the data. It took about 3 hours to import. Next, I created the indexes. They took another 5 hours, a bit more time than I expected. All set now to test it. I also have a web page (using PHP) that queries the data based on the key. The time it took to access the data was 0.0083 seconds! Of course this is with 1 query running at a time. Wait for the next post on performance testing, it's MongoDB vs SQL Server 2008!

Click here for - Part 3.
Location: Welwyn, Welwyn, Hertfordshire, UK

Comments

Post a Comment

Popular posts from this blog

Meta Prompting for Success

Image
Meta Prompt  Regardless of the Generative AI model used, sometimes the model may refuse to provide a response for a valid prompt. Often the reasons cited are - "I don't know", "I am not trained for the task", "violation of safety/responsible/ethical guardrails". This is sometimes more acutely felt when using Generative AI models for creating images. Overcoming this behavior can be frustrating and time consuming. An approach is to ask the AI model itself to generate prompts so that the prose and nuances of prompting are aligned with how the AI model has been built. This is called meta-prompting . The theory is that the prompts created by the AI model would have better chance of working. Let's dive in to see if and how this meta-prompting works in Gemini .  How to Create a Meta Prompt As with all good prompts, start with a persona/situation that describes the overall context. Next, describe the task in detail. In this case, it is about creating some ...
Read more

Book Review - The Price of Being Fair

Image
Overview  As the FairPrice group in Singapore completes 50 years, "The Price of Being Fair" book is a timely story of it's birth and evolution. The book has been authored by Sue-Ann Chia and Shing Huei Peh. While being true to history, the book has nuggets of "behind the scenes" information, given the time the authors have spent with various leaders at FairPrice. The book is already a best seller according to Kinokuniya book store. At under 200 pages, it makes for a breezy read. The Usual Suspects Having worked in retail industry for over two decades, the travails of  NTUC FairPrice seem familiar to me. A retail CEO once remarked that retail is a pretty "public" industry. There is not much behind which a retail business can hide. As such, retailers often have to follow the aspirations and concerns of the communities and customers they serve. The journey of FairPrice from being an EDLP (everyday low price) retailer to one having multiple formats, busin...
Read more

How to create a big picture

Image
Typically, you need wide view or fisheye lenses to capture a panoramic picture. An alternate method (though may not give the same quality) is to use software to create a panoramic image. There are several software available which you can use to create a panoramic image. The concept is called - "stitching". Some of the software available are - PTGui, Panavue, PAX-it, AutoStitch and so on. I sometimes use AutoStitch to create the panoramas since I have a 50mm lens for now. Each of these software vary in price and features. The added advantage is that you can take the pictures using any kind of camera - digital, phone or SLR. Steps: 1. Visualize the picture you want to take. 2. Use your camera and "divide" your image into rows and columns. An example being - 2 rows and 5 columns. 3. Take 10 pictures to cover your entire image. 4. Use any of the software to "stitch" them together. Here is an example of a panorama created using AutoStitch. I to...
Read more