Skip to main content

Migrating from MS SQL Server to MongoDB

The following contains notes from various attempts at migrating 2.5GB of MS SQL Server data to MongoDB, on an 8 GB, quad-core, 64-bit Windows 7 Enterprise machine.

  1. [TERMINATED] http://mssql2mongo.codeplex.com/ : Simple to use. Exponential slowdown observed in migration. For a total of the 50 million records spanning two tables, the following migration times were observed: 1 second for the first 100,000 records,  30 minutes for 1,000,000, 20 hours for 16 million (after which I terminated the process).
  2. [DID NOT WORK] http://rubydoc.info/gems/mongify/ : A ruby-based approach. Use Ruby 1.9.3 (tiny_tds dependency causes problems with Ruby 2.0). Install DevKit before installing the mongify gem. Also, use 'sqlserver' as the adapter in the .config file. Then, before running 'mongify check <config-file>', install the gems activerecord-sqlserver-adapter and tiny_tds. At this stage, it fails. 'mongify translation <config-file>' fails as well.
  3. [MODIFIED/WORKED!!] http://mssql2mongo.codeplex.com/ :
    1. introduced concurrency (since both SQL SERVER read and MongoDB write support it!)
    2. tweaked and customized page size for each table (trial-and-error)
    3. This version works great -- about 45 minutes to migrate the whole data.
    4. Note: Ensure that indices are created on the primary keys of all the tables.
    5. Only one source file is changed (see patch)


Popular posts from this blog

the brave new economy

A free(er) market is emerging and has been emerging ever since the advent of the Internet and the Web. Newer technologies have accelerated this. Old inefficiencies in knowledge and access to the market are quickly disappearing. Participation is increasingly peer-to-peer (P2P). Participants can produce, create, curate, give, lend, sell, share, auction, consume, take, borrow, buy, reuse, rent and/or barter goods, services, cash, credit, currency, equity, debt and/or knowledge. The resultant economy is highly collaborative and is sometimes referred to as the ‘sharing economy’. Efficiencies are being introduced and value is being created in the smallest of markets. Non-profit activities are thriving as well, in addition to for-profit ones. The marketplace is becoming, at once, global and local. An important subset of the new economy is collaborative and crowdsourced, and may be described as commons-based peer production. Asymmetries in information are diminished by algorithms (Google, of …

Prohibition of envy in Judaism, Christianity and Islam

"One of the sanest, surest, and most generous joys of life comes from being happy over the good fortune of others." (attributed variously to Robert Heinlein & Archibald Rutledge).


The idea of envy (or, covetousness) is an important subject in the Abrahamic faith traditions of Judaism, Christianity and Islam.

The prohibition against covetousness rounds out the Mosaic moral code, codified as the 10th and final commandment in the Ten Commandments -- the injunction against coveting one's neighbor's property. The rationale behind this is the acknowledgement that all blessings arise from God's sovereign providence and it is not for his creation to question whom and how and when he chooses to bless.

Merriam-Webster's dictionary defines covetousness as :

1. "feeling or showing a very strong desire for something that you do not have and especially for something that belongs to someone else", or
2. "marked by inordinate desire for wealth or possessi…