OK, so let’s focus back to the objective of this post.
Generally, to export data from a MongoDB collection, we use the following command –
mongoexport -vvvv –host 127.0.0.1 –db my-db –username=’usr’ –password=’pwd’ –collection profiles –csv –out /home/data/usernames.csv -f ‘username’
Definitely I prefer the above mentioned way. But let’s consider that you have millions or billions of records in a MongoDB collection. And you want to export all these.
Probably, this will just hang. Well, I am not sure what happens on your computer but on my machine it was just stuck for hours and then I terminated the process. And came up with scripting solution – where I created multiple files (chunks) of the exported data.
var size = 1000000;
var maxCount = 1;
var recToSkip = x*size;
var username = record.username;
mongo 127.0.0.1/my-db –username=’username’ –password=’pwd’ export-usernames.js > output.txt
The above shell command will render all the output (from export-usernames.js) to the output.txt file. And mind it, that this is single file where you get all exported data. But if you want to chunkify this – just write your wrapper script or run multiple times. It’s upto you.
I hope this might help someone. Or please feel free to add your comments.
Salil [at] IntelliGrape [dot] com