Generally, to export data from a MongoDB collection<\/strong>, we use the following command –
\n[shell]
\nmongoexport -vvvv –host 127.0.0.1 –db my-db –username=’usr’ –password=’pwd’ –collection profiles –csv –out \/home\/data\/usernames.csv -f ‘username’
\n[\/shell]<\/p>\nDefinitely I prefer the above mentioned way. But let’s consider that you have millions or billions of records in a MongoDB collection. And you want to export all these.
\nProbably, this will just hang. Well, I am not sure what happens on your computer but on my machine it was just stuck for hours and then I terminated the process. And came up with scripting solution – where I created multiple files (chunks) of the exported data.<\/p>\n
<\/p>\n
[javascript]
\nvar size = 1000000;
\nvar maxCount = 1;
\nfor (x=0;x<maxCount;x=x+1)
\n{
\n var recToSkip = x*size;
\n db.profiles.find().skip(recToSkip).limit(size).forEach( function(record){
\n var username = record.username;
\n print(record.username);
\n })
\n}<\/p>\n
[\/javascript]<\/p>\n
And ofcourse to run the script (if above javascript is in file named – ‘usernames.js’)
\n[shell]
\n mongo 127.0.0.1\/my-db –username=’username’ –password=’pwd’ export-usernames.js > output.txt
\n[\/shell]<\/p>\n
The above shell command will render all the output (from export-usernames.js) to the output.txt file. And mind it, that this is single file where you get all exported data. But if you want to chunkify this – just write your wrapper script or run multiple times. It’s upto you.<\/p>\n
<\/p>\n
I hope this might help someone. Or please feel free to add your comments.<\/p>\n
<\/p>\n
Cheers!
\nSalil Kalia
\nSalil [at] IntelliGrape [dot] com
\nTwitter LinkedIn<\/p>\n","protected":false},"excerpt":{"rendered":"
In this post, we will learn how to use javascript as MongoDB shell script to export (fields of) a collection. If you just want to execute MongoDB commands then look at the another post there.. OK, so let’s focus back to the objective of this post. Generally, to export data from a MongoDB […]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":26},"categories":[1],"tags":[55,260,4846,453],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/6026"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=6026"}],"version-history":[{"count":0,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/6026\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=6026"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=6026"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=6026"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}