{"id":23940,"date":"2015-07-27T22:16:38","date_gmt":"2015-07-27T16:46:38","guid":{"rendered":"http:\/\/www.tothenew.com\/blog\/?p=23940"},"modified":"2015-07-29T10:31:07","modified_gmt":"2015-07-29T05:01:07","slug":"mongo-delayed-node","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/mongo-delayed-node\/","title":{"rendered":"Mongo Delayed Node"},"content":{"rendered":"

 <\/p>\n

This was my live experience with Mongo<\/a> when I felt need of something like Delayed Replica. I had a Mongo ReplicaSet setup for some client with 3 nodes – 1 Primary and 2 Secondaries. It is known that whatever is written to Primary\u2019s Oplog, is replicated to the secondaries immediately without any lag or lag of some milliseconds (as per network capacity). Any disastrous command if issued by mistake on Primary will also impact the data residing in secondaries. That is what happened !! Someone having root rights inadvertently issued a \u2018drop\u2019 command on Primary which made complete application to go down since it also wiped out all the data on Secondaries too. To avoid such \u2018harmful replication\u2019, we should choose to implement a Delayed Replica Node in our setup.<\/p>\n

So, in next paragraphs, I would be focussing on configuring a Delayed Replica node in existing Mongo Replica set which causes a replication delay (lag) between Primary and Delayed Node.<\/p>\n

What is Delayed Node<\/h3>\n

A delayed node is like any secondary member in a Replica Set which have a \u2018replication lag\u2019 from Primary. That means this node will fetch group of operations from Primary\u2019s oplog & replay them with some predefined delay. This becomes extremely useful in those conditions where there are high chances of accidental commits on Mongo.<\/p>\n

Such Delayed Node can not become Primary while election. Also they are kept hidden from applications to prevent reading lagged data.<\/p>\n

How to configure a Delayed Node<\/h3>\n

This goes in continuation with adding members into a Replica Set on Primary. Click here<\/a> for more detail. Let\u2019s go for an example:<\/p>\n

\u00a0\u00a0\u00a0 primary.domain.in <\/i> -> Primary Mongo Node<\/i>
\n\u00a0\u00a0\u00a0 secondary1.domain.in<\/i> -> First Secondary Mongo Node.<\/i>
\n\u00a0\u00a0\u00a0 secondary2.domain.in<\/i> -> Second Secondary Mongo Node.<\/i>
\n\u00a0\u00a0\u00a0 delayed.domain.in<\/i> -> Third \u2018delayed\u2019 Secondary Mongo Node.<\/i><\/p>\n

Connect to to-be Primary Host:<\/b><\/h3>\n

[js]$ mongo –host primary.domain.in –port 27017 [\/js]<\/p>\n

Initialize the replica set:<\/b><\/h3>\n

[js]> rs.initiate() [\/js]<\/p>\n

Adding normal secondary Nodes:<\/b><\/h3>\n

[js]myReplicaset:PRIMARY> rs.add("secondary1.domain.in:27017")[\/js]<\/p>\n

[js]myReplicaset:PRIMARY> rs.add("secondary2.domain.in:27017")[\/js]<\/p>\n

\u00a0<\/strong><\/strong><\/p>\n

Adding Delayed secondary Node:<\/b><\/h3>\n


\n<\/strong><\/strong><\/p>\n

[js]myReplicaset:PRIMARY> cnf = {_id: ‘myReplicaset’, members: [{_id: 0, host: ‘primary.domain.in:27017’}, {_id: 1, host: ‘secondary1.domain.in:27017’}, {_id: 2, host: ‘secondary2.domain.in:27017’}, {_id: 3, host: ‘delayed.domain.in:27017’, priority: 0, hidden: true, slaveDelay: 600}]}[\/js]<\/p>\n

\"cnf\"<\/p>\n

 <\/p>\n

Note: <\/i> <\/span><\/p>\n

priority: Indicates the relative eligibility of a member to become a<\/i> primary.<\/i><\/p>\n

hidden: Replica Set hides this instance and does not include the member in the output of<\/i><\/p>\n

db.isMaster()<\/i><\/a> or<\/i> isMaster<\/i><\/a><\/p>\n

slaveDelay: Number of seconds \u201cbehind\u201d the primary that this replica set member should<\/i><\/p>\n

\u201clag\u201d. Here we have choosen 600 seconds (10 minutes).<\/i><\/p>\n

myReplicaset: Replica Set name.<\/i><\/p>\n

\u00a0<\/strong><\/strong><\/p>\n

Reconfiguration of Existing Replica Set<\/b><\/h3>\n

\u00a0<\/strong><\/strong><\/p>\n

[js]myReplicaset:PRIMARY> rs.reconfig(cnf)<\/p>\n

myReplicaset:PRIMARY> rs.conf()[\/js]<\/p>\n

\u00a0<\/strong><\/strong><\/p>\n

Check Slave Delay parameters<\/b><\/h3>\n

\u00a0<\/strong><\/strong><\/p>\n

[js]myReplicaset:PRIMARY> rs.conf()
\n{
\n\u00a0\u00a0 \u00a0"_id" : "myReplicaset",
\n\u00a0\u00a0 \u00a0"version" : 4,
\n\u00a0\u00a0 \u00a0"members" : [
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0{
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"_id" : 0,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"host" : "primary.domain.in:27018",
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"arbiterOnly" : false,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"buildIndexes" : true,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"hidden" : false,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"priority" : 1,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"tags" : {
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0},
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"slaveDelay" : 0,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"votes" : 1
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0},
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0{
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"_id" : 1,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"host" : "secondary1.domain.in:27019",
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"arbiterOnly" : false,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"buildIndexes" : true,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"hidden" : false,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"priority" : 1,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"tags" : {
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0},
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"slaveDelay" : 0,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"votes" : 1
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0},
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0{
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"_id" : 2,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"host" : "secondary2.domain.in:27020",
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"arbiterOnly" : false,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"buildIndexes" : true,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"hidden" : false,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"priority" : 1,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"tags" : {
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0},
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"slaveDelay" : 0,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"votes" : 1
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0},
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0{
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"_id" : 3,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"host" : "delayed.domain.in:27021",
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"arbiterOnly" : false,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"buildIndexes" : true,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"hidden" : true,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"priority" : 0,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"tags" : {
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0},
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"slaveDelay" : 600,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"votes" : 1
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0}
\n\u00a0\u00a0 \u00a0],
\n\u00a0\u00a0 \u00a0"settings" : {
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"chainingAllowed" : true,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"heartbeatTimeoutSecs" : 10,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"getLastErrorModes" : {
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0},
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"getLastErrorDefaults" : {
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"w" : 1,
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0"wtimeout" : 0
\n\u00a0\u00a0 \u00a0\u00a0\u00a0 \u00a0}
\n\u00a0\u00a0 \u00a0}
\n}[\/js]<\/p>\n

 <\/p>\n

 <\/p>\n

Warning<\/b>: rs.reconfig() can make Primary to step down which usually causes an election to choose a new Primary. Recommendation is to make these changes during scheduled downtime.<\/p>\n

Post these changes, any commit on Primary node will be replicated to the Delayed Node after \u2018slaveDelay\u2019 seconds, however rest secondary nodes will immediately apply it.<\/p>\n","protected":false},"excerpt":{"rendered":"

  This was my live experience with Mongo when I felt need of something like Delayed Replica. I had a Mongo ReplicaSet setup for some client with 3 nodes – 1 Primary and 2 Secondaries. It is known that whatever is written to Primary\u2019s Oplog, is replicated to the secondaries immediately without any lag or […]<\/p>\n","protected":false},"author":181,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":7},"categories":[1],"tags":[1596],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/23940"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/181"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=23940"}],"version-history":[{"count":0,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/23940\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=23940"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=23940"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=23940"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}