{"id":46553,"date":"2017-02-24T17:02:48","date_gmt":"2017-02-24T11:32:48","guid":{"rendered":"http:\/\/www.tothenew.com\/blog\/?p=46553"},"modified":"2017-02-24T17:04:57","modified_gmt":"2017-02-24T11:34:57","slug":"advantages-of-an-etl-tool-in-content-migration","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/advantages-of-an-etl-tool-in-content-migration\/","title":{"rendered":"Advantages of an ETL Tool in Content Migration"},"content":{"rendered":"<p>Recently, I was introduced to a new technology called ETL (Extract, Transform and Load) which I can use to simplify and to increase the productivity of my daily tasks as AEM migration developer. There are various ETL tools available, some of them are listed <a title=\"ETL Tools Available in Market\" href=\"https:\/\/www.etltool.com\/list-of-etl-tools\">here<\/a>.\u00a0I chose Talend, as it is built on top of Eclipse IDE and supports custom Java code.<\/p>\n<p>Outlined below are some of the advantages almost all ETL tools have over writing custom migration scripts from scratch:<strong><br \/>\n<\/strong><\/p>\n<ol>\n<li style=\"font-weight: 400\"><b>Visual flow<\/b><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">: All ETL tools provide a visual flow of the system&#8217;s logic which makes the migration process a lot more interactive than creating the scripts in a plain IDE. Generally, there is no manual coding required to create a basic <a title=\"content migration overview\" href=\"http:\/\/www.tothenew.com\/blog\/basics-of-content-migration-manual-and-automated\/\">migration process<\/a> due to the availability of the out-of-the-box modules. This reduces the learning curve for any new adopters.<\/span><\/span><a title=\"Talend Visual Flow\" href=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Visual-Flow-example.png\" target=\"_blank\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-46554\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Visual-Flow-example.png\" alt=\"Visual Flow Example\" width=\"749\" height=\"113\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Visual-Flow-example.png 749w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Visual-Flow-example-300x45.png 300w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Visual-Flow-example-624x94.png 624w\" sizes=\"(max-width: 749px) 100vw, 749px\" \/><\/a><\/li>\n<li style=\"font-weight: 400\"><b>Choice of Programming language<\/b><span style=\"font-weight: 400\">: Whenever there is a need to write code, it can be written in a programming language of your choice. This totally depends on coding languages supported by an ETL tool. This makes the tool more comfortable to use and hence reduces the learning curve.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Modularity<\/b><span style=\"font-weight: 400\">: An ETL tool is completely based on independent modules which are like black boxes and are only concerned about the input they receive and the output they produce. You can very easily plug in\/out, using drag and drop\/delete option respectively without disturbing other modules.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Performance<\/b><span style=\"font-weight: 400\">: The optimized and performance tested modules help achieve a task in the least time possible.\ufffc<\/span><\/li>\n<li style=\"font-weight: 400\"><b>Easy understanding and maintenance<\/b><span style=\"font-weight: 400\">: The click and drag programming snippets help achieve a basic task even if a person doesn&#8217;t know any programming language like append or replace some text in strings. The visual aspect helps in increasing understanding of the logic and in easy maintenance of the scripts. <\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><strong>Here are a few examples:<\/strong><\/p>\n<ul>\n<li style=\"font-weight: 400;text-align: left\"><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">A &#8216;split&#8217; module will visually show multiple workflows.<br \/>\nExample: Selecting one of the two different templates based on an input value is shown below. You can easily see the workflow for a\u00a0Page Template.<br \/>\n<\/span><\/span><a title=\"Talend Split Module Example\" href=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Split_Module.png\" target=\"_blank\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-46558\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Split_Module.png\" alt=\"Talend Split Module\" width=\"1029\" height=\"418\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Split_Module.png 1029w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Split_Module-300x121.png 300w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Split_Module-1024x415.png 1024w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Split_Module-624x253.png 624w\" sizes=\"(max-width: 1029px) 100vw, 1029px\" \/><\/a><\/li>\n<li style=\"font-weight: 400;text-align: left\"><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">Mapping the same input\u00a0to two different output formats at the same time.<br \/>\nExample: Same input row generates an output XMLs. It is also logged in an excel file as shown below.<br \/>\n<\/span><\/span><a title=\"Talend Multiple Outputs\" href=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talen-Multiple-Outputs.png\" target=\"_blank\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-46560\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talen-Multiple-Outputs.png\" alt=\"Talend Multiple Outputs\" width=\"579\" height=\"225\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talen-Multiple-Outputs.png 579w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talen-Multiple-Outputs-300x116.png 300w\" sizes=\"(max-width: 579px) 100vw, 579px\" \/><\/a><\/li>\n<li style=\"font-weight: 400;text-align: left\"><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">Selecting output format based on an input or custom value.<br \/>\nExample: Adding a selective condition to above example to log only those page URLs where a property named &#8220;Abstract&#8221; is either absent or empty in the Input as shown below.<br \/>\n<\/span><\/span><a title=\"Talend Selective Outputs\" href=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend-Selective-Outputs.png\" target=\"_blank\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-46561\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend-Selective-Outputs.png\" alt=\"Talend Selective Outputs\" width=\"583\" height=\"128\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend-Selective-Outputs.png 583w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend-Selective-Outputs-300x65.png 300w\" sizes=\"(max-width: 583px) 100vw, 583px\" \/><\/a><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400;text-align: left\"><b>Productivity<\/b><span style=\"font-weight: 400\"><span style=\"font-weight: 400\">: <a title=\"web content management service\" href=\"http:\/\/www.tothenew.com\/wcm\/web-content-management-services\">Content management<\/a> and migration is extremely challenging. Less boilerplate code and minimal &#8216;coding&#8217; increases the productivity of the migration process. An end-to-end basic ETL process can be achieved very quickly by adding just the out-of-the-box modules with basic configurations. For an Example: No file handling or parsing code is needed to read\/write any common file formats such as text, CSV, XML and Excel. Below image shows some of the many modules provided by default in <a title=\"ETL tool for content migration\" href=\"http:\/\/www.tothenew.com\/blog\/content-migration-using-talend\/\">Talend <\/a>for taking input in the form of a file:<br \/>\n<\/span><\/span><a title=\"Talend_File_Input_Modules\" href=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend_Input_Modules.png\" target=\"_blank\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-46556 aligncenter\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend_Input_Modules.png\" alt=\"Talend_Input_Modules\" width=\"478\" height=\"594\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend_Input_Modules.png 478w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend_Input_Modules-241x300.png 241w\" sizes=\"(max-width: 478px) 100vw, 478px\" \/><\/a><\/li>\n<li style=\"font-weight: 400\"><strong><strong><b>Easy refactoring<\/b><span style=\"font-weight: 400\">: Refactoring is a matter of replacing or adding a new input source box and a little configuration.<\/span><span style=\"font-weight: 400\"><br \/>\n<\/span><span style=\"font-weight: 400\">Example: As shown in the below image, Changing the input source from XML to SQL requires replacing the input module from XML to SQL and minimal configurations if the schema is same.<br \/>\n<\/span><\/strong><\/strong><a title=\"Talend Change Input\" href=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend-Change-Input.png\" target=\"_blank\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-46562\" src=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend-Change-Input.png\" alt=\"Talend Change Input\" width=\"522\" height=\"301\" srcset=\"\/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend-Change-Input.png 522w, \/blog\/wp-ttn-blog\/uploads\/2017\/02\/Talend-Change-Input-300x172.png 300w\" sizes=\"(max-width: 522px) 100vw, 522px\" \/><\/a><\/li>\n<\/ol>\n<p>With an advancement in <a title=\"digital technologies\" href=\"http:\/\/www.tothenew.com\/technologies\">digital technologies<\/a> and content centric approach for marketing, it is now mission critical for companies to use right web content management systems such as <a title=\"AEM Development Service\" href=\"http:\/\/www.tothenew.com\/wcm\/cq-aem-development-consulting\">AEM <\/a>or <a title=\"Drupal Development Service\" href=\"http:\/\/www.tothenew.com\/wcm\/drupal-development-consulting-services\">Drupal<\/a>. Talend and many such ETL tools could be extremely useful in migrating content and we hope this blog was helpful to you in understanding ETL and Talend.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently, I was introduced to a new technology called ETL (Extract, Transform and Load) which I can use to simplify and to increase the productivity of my daily tasks as AEM migration developer. There are various ETL tools available, some of them are listed here.\u00a0I chose Talend, as it is built on top of Eclipse [&hellip;]<\/p>\n","protected":false},"author":535,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":16},"categories":[1],"tags":[4847,258,2007,4458,2008],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/46553"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/535"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=46553"}],"version-history":[{"count":0,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/46553\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=46553"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=46553"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=46553"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}