{"id":71714,"date":"2025-05-06T10:45:57","date_gmt":"2025-05-06T05:15:57","guid":{"rendered":"https:\/\/www.tothenew.com\/blog\/?p=71714"},"modified":"2025-05-13T14:22:31","modified_gmt":"2025-05-13T08:52:31","slug":"seamless-migration-from-bitbucket-to-github-a-step-by-step-guide-and-key-takeaways-part-1","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/seamless-migration-from-bitbucket-to-github-a-step-by-step-guide-and-key-takeaways-part-1\/","title":{"rendered":"Seamless migration from Bitbucket to GitHub: A step-by-step guide and key takeaways \u2013 Part 1"},"content":{"rendered":"

Introduction<\/span><\/h2>\n

In the rapidly changing landscape of software today, companies have to constantly reassess their tools to remain competitive. This article explains the journey of migrating code repositories from Bitbucket to GitHub for an Enterprise customer, a leading university in Australia, as Github has better collaborative features, smoother integrations, and a developer-friendly ecosystem. With 1,600 repositories to be migrated with history intact, branches, tags, and metadata, manual migration was not an option. Therefore, a custom Bash script was designed to fully automate the migration. This blog will examine the difficulties faced, the tactics adopted to surmount them, and how the migration was effectively implemented within the agreed-upon timeline.<\/p>\n

Problem statement<\/h2>\n

The customer had been using Bitbucket as their primary version control platform older than 5 years but decided to transition to GitHub to take advantage of its enhanced collaboration features, tighter integration with CI\/CD workflows, and a more developer-friendly ecosystem.<\/p>\n

There were 1600 repos in Bitbucket required to be migrated to GitHub.<\/p>\n

Known challenges before migration:<\/h3>\n

The key challenges encountered:<\/p>\n

    \n
  1. Lack of Existing Tools: The search for migration tools that could meet the specifications led to disappointment since there was no direct match. This led to the creation of a custom Bash script for the needs of migration.<\/li>\n
  2. Repository Structure Differences: In Bitbucket, repositories exist under projects; in GitHub, all repositories need to exist under a single organization directly. Dynamic modification of the API URLs had to be performed during the migration process.<\/li>\n
  3. Feature and Format Differences: Some of its features e.g. User login exp. Client wanted to have Organization managed users on GitHub (SSO) with an enterprise-managed user base.\u00a0 On Bitbucket, user authentication was based on traditional SSO. Groups on GitHub were synchronized with the IDP, whereas on Bitbucket the access was managed on the project level which didn\u2019t necessarily require user groups.<\/li>\n
  4. Metadata Preservation: A critical part of the migration was ensuring that every repository\u2019s commit history, branches, and tags were transferred accurately. Managing this while addressing the platform-specific differences required extra attention.<\/li>\n
  5. Change in remote URLs: The repository URL has changed due to the migration, particularly in shared library code. This requires removing project-level dependencies in the Jenkinsfile and shared libraries.<\/li>\n<\/ol>\n

    Preparation work<\/h2>\n

    The feasibility of the migration has been assessed to determine which features need to be migrated. The checklist is provided below.<\/p>\n

    \"checklist\"

    checklist<\/p><\/div>\n

    To ensure a smooth and successful migration of repositories from Bitbucket to GitHub, the following key steps were followed:<\/p>\n

      \n
    1. Creation of repos.csv:\u00a0 A repos.csv file was prepared as input for the migration script, listing project keys and repository names in the following format: PROJECT_KEY, REPO_NAME<\/li>\n
    2. Configuration of variable.var File which is responsible for carrying certain variables that are needed during migration.<\/li>\n<\/ol>\n

      Migration planning<\/h2>\n

      Migration Planning, Script Development, and Execution \u2013 TTN Team<\/strong><\/p>\n

      The TTN team is responsible for designing the migration strategy, developing the required automation scripts, and executing the migration process. This includes:<\/p>\n

        \n
      1. Identifying repositories and defining the migration approach<\/li>\n
      2. Creating and testing migration scripts<\/li>\n
      3. Executing the migration in a structured and phased manner<\/li>\n<\/ol>\n

        Review and Validation \u2013 Customer Team<\/strong><\/p>\n

        The Customer team checks all the repositories migrated and verifies that they are correct and unchanged. Major areas of work are:<\/p>\n

          \n
        1. Verification of repository structure, commit history, metadatas<\/li>\n
        2. Branch protection and correct permissions<\/li>\n
        3. Find mismatches and resolve them after migration.<\/li>\n<\/ol>\n

          Effective migration requires careful planning to ensure a smooth and successful transition. Below are the key steps involved in the planning phase of the migration:<\/p>\n

            \n
          1. Prepare Input File<\/strong>: Created repos.list<\/span> containing project keys and repository names.<\/li>\n
          2. Pre-Checks:\u00a0<\/strong>preChecks.script<\/span> Scripts will check the presence of repositories on both Bitbucket and GitHub with their current state and also will reduce the redundant values from the input file created.<\/li>\n
          3. Migration Execution<\/strong>:\u00a0 getInventory.script<\/span> to transfers repositories with full history.<\/li>\n
          4. Group Permissions<\/strong>: getBitbucketPerm.script<\/span> & mapPermGithub.scriptUsed scripts to retrieve and map Bitbucket group permissions to GitHub.<\/li>\n
          5. Default Branch<\/strong>: Retrieved and set the default branch in GitHub using defaultBranch.script<\/span>.<\/li>\n
          6. Branch Protection Rules<\/strong>: brProtectRule.script<\/span> applies branch protection rules after setting the default branch.<\/li>\n
          7. Submodule URL Update:<\/strong>\u00a0 updateSubmodUrl.script<\/span> \u00a0applies submodule links to reference GitHub repositories.<\/li>\n
          8. Jenkins URL Update:<\/strong> applied Jenkins changes and repository URL changes across repo updates using updateJfRepoURL.script<\/span><\/li>\n
          9. Post-Migration Report:<\/strong> Generated a report to identify inconsistencies and verify migration success using migrationReport.script<\/span><\/li>\n
          10. Check Discrepancies\/Fix Issues:<\/strong> Reviewed and resolved any issues from the migration report. validateMigration.script<\/span><\/li>\n
          11. Archive GitHub Repositories:<\/strong> Archived successfully migrated repositories after validation archiveGithub.script<\/span><\/li>\n<\/ol>\n

            Execution flow diagram<\/h2>\n

             <\/p>\n

            \"execution

            execution flow diagrams<\/p><\/div>\n

            Migration Workflow<\/h2>\n

            The repository migration process was carefully structured to ensure consistency, accuracy, and minimal disruption. Here’s a high-level overview of the steps involved, which can be visualized in the accompanying flow diagram:<\/p>\n

              \n
            1. Repository Selection<\/strong>
              \nWe began by finalizing the list of repositories that needed to be migrated. This included identifying repositories across projects, both active and legacy.<\/li>\n
            2. Environment Variable Update<\/strong>
              \nEnvironment variables were updated to reflect the necessary credentials and configurations required for seamless migration.<\/li>\n
            3. Migration Script Execution<\/strong>
              \nA custom script (migraterepo) was executed to handle the actual migration. The script preserved the existing directory structure and metadata of each repository.<\/li>\n
            4. Legacy Project Handling<\/strong>
              \nIf a repository belonged to a legacy project, it was migrated to GitHub.Archived post-migration in GitHub.The original Bitbucket repository was also archived.If the repository was not part of a legacy project, we proceeded with further updates<\/li>\n
            5. Post-Migration Updates for Active Repositories<\/strong>Module Check:<\/strong> We checked if the repository contained any submodules.
              \nIf submodules were found, their URLs were updated accordingly.Go Projects:<\/strong> For Go-based repositories, we made updates to go.mod and other relevant files to reflect new library paths or dependencies post-migration.<\/li>\n
            6. Inventory Comparison<\/strong>
              \nAfter the migration and updates, we compared the GitHub repository inventory with the original Bitbucket inventory to identify any discrepancies and ensure completeness.<\/li>\n
            7. Batch Processing<\/strong>
              \nIf more repositories were pending, the above process was repeated for the next batch. This cycle continued until all repositories were successfully migrated and verified.<\/li>\n<\/ol>\n

              Architecture Diagram<\/h2>\n
              \"Architecture

              Architecture diagram<\/p><\/div>\n

              Seamless execution:<\/h2>\n

              With around 1,600 repositories to migrate, the process was divided into three manageable batches.<\/p>\n

                \n
              1. Batch 1:<\/strong> The first batch included less complex repositories for a smoother initial migration.<\/li>\n
              2. Batch 2:<\/strong> Dealing with more complex repositories that involve extra considerations.<\/li>\n
              3. Batch 3:<\/strong> Resolved the remaining repositories according to the insights from the previous batches.<\/li>\n<\/ol>\n

                Note<\/strong>: A topic in GitHub was added, tracking the migrated repository count. The workflow does not incorporate reusing Git LFS, references, or additional difficulties handling.<\/p>\n

                In the end, the migration was a success, with all repositories successfully moved to GitHub.<\/p>\n

                In the next part of this blog series, we\u2019ll dive deeper into the challenges we faced<\/strong>, the impact this migration<\/strong> we had, and the lessons we learned<\/strong> along the way – along with the best practices<\/strong> that helped us keep things smooth and scalable.<\/p>\n","protected":false},"excerpt":{"rendered":"

                Introduction In the rapidly changing landscape of software today, companies have to constantly reassess their tools to remain competitive. This article explains the journey of migrating code repositories from Bitbucket to GitHub for an Enterprise customer, a leading university in Australia, as Github has better collaborative features, smoother integrations, and a developer-friendly ecosystem. With 1,600 […]<\/p>\n","protected":false},"author":1628,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":332},"categories":[2348],"tags":[1853,4279,375,503,1703],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/71714"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/1628"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=71714"}],"version-history":[{"count":16,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/71714\/revisions"}],"predecessor-version":[{"id":71951,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/71714\/revisions\/71951"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=71714"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=71714"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=71714"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}