{"id":54875,"date":"2022-04-19T17:17:14","date_gmt":"2022-04-19T11:47:14","guid":{"rendered":"https:\/\/www.tothenew.com\/blog\/?p=54875"},"modified":"2024-06-10T15:40:38","modified_gmt":"2024-06-10T10:10:38","slug":"scaling-aem-using-aws-s3-with-tar-mk","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/scaling-aem-using-aws-s3-with-tar-mk\/","title":{"rendered":"Scaling AEM using AWS-S3 with TAR-MK"},"content":{"rendered":"

Prerequisites\u00a0<\/span><\/h2>\n
    \n
  1. AEM 6.5<\/span><\/li>\n
  2. Latest Service pack (I am using 6.5.8 here).<\/span><\/li>\n
  3. S3 connector (Feature pack version 1.10.10)<\/span><\/li>\n
  4. Amazon S3 bucket. <\/span><\/li>\n<\/ol>\n

    Required feature pack\u00a0<\/span><\/h2>\n

    com.adobe.granite.oak.s3connector-1.10.10.zip<\/i><\/b><\/p>\n

    Note :S3 connector feature pack has dependency over the service packs, as it uses oak as dependencies and AEM should be updated with a equal\/higher version of oak than the one s3 connector is using , look for the service packs POM to know which version of OAK is being used.<\/span><\/p>\n

    <oak.version>1.22.2<\/oak.version><\/span><\/p><\/blockquote>\n

    Run modes<\/span>\u00a0<\/b><\/h2>\n

    AEM needs to be started using<\/span> crx3tar-nofds<\/i><\/b> run mode if S3 Data store to be configured with TarMK.<\/span><\/p>\n

    java -jar <\/span><<\/span>aem-jar-file<\/span>><\/span>.jar -r crx3tar-nofds<\/span><\/p><\/blockquote>\n

    AEM has two stores\u00a0<\/span><\/h2>\n
      \n
    1. \n

      Node Store
      \n<\/b><\/h3>\n

      Content nodes are stored in a node store.<\/span><\/h3>\n

      Segment Node Store for TarMK and Document Node Store for MongoMK<\/span><\/i><\/p>\n

      The segment node store is the basis of Adobe\u2019s TarMK implementation in AEM6.<\/span><\/p>\n

      PID : <\/span>org.apache.jackrabbit.oak.segment.SegmentNodeStoreService<\/span><\/p>\n

      Segment Node Configuration Options<\/span><\/h4>\n\n\n\n\n\n\n
      Options\u00a0<\/b><\/td>\nDescription<\/b><\/td>\nDefault value<\/b><\/td>\n<\/tr>\n
      repository.home<\/span><\/td>\nPath to repository home under which repository-related data is stored.<\/span><\/td>\ncrx-quickstart\/segmentstore<\/span><\/td>\n<\/tr>\n
      tarmk.size<\/span><\/td>\nMaximum size of a segment in MB<\/span><\/td>\n256 MB<\/span><\/td>\n<\/tr>\n
      customBlobStore<\/span><\/td>\nBoolean value indicating that a custom data store is used.<\/span><\/td>\nTrue<\/b> for AEM 6.3 and later versions<\/span><\/p>\n

      False<\/b> for Prior to AEM 6.3<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

       <\/p>\n

      \u00a0<\/b><\/h3>\n<\/li>\n
    2. \n

      Data Store<\/b><\/h3>\n<\/li>\n<\/ol>\n

      The binary data is stored in a data store.<\/span><\/p>\n

      Data Store Configuration : Amazon S3 Data Store<\/span><\/h3>\n

      PID : org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore.config<\/span><\/p>\n

      Steps to configure Amazon S3 as Data store –<\/span><\/p>\n

        \n
      1. Extract the contents of the feature pack zip file to a temporary folder.<\/span><\/li>\n
      2. Go to the temporary folder and navigate to the following location: <\/span>jcr_root\/libs\/system\/install<\/span><\/li>\n
      3. Copy all the contents from the above location to <\/span><aem-install>\/crx-quickstart\/install<\/span><\/li>\n
      4. copy <\/span>org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore.config <\/b>file from the following folder:<\/span><\/li>\n<\/ol>\n

        <feature pack>\/jcr_root\/libs\/system\/config<\/span>\u00a0<\/span><\/p><\/blockquote>\n

        to\u00a0<\/span><\/p>\n

        <aem-install>\/crx-quickstart\/install<\/span><\/p><\/blockquote>\n

          \n
        1. Edit the file and add the configuration options required by your setup.<\/span><\/li>\n
        2. Start AEM.<\/span><\/li>\n<\/ol>\n

          AWS S3 connector configuration options<\/span><\/h4>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
          Options<\/b><\/td>\nDescription<\/b><\/td>\nDefault value<\/b><\/td>\n<\/tr>\n
          accessKey<\/span><\/td>\nThe AWS access key<\/span><\/td>\n<\/td>\n<\/tr>\n
          secretKey<\/span><\/td>\nThe AWS secret access key. <\/span>
          \n<\/span>Note: Alternatively, IAM roles can be used for authentication. If you are using IAM roles you no longer need to specify the accessKey and secretKey.<\/span><\/td>\n          		<\/td>\n<\/tr>\n
          s3Bucket<\/span><\/td>\nThe bucket name.<\/span><\/td>\n<\/td>\n<\/tr>\n
          s3Region<\/span><\/td>\nThe bucket region.<\/span><\/td>\n<\/td>\n<\/tr>\n
          path<\/span><\/td>\nThe path of the data store.<\/span><\/td>\n<AEM install folder>\/repository\/datastore<\/span><\/td>\n<\/tr>\n
          minRecordLength<\/span><\/td>\nThe minimum size of an object that should be stored in the data store.<\/span><\/td>\n16KB<\/span><\/td>\n<\/tr>\n
          maxCachedBinarySize<\/span><\/td>\nBinaries with size less than or equal to this size will be stored in the memory cache. The size is in bytes.<\/span><\/td>\n17408<\/span><\/td>\n<\/tr>\n
          cacheSize<\/span><\/td>\nThe size of the cache. The value is specified in bytes.\u00a0<\/span><\/td>\n64GB<\/span><\/td>\n<\/tr>\n
          secret<\/span><\/td>\nOnly to be used if using binaryless replication for shared datastore setup.<\/span><\/td>\n<\/td>\n<\/tr>\n
          stagingSplitPercentage<\/span><\/td>\nThe percentage of cache size configured to be used for staging asynchronous uploads.<\/span><\/td>\n10<\/span><\/td>\n<\/tr>\n
          uploadThreads<\/span><\/td>\nThe number of upload threads that are used for asynchronous uploads.\u00a0<\/span><\/td>\n10<\/span><\/td>\n<\/tr>\n
          stagingPurgeInterval<\/span><\/td>\nThe interval in seconds for purging finished uploads from the staging cache.<\/span><\/td>\n300 seconds<\/span><\/td>\n<\/tr>\n
          stagingRetryInterval<\/span><\/td>\nThe retry interval in seconds for failed uploads.<\/span><\/td>\n600 seconds<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

          Observation<\/strong><\/h3>\n

          All the binaries will be stored under S3, except the one has smaller size than <\/span>minRecordLength<\/b><\/p>\n

          \"\"<\/p>\n

           <\/p>\n

          Reference\u00a0<\/b><\/h3>\n
            \n
          • https:\/\/experienceleague.adobe.com\/docs\/experience-manager-65\/deploying\/deploying\/data-store-config.html?lang=en<\/li>\n<\/ul>\n

             <\/p>\n

            <\/div>","protected":false},"excerpt":{"rendered":"

            Prerequisites\u00a0 AEM 6.5 Latest Service pack (I am using 6.5.8 here). S3 connector (Feature pack version 1.10.10) Amazon S3 bucket. Required feature pack\u00a0 com.adobe.granite.oak.s3connector-1.10.10.zip Note :S3 connector feature pack has dependency over the service packs, as it uses oak as dependencies and AEM should be updated with a equal\/higher version of oak than the one […]<\/p>\n","protected":false},"author":1212,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":111},"categories":[5868,1174],"tags":[4847,1332],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/54875"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/1212"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=54875"}],"version-history":[{"count":3,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/54875\/revisions"}],"predecessor-version":[{"id":54925,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/54875\/revisions\/54925"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=54875"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=54875"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=54875"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}