{"id":13023,"date":"2014-04-18T20:00:11","date_gmt":"2014-04-18T14:30:11","guid":{"rendered":"http:\/\/www.tothenew.com\/blog\/?p=13023"},"modified":"2014-04-19T11:15:14","modified_gmt":"2014-04-19T05:45:14","slug":"apache-http-conditional-policy-requests-for-checking-whether-resource-is-modified-or-not-and-reduce-unnecessary-processing","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/apache-http-conditional-policy-requests-for-checking-whether-resource-is-modified-or-not-and-reduce-unnecessary-processing\/","title":{"rendered":"Apache HTTP Conditional Policy Requests for checking whether resource is modified or not and reduce unnecessary processing"},"content":{"rendered":"

Hi,<\/p>\n

Before writing this blog, I would first like to make you familiar with one quote “Be liberal in what you accept, and conservative in what you send”<\/em><\/strong>. Yes that’s what HTTP robustness protocol states. <\/p>\n

Scenario :<\/strong>
\nWe have a use case where our server need to read XML files that are large containing 400 outer element with extremely nested elements from provider server.<\/p>\n

Tasks Involved :<\/strong><\/p>\n

    \n
  1. Our server will run good amount quartz scheduling job for fetching XML file of different scenario’s from a server and process them differently.<\/li>\n
  2. Each job will run per minute to monitor the content, as real time content may change or not, which has to be displayed in real time to the end users.<\/li>\n<\/ol>\n

    Problem :<\/strong>
    \nReading, validating and processing large XML<\/strong> every minute to make them available to end user“even if file content is unmodified”<\/strong><\/em>. It will make unnecessary overhead for our server and consumption of IO operation even when data is not modified.<\/p>\n

    \"Don't<\/p>\n

    It’s like our server not able to find relevant data and ending up reading same data every time polluting the server environment<\/strong> with processing of large responses.<\/p>\n

    How to clean our server environment, from large unnecessary processing? Solution : HTTP conditional policy request using mod_policy<\/strong><\/p>\n

    Yes you can make conditional request “If-Modified-Since”<\/strong><\/em> to provider to determine whether the file is being modified or not.<\/p>\n

    Request Flow :<\/strong><\/p>\n

      \n
    1. Our server will make request via adding conditional header, “If-Modified-Since”, Using ApacheHTTPClient 4.3\n
      \r\n      apacheHTTPClientAdapter.requestObject.setHeader(\"If-Modified-Since\",\"Fri, 17 Jan 2014 19:10:11 GMT\");\r\n      \/\/Note : I have used ApacheHttpClient 4.3 to make request\r\n<\/pre>\n
       
       \n<\/li>\n
    2. Or Using HTTP URL Connection :\n
      \r\n     URL url = new URL({file});\r\n     HttpURLConnection http = (HttpURLConnection)url.openConnection();\r\n     Http. addRequestProperty (\"If-Modified-Since\", LastUpdate);\r\n     http.connect();\r\n<\/pre>\n<\/li>\n
    3. Now, provider server mod_policy filter check for the GMT lastUpdated date you have specified in “If-Modified-Since” header value of request, so :\n