{"id":57370,"date":"2023-05-15T16:19:55","date_gmt":"2023-05-15T10:49:55","guid":{"rendered":"https:\/\/www.tothenew.com\/blog\/?p=57370"},"modified":"2023-09-06T16:21:23","modified_gmt":"2023-09-06T10:51:23","slug":"structured-task-scope-new-concurrency-model-in-jvm","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/structured-task-scope-new-concurrency-model-in-jvm\/","title":{"rendered":"Structured Task Scope – New Concurrency Model in JVM"},"content":{"rendered":"

Since the advent of virtual threads in Java, we have had a new and nice concurrency model introduced as\u00a0StructuredTaskScope<\/strong>. We will see some patterns in this new model.<\/p>\n

We need to know that with virtual threads, since they are very cheap, we don\u2019t need to pool them. Also, we need to know that a virtual thread can pull its stack from the underlying platform thread to the heap when blocking for any operations and, when complete, can pin its stack to any of the available platform threads.\u00a0This is new in Java, and it\u2019s great.<\/strong><\/p>\n

Consider the following code snippets:<\/p>\n

 public<\/span> static<\/span> Weather readFromServerA<\/span>() throws InterruptedException<\/span> {\r\n        Thread.sleep(RandomUtils.nextLong(1<\/span>, 100<\/span>));\r\n        return<\/span> new<\/span> Weather(\"Partly Sunny\"<\/span>, \"server-A\"<\/span>);\r\n    }\r\n    public<\/span> static<\/span> Weather readFromServerB<\/span>() throws InterruptedException<\/span> {\r\n        Thread.sleep(RandomUtils.nextLong(1<\/span>, 100<\/span>));\r\n        return<\/span> new<\/span> Weather(\"Partly Sunny\"<\/span>, \"server-B\"<\/span>);\r\n    }\r\n    public<\/span> static<\/span> Weather readFromServerC<\/span>() throws InterruptedException<\/span> {\r\n        Thread.sleep(RandomUtils.nextLong(1<\/span>, 100<\/span>));\r\n        return<\/span> new<\/span> Weather(\"Partly Sunny\"<\/span>, \"server-C\"<\/span>);\r\n    }<\/span><\/pre>\n
The above code returns weather information. It mimics a server and returns information in 1\u2013100 ms. Our requirement is to query all servers and get the results. The one that sends the result first will be considered, and our logic should be such that we should immediately cancel the other two requests (by interrupting them).<\/p>\n
Let\u2019s write this logic with the help of the\u00a0StructuredTaskScope<\/strong>\u00a0object.<\/p>\n
public<\/span> static<\/span> Weather readWeather<\/span>() throws ExecutionException, InterruptedException, TimeoutException<\/span> {\r\n        try<\/span> (var<\/span> scope = new<\/span> StructuredTaskScope.ShutdownOnSuccess<Weather>()){\r\n\r\n           \/\/ need to call fork and pass runnables or callables<\/span>\r\n           scope.fork(() -> Weather.readWeatherFromServerA());\r\n           scope.fork(() -> Weather.readWeatherFromServerB());\r\n           scope.fork(() -> Weather.readWeatherFromServerC());\r\n\r\n           \/\/ now we block, blocking in cheap in virtual threas<\/span>\r\n           \/\/ so no issue here<\/span>\r\n           scope.join<\/span>();\r\n\r\n           Weather weather = scope.result();\r\n           return<\/span> weather;\r\n\r\n}\r\n<\/span><\/pre>\n
The pattern is quite simple; we have used the\u00a0ShutdownOnSuccess<\/strong>\u00a0variation of\u00a0StructuredTaskScope<\/strong>, which is provided out of the box by the JVM.<\/p>\n
We just submit the three tasks one by one to fetch weather from different servers.\u00a0fork()<\/strong>\u00a0method accepts\u00a0callables<\/strong>.<\/p>\n
Then we block with the help of scope. join(), and why do we block? Why not now? Blocking is cheap, and we should encourage developers to block.<\/p>\n
Finally, we call the result method explicitly provided by the\u00a0scope.result()<\/strong>\u00a0method.<\/p>\n
Now let\u2019s see which request gets the result and which of the other two is cancelled.<\/p>\n
public<\/span> static<\/span> Weather readWeather<\/span>()<\/span> throws<\/span> InterruptedException, ExecutionException {\r\n\r\n        try<\/span>(var<\/span> scope<\/span> =<\/span> new<\/span> StructuredTaskScope<\/span>.ShutdownOnSuccess<Weather>();){\r\n\r\n            scope.fork(() -> Weather.readWeatherFromServerA());\r\n            scope.fork(() -> Weather.readWeatherFromServerB());\r\n            scope.fork(() -> Weather.readWeatherFromServerC());\r\n\r\n\r\n            Future<Weather> futureA = scope.fork(Weather::readWeatherFromServerA);\r\n            Future<Weather> futureB = scope.fork(Weather::readWeatherFromServerB);\r\n            Future<Weather> futureC = scope.fork(Weather::readWeatherFromServerC);\r\n\r\n            scope.join();\r\n\r\n            System.out.println(\"futureA.state() = \"<\/span> + futureA.state());\r\n            System.out.println(\"futureB.state() = \"<\/span> + futureB.state());\r\n            System.out.println(\"futureC.state() = \"<\/span> + futureC.state());\r\n\r\n            var<\/span> weather<\/span> =<\/span> scope.result();\r\n\r\n           \r\n\r\n            return<\/span> weather;\r\n        }\r\n\r\n}\r\n\r\n public<\/span> static<\/span> void<\/span> main<\/span>(String[] args)<\/span> throws<\/span> InterruptedException, ExecutionException {\r\n       var<\/span> weather<\/span> =<\/span> Weather.readWeather();\r\n\r\n    }<\/span><\/pre>\n
Let\u2019s run this programme quite a few times.<\/p>\n
WARNING: Using incubator modules: jdk.incubator.concurrent\r\nfutureA.state<\/span>() = FAILED\r\nfutureB.state<\/span>() = FAILED\r\nfutureC.state<\/span>() = SUCCESS<\/span><\/pre>\n
futureA.state<\/span>() = FAILED<\/span>\r\nfutureB.state<\/span>() = SUCCESS<\/span>\r\nfutureC.state<\/span>() = FAILED<\/span><\/span><\/pre>\n
We can see that different serves returned success, while the other two failed. It\u2019s just for explanation; it\u2019s technical code; we just need to query different servers as below.<\/p>\n
public<\/span> static<\/span> Weather readWeather<\/span>() throws ExecutionException, InterruptedException, TimeoutException<\/span> {\r\n        try<\/span> (var<\/span> scope = new<\/span> StructuredTaskScope.ShutdownOnSuccess<Weather>()){\r\n\r\n           \/\/ need to call fork and pass callables<\/span>\r\n           scope.fork(() -> Weather.readWeatherFromServerA());\r\n           scope.fork(() -> Weather.readWeatherFromServerB());\r\n           scope.fork(() -> Weather.readWeatherFromServerC());\r\n\r\n           \/\/ now we block, blocking is cheap in virtual threads<\/span>\r\n           \/\/ so no issue here<\/span>\r\n           scope.join<\/span>();\r\n\r\n           Weather weather = scope.result();\r\n           return<\/span> weather;\r\n\r\n}<\/span><\/pre>\n
And by the way, the above code can be written even more clearly with the help of method references.<\/p>\n
public<\/span> static<\/span> Weather readWeather<\/span>() throws InterruptedException, ExecutionException {\r\n        try<\/span>(var<\/span> scope = new<\/span> StructuredTaskScope<\/span>.ShutdownOnSuccess<Weather>();){\r\n            scope.fork<\/span>(Weather<\/span>::readWeatherFromServerA<\/span>);\r\n            scope.fork<\/span>(Weather<\/span>::readWeatherFromServerB<\/span>);\r\n            scope.fork<\/span>(Weather<\/span>::readWeatherFromServerC<\/span>);\r\n            scope.join<\/span>(); \r\n            var<\/span> weather = scope.result<\/span>();\r\n            return<\/span> weather;\r\n        }\r\n    }<\/span><\/pre>\n
How is it different from the ExecutorService?<\/strong><\/p>\n
The ExecutorService lifecycle runs along with the lifecycle of the application. Once the application is started, the executor service framework starts and shuts down once the application is terminated.<\/p>\n
But here, for every request call, StructuredTaskScope is created and destroyed as soon as we exit the method. We can do it because virtual threads are cheap \u2014 around 1000 times cheaper than platform threads.<\/p>\n
Secondly, the above code can be written with the help of complete stage APIs, but with those frameworks, we need to welcome callback hells and long nesting of codes. Who likes callback hell? Nobody.<\/p>\n
We will look into more such patterns. It\u2019s a completely new concurrency model, and we need to see exactly what patterns are going to become the norm, but I think that this is the pattern that we are going to use.<\/p>\n
One reminder: it will work on Java 19 and above with the preview feature enabled.<\/p>\n
This is just the start, in coming blogs, we will see some variations of StructuredTaskScope<\/strong>. Stay tuned.<\/p>\n
The original blog is here<\/a>.<\/p>\n
Explore our\u00a0blogs<\/a> for an in-depth understanding of various trending topics.<\/p>\n
<\/div>","protected":false},"excerpt":{"rendered":"
Since the advent of virtual threads in Java, we have had a new and nice concurrency model introduced as\u00a0StructuredTaskScope. We will see some patterns in this new model. We need to know that with virtual threads, since they are very cheap, we don\u2019t need to pool them. Also, we need to know that a virtual […]<\/p>\n","protected":false},"author":1520,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":5},"categories":[446],"tags":[5134,5225,5224],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/57370"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/1520"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=57370"}],"version-history":[{"count":3,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/57370\/revisions"}],"predecessor-version":[{"id":58364,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/57370\/revisions\/58364"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=57370"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=57370"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=57370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}