{"id":56856,"date":"2023-03-13T13:07:35","date_gmt":"2023-03-13T07:37:35","guid":{"rendered":"https:\/\/www.tothenew.com\/blog\/?p=56856"},"modified":"2023-03-17T12:13:41","modified_gmt":"2023-03-17T06:43:41","slug":"java-project-loom-virtual-threads-part-1","status":"publish","type":"post","link":"https:\/\/www.tothenew.com\/blog\/java-project-loom-virtual-threads-part-1\/","title":{"rendered":"Java Project Loom – Virtual Threads (Part 1)"},"content":{"rendered":"

This JEP No 425 is something I am waiting for with bated breath. It\u2019s a new notion that has been added to the concurrent\u00a0APIs of JDK 19.<\/strong>\u00a0It\u2019s in the preview stage, and soon it will be a permanent feature in\u00a0JDK<\/strong>\u00a0in the coming few releases.<\/p>\n

Virtual Threads are light weight that reduce the effort of writing, maintaining and observing high throughput applications.<\/strong><\/p>\n

In this part, we will see the goals of this JEP.<\/p>\n

The first one is to enable server applications written in the simple thread-per-request style to scale with near-optimal hardware utilisation.<\/p>\n

Let\u2019s understand the \u201cthread per request<\/strong>\u201d model. Well, for an HTTP server, it means that each HTTP request is handled by its own thread. For a relational database server, it means that each SQL transaction is also handled by its own thread. Simple stuff.<\/p>\n

One request = one transaction = one thread.<\/strong><\/p>\n

What is the cost of this model?\u00a0<\/strong>Well, to understand this cost, we need to understand the cost of a thread in Java. A Java Thread, as created in the very early versions of Java is a thin wrapper on a platform thread, also called\u00a0Operating System Thread.<\/strong><\/p>\n

\"\"<\/p>\n

Now, some mathematical stuff coming underway. Take a calculator by your side.<\/p>\n

There are two things we need to know about them. First, a platform thread needs to store its call stack in memory. For that, 20MB are reserved upfront in memory. Second, it is a system resource. It takes about 1 ms to launch a platform thread. So, 20MB of memory, 1ms to launch. A platform thread is in fact a rather expensive resource.<\/p>\n

How can we optimise hardware utilisation with such threads?<\/p>\n

Suppose we have\u00a016GB<\/strong>\u00a0of memory available for your application. Divided by\u00a020MB\u00a0<\/strong>for a thread, you have room for\u00a0800 threads(16 * 1000 \/ 20)<\/strong>\u00a0on such a machine. Suppose these threads are doing some I\/O, like accessing resources on a network. And suppose that this resource is accessed in\u00a0100 milliseconds<\/strong>. Preparing the request and processing the response will be done in the order of 500\u00a0nanoseconds\u00a0<\/strong>each(as seen above.). Suppose all this in-memory computations are taking\u00a01000<\/strong>\u00a0nanoseconds<\/strong>. Now from figure, since\u00a01ms<\/strong>\u00a0<\/em>=\u00a01000,000 nanoseconds<\/strong>, it means that there is a factor in the order of\u00a0100,000\u00a0<\/strong>between the preparation of the request and the processing of the response.\u00a0During this, our thread is sitting there, doing nothing. Shelling thousands of dollars on CPU to make it sit idle. What a pity?<\/strong><\/p>\n

So if we have\u00a0800<\/strong>\u00a0such threads, our CPU will used at\u00a00.8%<\/strong>.\u00a0Less than 1%<\/strong>. And if we double the memory to\u00a032GB<\/strong>, it will be used at\u00a01.3%<\/strong>. If we want a CPU usage of 90%, then we need 90,000 such threads. Launching them will take 90s, that is 1 minute and a half, and they will consume\u00a01.8 TB of memory<\/strong>. If such RAM is available on market, I am pretty sure, only few people can afford it. So clearly, platform threads are far too expensive to scale with near-optimal hardware utilisation.\u00a0Project Loom is addressing this problem.<\/strong><\/p>\n

The second goal is to enable the existing code built on the classical Java Threads to adopt Virtual Threads with minimal changes. This goal is also quite ambitious, because it means that everything you could do with classical Threads, you should be able to do it in the same way with virtual Threads. That covers several key points:<\/p>\n

\n

First<\/strong>, virtual thread can run any Java code or any native code.<\/p>\n

Second<\/strong>, you do not need to learn any new concepts.<\/p>\n

Third<\/strong>, but you need to unlearn certain ideas. Virtual threads are cheap, about 1000 times cheaper than classical platform threads, so trying to avoid blocking a virtual thread is useless. Writing classical blocking code is OK. And that\u2019s a good news, because blocking code is so much easier to write than asynchronous code.<\/p>\n<\/blockquote>\n

Is it a good idea to pool virtual threads?<\/strong>\u00a0Answer is a big no-no, since its cheap, no need to pool them, create and throw on demand.<\/p>\n

Two more good news with virtual threads First,\u00a0thread local\u00a0<\/strong>variables also work in the same way. And second,\u00a0synchronization<\/strong>\u00a0also works. Now several things needs to be said about synchronization. A virtual thread is still running on top of a platform thread. There is still a platform thread underneath. The trick is, this virtual thread can be detached from its platform thread so that this platform thread can run another virtual thread.\u00a0When would it be detached?<\/strong>\u00a0Well, a virtual thread can be detached from its platform thread as soon as it is blocking. It could be blocked on an I\/O operation, or on a\u00a0synchronization<\/strong>\u00a0operation, or when it is put to\u00a0sleep<\/strong>.<\/p>\n

One caveat, if a virtual thread is executing some code inside a\u00a0synchronized<\/strong>\u00a0block, it cannot be detached from its platform thread. So during the time it is running the\u00a0synchronized<\/strong>\u00a0block of code, it blocks a platform thread. If this time is short that\u2019s ok, there is no need to panic. If this time is long, that is, if it\u2019s doing some long I\/O operation, then it\u2019s not a great situation, we may need to do something. We need to replace\u00a0synchorized<\/strong>\u00a0block with Reentrant Lock APIs. This problem with\u00a0synchronized<\/strong>\u00a0blocks may be solved in the future, in fact it may be solved by the time virtual threads become a final feature of the JDK.<\/p>\n

Reactive programming<\/strong>\u00a0is also available on the market and is also trying to solve this problem. But the problem with reactive is that it\u2019s a complete paradigm shift in the way we code.\u00a0It\u2019s hard to learn, hard to understand, hard to profile the code, even harder to debug, and a nightmare to write test cases<\/strong>.<\/p>\n

Hope you have understand the problems this Project Loom is trying to solve. In coming parts, we will look into some coding. I am excited about this project. Let me know about your reaction in comments.<\/p>\n

This blog is originally published in Medium, read here<\/a>.<\/p>\n

\n

<\/div>","protected":false},"excerpt":{"rendered":"

This JEP No 425 is something I am waiting for with bated breath. It\u2019s a new notion that has been added to the concurrent\u00a0APIs of JDK 19.\u00a0It\u2019s in the preview stage, and soon it will be a permanent feature in\u00a0JDK\u00a0in the coming few releases. Virtual Threads are light weight that reduce the effort of writing, […]<\/p>\n","protected":false},"author":1520,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":3},"categories":[446],"tags":[5134,5133],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/56856"}],"collection":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/users\/1520"}],"replies":[{"embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/comments?post=56856"}],"version-history":[{"count":4,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/56856\/revisions"}],"predecessor-version":[{"id":56892,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/posts\/56856\/revisions\/56892"}],"wp:attachment":[{"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/media?parent=56856"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/categories?post=56856"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.tothenew.com\/blog\/wp-json\/wp\/v2\/tags?post=56856"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}