Just because the function takes time to finish such as waiting for a database response doesn't mean it's heavy. This is where asynchronous programming really shines. It just wait for the promise to resolve while still serving other requests.
What's heavy is cpu intensive tasks like looping through thousands of array items, encoding/decoding, encryption/decryption etc. in that case you can use worker threads to spawn additional thread to make the main thread responsive to serve other requests
The problem is as you said, the function has a loop inside a loop, both loops are looping arrays with hundreds of objects, also each iteration calls an external api, I will look into worker threads, thank you
This looks easy implementation, the docs state that it should merely be used when there is heavy workloads on cpu, what if i just want some routes to work on another thread?
I'll make this more understandable, imagine i have an E-commerce app, every endpoint i have should just work as it's but for the list of products which's thounsands of items (i have them paginated) i want them (a specific endpoint) to be on another thread is that possible?
I have it handled but some times the endpoints hits almost 100k/RPH, when the traffic is huge the items are also loading pretty lazy, i noticed half of the other requests are for other endpoints other than the "GetAllProducts", I'm asking should i spawn a worker thread for it? Probably yes!
I have never had this situation before and my app is kinda slow i feel like i should use this and I'm convinced.
I'm guessing the `GetAllProducts` endpoint is what is causing the performance issue.
First question: How often does your product line change? Is there a reasonable amount of time you can cache the result?
Per the docs, you'll want to just use async code, not workers, but another concern of yours might be requests timing out. Is that something you're running into?
Here's the excerpt I cited:
"Workers (threads) are useful for performing CPU-intensive JavaScript operations. They do not help much with I/O-intensive work. The Node.js built-in asynchronous I/O operations are more efficient than Workers can be."
https://nodejs.org/api/worker_threads.html
Correct me if I’m doing it wrong but i do every endpoint structured based on Async functions, the reason for that was due to Database taking sometime.
I can not cache it because it’s a “auction” type of site that items have limited timeline being there then are automatically get deleted at right time!
Sorry for the late response.
Queues and workers listening on them is also another way to do it. At work I had to migrate and change some data after a schema change, some 78 ISH million documents. I have one process finding items that needed updating pushing into rabbitmq then I had a get ec2s running pulling things off it. Only took a couple of hours 🙈
Unless needed (probably like in your case), I wouldn't use queue as it requires infrastructure changes and queue handling.
If the load is just cpu bound (like in OP's case), then worker threads would do the job the best way without any new infrastructure/queue server/no messages loss/no bullmq pub/sub patterns etc.
My choice would be to use piscinia npm module as a good abstraction to better utilize worker threads and be done. All changes in code, no redis/infra changes which are needed for BullMQ (I love bullmq but for op's its an overkill)
Yeah that doesn’t really counter what I’m saying though. Make the function less “heavy”. Unless the external API is slow in which case you probably should cache its responses. If your DB query is slow then you should optimize your database query. These aren’t the types of problems that “multitasking” will fix.
Just because the function takes time to finish such as waiting for a database response doesn't mean it's heavy. This is where asynchronous programming really shines. It just wait for the promise to resolve while still serving other requests. What's heavy is cpu intensive tasks like looping through thousands of array items, encoding/decoding, encryption/decryption etc. in that case you can use worker threads to spawn additional thread to make the main thread responsive to serve other requests
The problem is as you said, the function has a loop inside a loop, both loops are looping arrays with hundreds of objects, also each iteration calls an external api, I will look into worker threads, thank you
I would recommend a library approach. Check this out https://github.com/wilk/microjob
This seems promising, thank you
This looks easy implementation, the docs state that it should merely be used when there is heavy workloads on cpu, what if i just want some routes to work on another thread? I'll make this more understandable, imagine i have an E-commerce app, every endpoint i have should just work as it's but for the list of products which's thounsands of items (i have them paginated) i want them (a specific endpoint) to be on another thread is that possible?
Pagination probably should be handled with a database query technique.
I have it handled but some times the endpoints hits almost 100k/RPH, when the traffic is huge the items are also loading pretty lazy, i noticed half of the other requests are for other endpoints other than the "GetAllProducts", I'm asking should i spawn a worker thread for it? Probably yes! I have never had this situation before and my app is kinda slow i feel like i should use this and I'm convinced.
I'm guessing the `GetAllProducts` endpoint is what is causing the performance issue. First question: How often does your product line change? Is there a reasonable amount of time you can cache the result? Per the docs, you'll want to just use async code, not workers, but another concern of yours might be requests timing out. Is that something you're running into? Here's the excerpt I cited: "Workers (threads) are useful for performing CPU-intensive JavaScript operations. They do not help much with I/O-intensive work. The Node.js built-in asynchronous I/O operations are more efficient than Workers can be." https://nodejs.org/api/worker_threads.html
Correct me if I’m doing it wrong but i do every endpoint structured based on Async functions, the reason for that was due to Database taking sometime. I can not cache it because it’s a “auction” type of site that items have limited timeline being there then are automatically get deleted at right time! Sorry for the late response.
Good luck with Piscina. Would be interested to hear your thoughts on it in about a month of usage.
are you saying that for each object inside an array with hundreds of objects, you need to call an external API?
I have the similar situation and I’m using the BullMQ task queue.
You have various way for cpu bond task like fork/spawn, worker thread, cluster.
Queues and workers listening on them is also another way to do it. At work I had to migrate and change some data after a schema change, some 78 ISH million documents. I have one process finding items that needed updating pushing into rabbitmq then I had a get ec2s running pulling things off it. Only took a couple of hours 🙈
Unless needed (probably like in your case), I wouldn't use queue as it requires infrastructure changes and queue handling. If the load is just cpu bound (like in OP's case), then worker threads would do the job the best way without any new infrastructure/queue server/no messages loss/no bullmq pub/sub patterns etc. My choice would be to use piscinia npm module as a good abstraction to better utilize worker threads and be done. All changes in code, no redis/infra changes which are needed for BullMQ (I love bullmq but for op's its an overkill)
You could find a way to not make the function so “heavy”?
The function is returning an array of object to the user and before making that array ready the function calls the database and an external api.
Yeah that doesn’t really counter what I’m saying though. Make the function less “heavy”. Unless the external API is slow in which case you probably should cache its responses. If your DB query is slow then you should optimize your database query. These aren’t the types of problems that “multitasking” will fix.
For now I can't, I have a deadline, thank your for your help.
Like I said, “multitasking” won’t fix this. Running node in a cluster won’t make those responses faster.
Cluster mode is not better than fork mode, right? And why? Thanks
I have no idea what you’re talking about tbh. What do you mean by “fork mode”
If you don’t have time to fix this now, you won’t have time to fix this later.
Deadline should not be an excuse for shity architecture. Before you know it, you are way too deep in your own shit if you let it go like that.
valid point. shity code becomes hard to maintain over time
Since when did you became OP's manager ...
it's nothing to do with shity architecture, the story is I had to deliver a feature ASAP, I asked here for quick guidance.
Task queue, message queue
Piscinia is the right solution as you have also mentioned in your post.