The mapping functions are handled in the same function? #761
|
Lithops with AWS Lambda backend handles user map function requests by using multiprocess lib to enable parallel processing. |
Replies: 1 comment
|
Independently of the memory size, AWS Lambda provides the parallelism by starting a separate container for each activation of a For memory sizes > 1769MB, where you get more than one cpu per container, you can use the |
Independently of the memory size, AWS Lambda provides the parallelism by starting a separate container for each activation of a
map(). By default, each started container processes 1 element of theiterable. For example, if theiterableof amap(func, itrerable)contains 1000 elements, Lithops + AWS lambda will start 1000 separate containers, each one processing one element of theiterable.For memory sizes > 1769MB, where you get more than one cpu per container, you can use the
worker_processesconfig key to send multiple activation to the same worker, this way you will leverage all the CPUs in a container.