CPU:s per task

[Magnus Nilsson <ma...@gmail.com>]

Hello all,

TLDR; Can the number of cores used by a task vary or is it always one core
per task? Is there a UI, metrics or logs I can check to see the number of
cores used by the task?

I have an ETL-pipeline where I do some transformations. In one of the
stages which ought to be quite CPU-heavy there's only a single task running
for a few minutes. I'm trying to determine if this means only one cpu core
is in use or if a single task could use many cores under the cover?

When I read data from an Event Hub the stage includes as many tasks as
there are partitions in the Event Hub up to the maximum nr of cores
available in the cluster. Clearly those tasks use one core each and are
limited in parallellism by the cluster size.

Regards,

Magnus