Re: Work queue questions

[anish singh]

On Sat, Sep 22, 2012 at 10:57 AM, Daniel Taylor <Daniel.Taylor@xxxxxxx> wrote:
>
>
>> -----Original Message-----
>> From: linux-kernel-owner@xxxxxxxxxxxxxxx
>> [mailto:linux-kernel-owner@xxxxxxxxxxxxxxx] On Behalf Of anish singh
>> Sent: Friday, September 21, 2012 9:25 PM
>> To: Deepawali Verma
>> Cc: Tejun Heo; linux-kernel@xxxxxxxxxxxxxxx
>> Subject: Re: Work queue questions
>>
>> On Sat, Sep 22, 2012 at 1:05 AM, Deepawali Verma
>> <dverma249@xxxxxxxxx> wrote:
>> > Hi Tajun,
>> >
>> > These three tasks are writing the three chunks of data in
>> parallel. I
>> > am not getting improvement here otherwise what is difference between
>> > writing these chunks one by one in single thread instead of
>> trying to
>> > write the data by scheduling the work on three different workqueues
>> > means 3 worker threads?
>> You should have carefully read "If none of them blocks, there
>> isn't much point in throwing more threads at them.  What are those
>> thread doing?" what Tejun said.
>>
>> I think what he means is that concurrency is the concept of
>> keeping the
>> system busy.
>> If you see the below logs:
>> kworker/u:1-21    [000]   110.964895: task_event:
>> MYTASKJOB2381 XStarted
>> kworker/u:1-21    [000]   110.964909: task_event:
>> MYTASKJOB2381 Xstopped
>> Here your first worker thread blocked.
>>
>> So the system will try to get other workqueue started which is:
>> kworker/u:1-21    [000]   110.965137: task_event:
>> MYTASKJOB2382 XStarted
>> kworker/u:1-21    [000]   110.965154: task_event:
>> MYTASKJOB2382 Xstopped
>> Here again your second worker thread blocked.
>>
>> So on so forth.
>> Anyway how can you write chunks of data in parallel when
>> already some worker
>> thread is writing i.e. the system is busy.
>> Analogy: Suppose you are ambidextrous and you are eating.Can
>> you eat with
>> both of your hands at a time?So worker thread are like your
>> hands and keeping
>> you fed all the time is the concept of concurrency.
>>
>> I am not an expert on this but from Tejun's reply I could
>> make out this.
>> Please correct me If I have wrongly understood the concept
>> based on this mail
>
> I don't know how many cores are in the CPU Deepawali's using, but if I have four,
Assuming single core,Is my explanation correct about concurrency?
> for example, I could do something simplistic like copy pages A-G with one, pages
> H-O with another, and pages Q-Z with a third.  There are memory and cache bottlenecks
> (like the mouth, in your example), but all three copies could be running concurrently.
>
> Copying, of course, is a silly, trivial example, and I hope there's a better reason
> than that for the concurrency, but, if, for example, your needed to byte-swap, XOR,
> or checksum, as core functionality of an embedded system, and the processing units were
> available to do these things in parallel, then interleaving those operations with memory
> accesses could provide higher throughput.
>
> I think what he's asking is why there's no apparent concurrency, presuming that NONE
> of his threads has a real reason to block.  With examining his code, I cannot tell,
> but it looks like, from the messages, that the kernel did not attempt concurrency.
> Perhaps he needs to pass additional state to the scheduler?
>
>
>> chain.
>> >
>> > Regards,
>> > Deepa
>> >
>> > On Fri, Sep 21, 2012 at 8:27 PM, Tejun Heo <tj@xxxxxxxxxx> wrote:
>> >> On Fri, Sep 21, 2012 at 08:26:01PM +0100, Deepawali Verma wrote:
>> >>>      kworker/u:1-21    [000]   110.964895: task_event:
>> MYTASKJOB2381 XStarted
>> >>>      kworker/u:1-21    [000]   110.964909: task_event:
>> MYTASKJOB2381 Xstopped
>> >>>      kworker/u:1-21    [000]   110.965137: task_event:
>> MYTASKJOB2382 XStarted
>> >>>      kworker/u:1-21    [000]   110.965154: task_event:
>> MYTASKJOB2382 Xstopped
>> >>>      kworker/u:5-3724  [000]   110.965311: task_event:
>> MYTASKJOB2383 XStarted
>> >>>      kworker/u:5-3724  [000]   110.965325: task_event:
>> MYTASKJOB2383 Xstopped
>> >>>
>> >>> I have this one big task to whom I divided into small sub
>> tasks. These
>> >>> are numbered 2381, 2382 and 2383, what was I expecting
>> that task 2381,
>> >>> 2382, 2383 run in parallel. I have put start and stop
>> markers here so
>> >>> that I can see how this concurrency managed work queue is
>> distributing
>> >>> the load.
>> >>>
>> >>> I found that task no 2381 is started first and exited
>> before starting
>> >>> task 2382 and so on. What I expected that it should start
>> the three
>> >>> sub tasks in parallel, not one by one.
>> >>>
>> >>> Where is concurrency here?
>> >>
>> >> If none of them blocks, there isn't much point in throwing more
>> >> threads at them.  What are those thread doing?
>> >>
>> >> Thanks.
>> >>
>> >> --
>> >> tejun
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe
>> linux-kernel" in
>> > the body of a message to majordomo@xxxxxxxxxxxxxxx
>> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> > Please read the FAQ at  http://www.tux.org/lkml/
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> linux-kernel" in
>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/