PLINQ Tour Wrap-up and a Question Answered

May 18, 2009

Last week I took a tour of Michigan giving my “Doing More With Less – PLINQ” talk to three user groups (Grand Rapids, Ann Arbor and Flint). It was a great time! All three groups were fun to visit. It’s always interesting to see how other groups are run and how their meetings go.

While I was traveling around I stayed one evening with Dave Giard, a friend of mine I worked with several years ago. He has a series of interviews he’s been doing with various people in the community, and he recorded a short interview with me regarding PLINQ (Dave: I told you the camera would accentuate my bald spot! I apologize to anyone who may be temporarily blinded). Chris Woodruff was also kind enough to put me up for an evening as well.

At Ann Arbor one of the attendees had asked an interesting question that I did not know the answer to. His question was, “Does thread context carry over to the new threads that are spun up under the hood by PLINQ? Specifically, what about HTTPContext?”

For a quick run down, AsParallel is a new extension method provided by PLINQ for IEnumerable. This method takes the IEnumerable and wraps it in an IParallelEnumerable. PLINQ then supplies parallel implementations of all the standard LINQ methods (Select, Where, Min, Max, Sum, etc.) as extension methods off of IParallelEnumerable. This makes it really easy to add parallelism to your query simply by adding the AsParallel method to your query. Check out more about PLINQ on MSDN (Really old article: http://msdn.microsoft.com/en-us/magazine/cc163329.aspx).

Just like a standard LINQ query, when you cause the query to actually execute, such as iterating the result, a PLINQ query’s calling thread starts the execution of the query. However, the implementation of the query execution calls into the new Task Parallel Library (TPL - new in .NET 4.0) which knows how to break the work up across multiple processors. The TPL spins up the extra threads and assigns them to processors, etc. The question the attendee asked was, does any context on the original thread the query was started on carry over to the spawned threads. He had specifically given the example of HTTPContext.

To answer this I tried two things:

NOTE: The following tests were attempted on a copy of the PLINQ CTP from last June. My copy of VS 2010 and .NET 4.0 isn’t finished burning to a DVD yet. It was just released today to people with MSDN Subscriptions. I’ll be building a Virtual Machine sometime in the next week or so. I’ll try to remember to test this again with that to see if the behavior has changed.

First, I created a web application, referenced System.Threading and added a simple PLINQ query. In the filtering Where clause of the query I attempted to get at the HTTPContext. What happened made complete sense to me. The short answer is the context is NOT copied to the other threads that PLINQ spins up under the hood; however, the calling thread is one of the threads involved in the execution of the query so any part of the query being executed on that thread could find the HTTP Context. None of the other threads could find the HTTPContext though.

Second, I created a class in a Console application, referenced System.Threading and again added a simple PLINQ query. This time I added something in the Thread CallContext prior to the execution of the query and attempted to get it back out for while in the Where clause of the query. Again, the other threads spun up by PLINQ did not have the original CallContext data copied to it, so the calling thread could see the data, but none of the other threads.

So, the answer is that threads spun up by PLINQ do not have the callcontext data or HttpContext copied over to them. You will not be able to count on access to that. I will also point out that I do not believe that the use of PLINQ queries is a good choice within a web application. If you are concerned about the performance of your LINQ queries so much that you want to parallelize them they are likely running longer than a user of a web site needs to be waiting even with PLINQ. Also, PLINQ is going to try to use as many processors as you allow it to get the query done as efficiently as possible. This is going to affect your scalability and performance of your website. I’d be focusing the use of PLINQ to asynchronous tasks executed on an application server and not my web server.