Iterating Lists in Small Chunks And Performance

A couple of months ago I created a post about it and in the conclusion I stated that it would probably be faster to act on the whole list by chunks than it is using the big query… It’s not.

I have done the test with a list of 100K items so you don’t have to and, depending on the size of the chunks it can take slightly longer or a lot longer.

If you are using a very small RowLimit (say 5) the number of times you have to perform the query in order to retrieve the whole list will make the process take twice as long or even longer. If you are using a bigger number (like 1000) the performance will be similar to the one achieved retrieving the whole list and processing it in just one query.

In summary, if you are looking for a small number of values and they are likely to be found in the first iterations of the paginated query then the performance boost will be obvious, in some cases I have noticed an average improvement of up to a 95%, but bear in mind that if you are using very small chunks and you are likely to go through the whole list to get the values back your query will be slower, up to an 400% slower or even more.

The Ultimate Blog Challenge

April fools!

Actually it is not a joke. I want to post more and this is a great excuse.

Going from my usual one post per month to the required one post per day sounds a bit daunting a lot, actually, but it also looks like a very interesting experiment and a great challenge.

Most of the times I don’t get to post because I think a bit of code is not interesting enough to be published, discussed or even commented and I am probably right, but in the other hand sometimes there are posts that I find not so good and they end up reporting the most visits to the blog.

The Ultimate Blog Challenge will be the perfect alibi to start posting more and to see if it improves the blog or not.

Let the posting begin.