Social Icons

twitter google plus linkedin rss feed

Pages

19.9.13

Get the fields available in an SPListItem

When you retrieve the items from a CAML query with the ViewFields parameters set there’s no way (or at least i don’t know it) to find out which fields you have available and populated with data.

The usual way of getting the list of fields from the content type doesn’t work because the content type is null in that kind of items…

The answer, my friend, is sitting in the Xml.

static List<string> ExcludedFields = new List<string> { "z", "ows_ServerRedirected", "ows_FileRef", "ows_PermMask", "ows_FSObjType", "ows__Level", "ows__ModerationStatus" };
/// <summary>
/// Gets a dictionary with the available fields and its values.
/// </summary>
public static Dictionary<string, string> getAvailableFields(this SPListItem item)
{
    XElement row = XElement.Parse(item.Xml);
    Dictionary<string, string> Fields = new Dictionary<string, string>();

    foreach (XAttribute field in row.Attributes())
    {
        if (!ExcludedFields.Contains(field.Name.LocalName))
            Fields.Add(field.Name.LocalName.Substring(4), field.Value);
    }

    return Fields;
}

Using this I have been able to reduce the traffic on a web service method in a 65%.

23.8.13

Scopes in a CAML Query

I have been working for quite a while now with CAML queries and the scope is always something very important to bear in mind. How many times my queries returned nothing when I was sure they should bring back something…

Basically we have two modifiers Recursive and All and nothing, could we call nothing a modifier? All will bring back folders and files. Recursive will repeat the query in all the folders under the one we are working with.

If you don’t set the scope to All it will only bring files. If you don’t set it to recursive it will only retrieve items from the folder you are at. There are not that many variants so let’s make an example of each.

Let’s imagine we have a SharePoint folder like this one and we want to query it:

CamlScopeTreeSample

I am not good at paint, I know but what I want to show here is a tree where we have a Root folder (the root of the queries) and two sub-folders with files. For each scope possible I’ll highlight what you can expect to retrieve.

Just before we start, allow me to remind you that the scope is set in the property ViewAttributes of the SPQuery item.

To add a bit more of clarity I have also painted the levels:

CamlScopeTreeSampleLevels

The green line marks what’s inside the root folder, the blue line marks the contents of SubFolder1 and the red line SubFolder2.

ViewAttributes left by default:

CamlScopeByDefault
This is just the files under the root folder.

ViewAttributes = "Scope='Recursive'"

CamlScopeRecursive

This means all the files in all the folders.

ViewAttributes = "Scope='All'"

CamlScopeAll

This scope will bring folders and files under root.

ViewAttributes = "Scope='RecursiveAll'"

CamlScopeRecursiveAll

And finally with RecursiveAll you can bring back everything under the root entity.

Good luck with your queries.

19.8.13

SharePoint Client Object Model Is Great

This is one of those things that you know they are there but never use because you already know a different way.

Just four years after I started working with SharePoint 2010 I thought “Why not giving the Client Object Model a go?” actually I had a requirement from a client. and I must say I am VERY impressed. The simplicity, the speed and the predictability is pretty good for the standards we are used to.

Even though the COM is very good I have created a series of methods, wrappers and extensions to help me deal with my most common functions. Some of them translated from the methods I use in the SharePoint Object Model and some of them new. Let’s begin.

It is really easy to connect using Windows authentication or Forms Based Authentication. Easier than anything else I have seen to date.

public static ClientContext GenerateClientContextWinAuth(string URL)
{
    return new ClientContext(URL);
}

public static ClientContext GenerateClientContextFBAAuth(string URL, string userName, string password)
{
    ClientContext ctx = new ClientContext(URL);
    ctx.AuthenticationMode = ClientAuthenticationMode.FormsAuthentication;
    ctx.FormsAuthenticationLoginInfo = new FormsAuthenticationLoginInfo(userName, password);

    return ctx;
}

Windows auth 1 line of code FBA 3. Nice.

Do you remember how it was reading and writing files to SharePoint? Look how easy it is using the COM.

public static string ReadFile(ClientContext Context, string ListName, string FileName)
{
    List StorageList = Context.Web.Lists.GetByTitle(ListName);
    string SharePointFilePath = GetFilePathInSharePoint(Context, StorageList, FileName);

    FileInformation fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(Context, SharePointFilePath);

    using (fileInfo.Stream)
    {
        using (StreamReader sr = new StreamReader(fileInfo.Stream))
        {
            return sr.ReadToEnd();
        }
    }
}

public static void WriteFile(ClientContext Context, string LocalFilePath, string ListName, string SPFileName)
{
    List StorageList = Context.Web.Lists.GetByTitle(ListName);

    string SharePointFilePath = GetFilePathInSharePoint(Context, StorageList, SPFileName);

    using (FileStream fs = new FileStream(LocalFilePath, FileMode.Open))
        Microsoft.SharePoint.Client.File.SaveBinaryDirect(Context, SharePointFilePath, fs, true);
}

private static string GetFilePathInSharePoint(ClientContext ctx, List StorageList, string FileName)
{
    if (StorageList.RootFolder.ServerObjectIsNull != false)
    {
        ctx.Load(StorageList.RootFolder);
        ctx.ExecuteQuery();
    }

    string ListRootFolderURLDocuments = StorageList.RootFolder.ServerRelativeUrl;
    return Path.Combine(ListRootFolderURLDocuments, FileName);
}

Other than the small function to find the url of the item in the document list using folders it's as simple as it can be. I have also created a couple of methods for making easier to work with the StratexFramework.

If you are creating some program and want to use some of the code feel free. If you are not working with StratexPoint feel free to modify it to suit your environment.

Here's a method to bring back the root entity of the framework:
public static ListItem GetRootEntity(this List ComList)
{
    CamlQuery camlQuery = new CamlQuery();
    camlQuery.ViewXml = string.Format(@"<View>
                                            <Query>
                                                <Where>
                                                    <Eq>
                                                        <FieldRef Name='ContentType'/>
                                                        <Value Type='Choice'>Entity</Value>
                                                    </Eq>
                                                </Where>
                                                <RowLimit>1</RowLimit>
                                            </Query>
                                        </View>");

    ListItemCollection items = ComList.GetItemsExecuted(camlQuery);

    if (items.Count == 1)
        return items[0];
    else
        return null;
}

Nice and easy.

The query is a bit different to the usual we do in the SPQuery, but still very similar.

Then I have created a method that will help when you want to run a query under a folder regardless of the rest of the framework. First I will paste the helper query and then I will paste the samples on how to use it:

public static CamlQuery CreatePositionedQuery(this ListItem StartEntity)
{
    CamlQuery camlQuery = new CamlQuery();
    camlQuery.ViewXml = string.Format(@"<View Scope='RecursiveAll' >
                                            <Query>
                                                <Where>
                                                    <And>
                                                        <Eq><FieldRef Name='FileDirRef' /><Value Type='Text'>{0}</Value></Eq>
                                                        {1}
                                                    </And>
                                                </Where>
                                            </Query>
                                        </View>", GetChildrenFolder(StartEntity), "{0}"); //This is the folder url and the placeholder for the real query.


    return camlQuery;

}

private static string GetChildrenFolder(ListItem startItem)
{
    startItem.InitializeIfNeeded("FileDirRef");
    startItem.InitializeIfNeeded("FileLeafRef");

    return startItem["FileDirRef"] + "/" + startItem["FileLeafRef"];
}

With this we basically are saying SharePoint to execute the query under the folder of the item we are passing as a parameter. And we can use this code easily to position our queries in the folder tree.

First one function to get all the child entities under a given entity, and then another function to bring an item with a given title inside a given folder:

public static ListItemCollection GetChildEntities(ListItem StartEntity)
{
    List ComList = StartEntity.ParentList;

    CamlQuery camlQuery = CreatePositionedQuery(StartEntity);
    camlQuery.ViewXml = string.Format(camlQuery.ViewXml,
                                    @"<And>
                                        <Eq><FieldRef Name='State' /><Value Type='Choice'>Live</Value></Eq>
                                        <Eq><FieldRef Name='ContentType' /><Value Type='Choice'>Entity</Value></Eq>
                                        </And>");

    return ComList.GetItemsExecuted(camlQuery);
}

public static ListItem GetChildItem(this ListItem StartEntity, string Title)
{
    List ComList = StartEntity.ParentList;

    CamlQuery camlQuery = CreatePositionedQuery(StartEntity);
    camlQuery.ViewXml = string.Format(camlQuery.ViewXml,
                                    string.Format("<Eq><FieldRef Name='Title' /><Value Type='Text'>{0}</Value></Eq>", Title));

    ListItemCollection result = ComList.GetItemsExecuted(camlQuery);

    if (result.Count > 0)
        return result[0];
    else
        return null;
}
Creating a new item is also easy. The hardest thing to do is deciding if it's a folder or a leaf:
public static ListItem CreateItemUnder(this ListItem ParentItem, string Title, string ContentType, bool isLeaf)
{
    ListItemCreationInformation itemCreateInfo = new ListItemCreationInformation();
    itemCreateInfo.FolderUrl = GetChildrenFolder(ParentItem);
    itemCreateInfo.LeafName = Title;

    if (isLeaf)
        itemCreateInfo.UnderlyingObjectType = FileSystemObjectType.File;
    else
        itemCreateInfo.UnderlyingObjectType = FileSystemObjectType.Folder;

    ListItem NewItem = ParentItem.ParentList.AddItem(itemCreateInfo);

    if (ContentType != null)
        NewItem["ContentTypeId"] = GetContentType(ParentItem.ParentList, ContentType).Id;

    return NewItem;
}

A small shortcut to execute queries:


public static ListItemCollection GetItemsExecuted(this List listToQuery, CamlQuery query)
{
    ListItemCollection childItems = listToQuery.GetItems(query);

    listToQuery.Context.Load(listToQuery);
    listToQuery.Context.Load(childItems);
    listToQuery.Context.ExecuteQuery();

    return childItems;
}

In some cases the field you want to read didn't come in the execution maybe because you were a bit too restrictive with the viewfields clause... You can try with this trick:


public static void InitializeIfNeeded(this ListItem item, string InternalName)
{
    if (!item.FieldValues.ContainsKey(InternalName))
    {
        item.Context.Load(item);
        item.Context.ExecuteQuery();
    }
}

Are you trying to get a content type from a List? Easy.


public static ContentType GetContentType(List list, string cTypeName)
{
    list.Context.Load(list.ContentTypes);
    list.Context.ExecuteQuery();

    foreach (ContentType ctype in list.ContentTypes)
    {
        if (ctype.Name == cTypeName) return ctype;
    }

    return null;
}

Are you trying to save a SPFieldUserValue in a SPFieldUser using the client object model or read it back? Complex, I would say unnecessarily complex, but I have one last trick:

public static string UserToString(object fieldUserValue)
{
    if (fieldUserValue == null) return string.Empty;

    FieldUserValue user = fieldUserValue as FieldUserValue;

    return string.Format("{0};{1}", user.LookupId, user.LookupValue);
}

public static FieldUserValue StringToUser(object fieldUserValueString)
{
    FieldUserValue user = new FieldUserValue();

    if (string.IsNullOrEmpty(fieldUserValueString.ToStringSafe())) return user;

    string[] tokens = fieldUserValueString.ToStringSafe().Split(';');

    user.LookupId = tokens[0].ToNullableInt() ?? 0;

    return user;
}

The client object model has been really powerful and really easy to work with for me so far. I really encourage you to give it a try if you haven't already.

9.8.13

The process wlms.exe has initiated the power off of computer

I was getting random shutdowns of one of my virtual machines hosted in azure... I thought it was something related to azure but no.

I had a look to the shutdown log and found out this:

Log Name:      System
Source:        USER32
Date:          08/08/2013 17:39:04
Event ID:      1074
Task Category: None
Level:         Information
Keywords:      Classic
User:          SYSTEM
Computer:      COMPUTERNAME.local
Description:
The process wlms.exe has initiated the power off of computer COMPUTERNAME on behalf of user NT AUTHORITY\SYSTEM for the following reason: Other (Unplanned)
 Reason Code: 0x0
 Shutdown Type: power off
 Comment: 
Event Xml:
<event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <system>
    <provider name="USER32">
    <eventid qualifiers="32768">1074</eventid>
    <level>4</level>
    <task>0</task>
    <keywords>0x80000000000000</keywords>
    <timecreated systemtime="2013-08-08T16:39:04.000000000Z">
    <eventrecordid>147067</eventrecordid>
    <channel>System</channel>
    <computer>COMPUTERNAME.local</computer>
    <security userid="S-1-5-18">
  </security></timecreated></provider></system>
  <eventdata>
    <data>wlms.exe</data>
    <data>COMPUTERNAME</data>
    <data>Other (Unplanned)</data>
    <data>0x0</data>
    <data>power off</data>
    <data>
    </data>
    <data>NT AUTHORITY\SYSTEM</data>
  </eventdata>
</event>

A fast googling told me that in order to stop the VM from shutting down I needed to activate it :)

30.4.13

The Ultimate Blog Challenge Has Finished

Finally!

It has been really hard to find the time to write one post a day. I suppose it would also depend on the month and this one for me has been hectic. All of them are...

I thought it was going to be harder to find topics to write about, the length and the amount of code in the posts have also decreased a lot during this month to be true.

The idea of the challenge was that if you post often your blog will get more visits. I can tell you that's not true.

I suppose, if you have a blog where people use to come and your posts are interesting you'll get them back more often if you post more, but in my case I only get a couple of visits from searches in Google Bing refuses to index the blog for some obscure reason and posting more doesn't really make a difference.

Writing about topics that are very actual helps. But if you are busy working in your projects a 125% of your time you don't get to do that many interesting things... I would really like to change that.

In the end the results of the Ultimate Blog Challenge in the visits of the blog look like this:


The first two weeks I had a lot of visits, I suppose from my fellow bloggers in the challenge, after that the amount of attention the blog draws has decreased to the usual levels. You see that peak? Yes, that one was me changing the template.

I have now a couple more of followers in twitter and not a single more subscription than I had last month.

Anyway I am happy to have been able to write the same amount I usually write in two years in just one month.

I used to think a lot about what I was going to write in the blog and what wasn't interesting enough and now it has become more natural. I don't think I am going to write one post a day anymore or not until I get paid for it somehow but I will definitely write more often.

I'm sorry for the twits and the messages in LinkedIn etc. In the other hand you don't hear from me that much in the social networks usually.

Back to normal now.

How Much Are you Bringing Back In Your CAML Queries?

I don't know you but I am bringing back too much.

This is one of those things you don't notice until it's too late. I thought just adding very strict filters to the CAML Queries to bring back just the items you needed was enough but there's one more thing you can do.

You can restrict the fields you are retrieving... and you should.

The SQL Queries that SharePoint generates for retrieving the items when you let the ViewFields parameter of the CAML empty almost doubles the complexity of the one it generates when you specify the fields you want to query, or use a view.

Two simple lines like these:

query.ViewFields = "<FieldRef Name=\"Value\" />";
query.ViewFieldsOnly = true;
Can make your query better.

Who could resist to do it right when it's this simple?

Disposing every SPWeb you use... Is it necessary?

I am becoming a bit paranoid about disposing lately. In example.

Until now I would have used things like Web.ParentWeb and then forgotten about them. As I was not "Opening" them manually I was not disposing them either.

Now I usually do things like this:
using (SPWeb web = anotherWeb.ParentWeb)
{
      //Do Whatever you need here with the parent web
 }
I haven't had the time to check whether this is reducing the memory leaks or just making my application more unstable (if I close the web while I am using it somewhere else it could cause an exception it the other part of the code)

Do you think when you dispose a web it automatically disposes every parent web automatically?
It could be...



I need to test this and once I do it I'll post about it.

SharePoint Makes you Humble

I have learn this well enough during the last seven years. SharePoint is a hard beast to tame, so hard I have never met anyone working in this platform who claims to know everything about it actually if anyone had I would regard it as a proof of complete ignorance.

In a world where trying beyond extenuation is common and the deadlines are pressing you find individuals whose levels of tolerance to frustration are sky high.

You definitely need a sense of humor to be a SharePoint developer. I would even say a taste for black humor.

I remember smiling back in the 2007 after selling a SharePoint based solution when the client asked my for the instructions manual of SharePoint.

There are so many things working together in so many different places that it's really easy for some small and hidden piece to make the whole farm lose balance. Do you remember the books of the D'nis? something like that.

I am writing this because I am four posts short for completing the Ultimate Blog Challenge and it has been impossible for me to write on the last four days... but I won't give up. Not when I am so close.

As a SharePoint Developer I can overcome any issue or workaround it

25.4.13

customErrors mode Doesn't Work in my Server

I have been receiving the usual:

Server Error in '/' Application.

Runtime Error

Description: An application error occurred on the server. The current custom error settings for this application prevent the details of the application error from being viewed.

Details: To enable the details of this specific error message to be viewable on the local server machine, please create a <customErrors> tag within a "web.config" configuration file located in the root directory of the current web application. This <customErrors> tag should then have its "mode" attribute set to "RemoteOnly". To enable the details to be viewable on remote machines, please set "mode" to "Off".


<!-- Web.Config Configuration File -->

<configuration>
    <system.web>
        <customErrors mode="RemoteOnly"/>
    </system.web>
</configuration>


Good, I now how to solve this one, it's easy, you just have to go to C:\inetpub\wwwroot\wss\VirtualDirectories\80 and change the customErrors mode in the web.confing

Well it didn't work.

After trying a thousand things I found out that there's a more obscure web.config you need to change in SharePoint 2010, it's at c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\LAYOUTS\

After changing the set there everything worked as expected.

It's funny it only happens in some servers I suppose it's one of the reasons why SharePoint is so funny.


Virtual Box Fails after installing WP8 SDK

After installing the WP8 SDK I round that I couldn't open my VBox VMs any more. I was getting the error:

VT-x is not available. (VERR_VMX_NO_VMX)
 Result Code: E_FAIL (0x80004005) Component: Console Interface: IConsole {db7ab4ca-2a3f-4183-9243-c1208da92392}
It was completely unexpected because i had the WP7 SDK installed and everything was working fine.

Looks like the new SDK needs Hyper-V to run the emulators and having the Hyper-V services initiated won't let Virtual Box use the virtualization hardware.

The solution is a quick and dirty:
bcdedit /set hypervisorlaunchtype off
And a restart to be able to work with your VBox again.

If you want to start your Hyper-V services again you just have to run
bcdedit /set hypervisorlaunchtype auto
And restart.

After saving what I was doing I have converted the vdi to vhdx and started using Hyper-V for my development virtual machine. Now I feel less heretic.

23.4.13

Andrew Smart Demoing StratexPoint

This video was recorded at the Executive Breakfast Briefing too but I decided to put it in a different post because of the content.

In this one, apart from talking about some briefing stuff, +Andrew Smart shows the basics of StratexPoint and the Risk Based Performance Management methodology.

For those of you curious about what I do, here you have a taste.

 

Great videos James.

A word from our clients

In the first executive briefing of 2013, StratexSystems invites two of it's clients to share their experiences at different stages of the Enterprise Risk Management journey.

The first keynote presentation was a case study from one of the UK's newest and fastest growing banks (Aldermore Plc) presented by their Head of Operational Risk, Kevin Pearce. This case study will tell the story of an organisation relatively early in their ERM Journey. It will include insights into establishing the ERM agenda and making it relevant in a fast growing organisation focused on delivering significant client and shareholder value.

 

The second keynote presentation will be a case study from the UK's largest financial services outsource provider (HML) and was presented by the Head of Enterprise Risk Management, Gillian Weatherill. This case study will tell the story of, and share the insights from a 5+ year ERM journey and how they achieved a 60% reduction in operational losses and a 23% reduction in capital provision within 18 months.

 

You can find more information on our Executive Briefing events, Webinars and Software Solutions at http://www.stratexsystems.com

Thank you very much Gillian and Kevin!

22.4.13

Editing SPListItems from SPWeb.GetSiteData

In my solution I have an unknown number of subsites that contain an unknown number of SPListItems that need to be updated. Looks like the perfect way of testing GetSiteData.

The documentation is useful and you'll be able to test the queries without problems really? yes, the documentation is ok but this method brings back a DataTable and what I needed was to update the SPListItems.

Well if you look at the DataTable you'll see that you have all the fields you need there to retrieve the SPListItem easily.

I have created an extension method, well two really... I love extension methods.
public static SPListItem GetListItemFromSiteData(this DataRow ItemRow, SPSite ParentSite)
{
    using (SPWeb Web = ItemRow.GetWebSiteData(ParentSite))
    {
        return Web.Lists[new Guid(ItemRow["ListId"].ToString())].GetItemById(Convert.ToInt32(ItemRow["ID"]));
    }
}

public static SPWeb GetWebSiteData(this DataRow ItemRow, SPSite ParentSite)
{
    return ParentSite.OpenWeb(new Guid(ItemRow["WebId"].ToString()));
}

Using these two you will be able to iterate through the collection of rows of the DataTable, select which elements need to be updated and update them.

I haven't tested how fast is this compared with bringing the items with a CAML query. Not having to create the SPListItemCollection could make this method faster and more convenient for some things... I'll give it a go.

21.4.13

Battery and Shortcut Tiles for WP8

As you know I had my WP8 device some weeks ago. One of the things that annoy me of Windows Phone is  having to go to Settings>Whatever>What you want and I do it a couple of times a day every day. That's being annoyed twice every day and that makes a total of 730 times annoyed every year.

I thought, Great! I am going to make an app with shortcut tiles and I am going to have finally an useful app in the store, one that makes sense.

While I was downloading the SDK I thought it could be good to have an app that tells you about the battery level in a tile. "Someone should have done this, I'll go to the marketplace and grab one" And I was right.

In the marketplace I found Battery, a great app that shows a chart with the battery life, has statistics about everything, tells you the battery level beautifully and even the estimated amount of time until the next charge. Fantastic app! this guy has thought of everything! But, what's that? Oh not again Yes. Shortcut tiles.

The app also provides a set of shortcut tiles. And they work beautifully. WiFi, Location, Bluetooth, Flight Mode and Mobile Network. And my plan was to create shortcut tiles for WiFi and Flight Mode only...

Obviously I am not going to create the app for the shortcuts, there are plenty in the market. But if you read this post and have a WP8 give this app a go. It helps you work faster with the phone and I think the tiles make it a bit geekier. Now I feel happy of using those shortcuts and that makes me happy 730 times a year.


Will I come up with a good idea for an app?.

Go Electronic at More Levels

We live connected to the phone, the computer, the tablet, the TV...

We work creating bits of information, but not bits as in pieces, I mean bits like the ones when you put eight of them together make a byte.

Maybe we should change the direction and start doing more things analogically but it's also cool to go with the flow and find yourself doing more things electronically... like smoking.

I know this is completely out of topic in my blog, but I am three posts late in the Ultimate Blog Challenge and I can't think of anything else light enough to be posted on a Sunday.

I have tried the SkyCigs and they are... interesting. It's not the same as smoking a real one but only because of the taste. The smoke is nice, the cigarettes are nice, they have a kick.

If you are bored enough give them a go. In my office two of the guys are already smoking just e-cigs that's a 100% of the people who tried... Not counting me, I went back to the analogical ones because I need to do anything else than eating analogically until today.

Today I have decided I am quitting. Analogicals, Electronicals and everything.

I am one step closer to becoming a monk, in fact I consider my self closer to the monk ideal than most of the real monks except for the praying... well if I can't make something work and we are close to the deadline I am a monk with the praying and everything.

University courses for free? Coursera

Education for free. It doesn't get much better that this.

I heard about this courses just a month ago and I have joined two. The fist one starts next Monday so I can't say yet how good they are or if they are worth it, but they really look as if they are.

I have joined a course about artificial intelligence and another one about programming.

Artificial intelligence is a very interesting field in which I have no experience at all. Wouldn't it be cool to be able to add a bit of intelligence to whatever solution you are working with at the moment? I think it could add a lot of value to some things I do and if it doesn't I still get my curiosity satisfied.

And the course about programming is also interesting, if things have changed in the last 10 years I want to know and if they haven't I want to remember how it was like :)

Before I join any more courses I'll try to finish these two, but I already have my eye set on a pattern oriented architectures for concurrent software course...

Why don't you join me at https://www.coursera.org?

19.4.13

We need a Developer's Pride Day

Every time I am looking for a senior developer is the same.

Is as if you have been a developer for four years you can't be a developer any more. Now you have to be a team manager, a lead developer, or a presales consultant and anything less than that is a shame for you and your career. Why a developer is less than a team manager? Can anyone explain that to me?

How come anyone with four years of experience can claim they are so experienced that they don't want to code any more?

I remember once I was hiring for a developer role in Madrid and a guy looking for his first job after the university told me he wouldn't code because he was an engineer. I was puzzled... I went to the university to learn how to code properly...

To me is as if a footballers didn't care about playing football because he just wanted to be a coach.

How do you think you are going to be able to tell people what to do and how to do it if you don't know yourself because you never cared?

Managers are necessary but it looks to me like people think a developer is the byproduct of a manager.

Let me tell you something, a developer is not a guy who wasn't good at anything else and he had to settle for being ignored in a corner of an remote office. Being a developer takes more than most people think.

In my case I have been a developer for seven years, that's professionally only. I have been developing for the sheer pleasure of doing it since I was 9 and I got my MSX HB-20P and after all this time I feel like I have just scratched the surface of what development is.

I have seen good developers working and they really make a difference. The amount of knowledge they amass is impressive as well as clarity on their thoughts and the elegance of the code they deliver.

They are artists, and a very impressive kind of artists. They can deliver every day, they manage the pressure they overcome an infinite number of issues learning from them and improving their code and their environments to make sure they won't have those issues again (nor any of their team mates)

Great developers are generous with their knowledge, ask them about what they do and you'll be probably overwhelmed about the amount of details and insights they'll provide. They don't hide what they know from anyone, they love sharing. There's so much to learn and so little time.

You don't get that kind of expertise in a couple of years, it takes a whole life of effort to become a real expert in this field. And if we don't change our mindset about what a coder is the amount of good developers we are going to have in the future is going to be ridiculous.

If you are a developer take pride in what you do. An engineer with an infinite number of pieces to choose from. A general with an infinite number of soldiers doing exactly what you tell them to do.You are the maker.

We, developers, need to improve the way we think about ourselves, we need to improve the way companies look at us.

We need a Developer's Pride Day.

If you love learning new tricks, if you love it when a class does exactly what you want and how beautiful is the way you made it do it. If you love to improve the code in every iteration or if you manage to change a method, improve it by an order of complexity and feel the happiest guy. If you love coding... Don't change.

We need you to be a developer. The world needs you to become a great developer. The world needs you to improve it.

And that takes time.

17.4.13

The Clean Up Your Own Mess Pattern

Is this a pattern?

In most cases you want to change the value of a variable and leave it that way but in some occasions you want to change the value of a variable, perform some action and then set it back to whatever it was.

With this piece of code it's really easy and you are certain you won't forget. Ever.

The idea is to create a class that implements the IDisposable interface and saves the values to the state they were when we created it and set them to whatever we need to work. Later, in the Dispose, it changes back the variables to the initial values:
public class ManageAllowUnsafeUpdates : IDisposable
{
    private readonly bool allowUnsafeUpdatesStatus;
    private readonly SPWeb Web;

    public ManageAllowUnsafeUpdates(SPWeb Web)
    {
        this.Web = Web;
        allowUnsafeUpdatesStatus = Web.AllowUnsafeUpdates;
        Web.AllowUnsafeUpdates = true;
    }

    public void Dispose()
    {
        Web.AllowUnsafeUpdates = allowUnsafeUpdatesStatus;
    }
}

Using it later is as simple as:
using (new ManageAllowUnsafeUpdates(Web))
{
    ListItem[this.FieldName] = (String)value;
    ListItem.Update();
}

And you don't have to think any more if the value was set to true or false before because the class will manage that for you.

This is a silly piece code but I think serves well to demonstrate the idea.

Tracing Untraceable Exceptions

My application has been working nicely until today. Now, every time I launch it, it crashes with no obvious information but the dreaded CLR20r3.


Description:

  Stopped working



Problem signature:

  Problem Event Name: CLR20r3

  Problem Signature 01: appName.exe

  Problem Signature 02: 1.0.0.0

  Problem Signature 03: 516eb9fb

  Problem Signature 04: mscorlib

  Problem Signature 05: 2.0.0.0

  Problem Signature 06: 4a27471d

  Problem Signature 07: 20c8

  Problem Signature 08: 100

  Problem Signature 09: N3CTRYE2KN3C34SGL4ZQYRBFTE4M13NB

  OS Version: 6.1.7600.2.0.0.274.10

  Locale ID: 2057


I have read on the internet that I'm not the only one.

And the most annoying part is that if I try to debug it it won't fail...

Finally I found a blog post with answers Handling “Unhandled Exceptions” in .NET 2.0.

And I came up with this piece of code, that I placed in the Main method of my application:
AppDomain.CurrentDomain.UnhandledException += delegate(object sender, UnhandledExceptionEventArgs e)
 {
  EventLog.WriteEntry("appName", string.Format("{0} – IsTerminating = {1}", e.ExceptionObject.ToString(), e.IsTerminating), EventLogEntryType.Error, 001);
 };
Then I ran my app again and it crashed, but this time I got a NICE exception in the event viewer.

If you want to know, it was because it was not being able to load a DLL it was expecting

Thanks Mark.

Windows Azure IaaS Virtual Machines prices are competitive, but not cheap

As you would probably know Microsoft has made available to everyone the new Windows Azure Virtual Machines and Virtual Network. You can see more detail on the matter here.

Everything is fine but I still don't like the pricing...

The screenshots look great (I haven't had time to test it lately) and they have included more templates, including one trial VM for SharePoint 2013 that will expire on October this year.

I put the trial in bold because who would pay $270 a month for testing in a VM that will probably self destruct in six months?

The greatness of IaaS is that anyone can create anything without having to spend a lot of money in hardware, I see that. But in the other hand, if the pricing of the platform is so close to what the traditional hosting already offers... What's the point?

A fast search will show you that you can get a 4 Core 2GB (RAM) and 500GB (HDD) for £79 a month, why paying £85 for a similar machine hosted in the cloud?

The main upside is in Azure you can create VMs and delete them in a matter of minutes and in a traditional vendor you have to phone them and normally tell them a month in advance that you want to terminate the service.

If you want to create a farm and test how to connect everything you can do it, configure it, play around a bit and then delete everything for something like $1 an hour, that's good. But if you want something more stable the differences between Azura and a traditional hosting are not so big.

Needless to say the technology is amazing, built in load balancer, easy creation of virtual networks, new templates and the VMs with 56GB of RAM.

And needless to say I am very excited about it and can't wait to put my hands on it.

But I still see it a bit pricey.

13.4.13

Changing the Branding on Blogspot

After changing the name of the blog it needed a new look.

My first choice, as always, was to try to do it by myself. I thought about it for a while, I'm awful when it comes to design and Quite Surprisingly now I don't have the time to play the One Pixel Up - One Pixel Down game so I tried to find some templates.

Great idea! There are thousands of blogger templates out there and even better, most of them are free.

I got mine from Deluxe Templates and the hardest part was to chose one. I wanted something more fantastic but finally I decided for this one because it looks more professional. The name of Code Grimoire is weird enough, I don't need the picture of a lich even though I would like to have one.

The only thing you have to bear in mind when changing your template is that most of the content of the widgets will be lost and replaced by the one in the template so make sure you have it saved somewhere before you upload the new one.

The way of applying the new template is really easy, in Template click on the button on the top right corner's button and upload the new template. After that go to design and play around with the widgets until you are happy with your "new" blog.

I was so easy now I will be tempted to do it again... soon...

Abstract Classes and Collection Types in Service Reference Settings

I have been adding some objects to one of my web services, they were representing a hierarchy of similar items so I used a nice abstract factory.

The code was working perfectly on the server and then I added one object as a response of a web service method.

Nice!

Finally I went to a solution that was using that web service with the naive intention of using it as i always do and updated it but to my surprise, after the update I got a couple of hundred errors in the code.

What used to be ObservableCollections were now Arrays.

I tried to change the Service Reference Settings back to normal but it was as usual with the Collection Type parameter set as Observable Collection. What was happening?

Back to the code I tried everything I could think of in a Saturday afternoon and nothing changed until I commented the abstract class. That made everything go back to normal.

In the end I have changed all the abstract classes to normal classes and the abstract methods to virtual. It's not the same and the code is awkward but now, at least I can use my web services as usual.

By the way, I had a lot of private setters I had to get rid of. I was getting:

error CS0200: Property or indexer "propertyName" cannot be assigned to -- it is read only
So I had to change the properties from this:
public string Label { get; private set; }
To this:
public string Label;

So annoying!

By the way, thanks Fiddler2.

I don't usually post about unsolved issues but I don't use to write one post a day either it is not my fault, it's The Ultimate Blog Challenge's ...



Saturday Afternoon Fever

Isn't is nice to work in the night when everyone is sleeping or in a Saturday afternoon when everyone else is having fun with their friends and family or simply resting after the week. If you want a short answer No, it's not.

But if you want a long answer...

The world is quiet, no one asks you anything, no phone calls. It's the perfect ambiance to dive into The Zone for a couple of hours in a row.

  • You chose the exact time when you want to start to work
  • There's no noise around
  • No meetings
  • No calls to see how you are doing
  • No emails with new problems
It's just you and your code.

I know there are coders who don't recommend it... You can be against the zone but you know you like it.

And as it's Saturday and I have to code that's where I'm going now.

See you later.


Please do not disturb.



No wonder why my weeks are so long and my weekends seem so short to me.

12.4.13

From Developing is Painful to Code Grimoire

Yesterday I didn’t write anything because I spent the time for the writing in buying a domain for the blog. You might have noticed that the URL has changed already, the new look will come today, hopefully.

 

I have been called a freak just for using the word Grimoire, in case you don’t know what a grimoire is as my spelling dictionary did a grimoire is a textbook of magic.

 

I have always thought about the code a bit as magic, sometimes you can’t see why a bit of code doesn’t work and sometimes after you fix it, you can’t understand why it is working. That’s what I call magic.

 

Wizards spent their whole lives investigating, most of the time by testing and, after living in their towers for years and years, they managed to acquire a kind of knowledge that was completely foreign to the rest of the population, just as we do… except for the towers.

 

Every wizard would have a spell book or a grimoire to write down their findings and experiences, in the XXI century we use blogs.

 

I’ll change the branding of the blog, I don’t want it to look like a WoW page, it will be clumsy, but maybe just a bit more esoteric than it is now.

 

Hopefully Adam will help me with it.

10.4.13

Tasks are Dead, Long Live Stories

The worst nightmare for a scrum master or any agile project manager is to find out that at the end of the sprint the 80% of the tasks are done and none of the stories have been finished.

The theory says stories should be split in shorter stories if they don’t fit in a sprint buy I would like to take that a step further… Where possible tasks inside each story should be promoted to stories.

Of course this is not always possible, or is it? It depends on how you plan the tasks in your story, of course if you create a task for every new function you plan to add then you can’t make a story of it but, in the other hand, don’t you think you spend too much time in the scrum board sticking post-its?

Maybe if you have a huge development team that takes a week to coordinate it is impossible but in the other hand if you take a week to coordinate your team you might be doing something wrong.

I don’t know if this is possible in all cases but I think it’s worth just trying to do it. The smaller the story the tighter control you’ll have on the project and the more feedback you’ll get.

Wouldn’t it be nice in the stand up meeting to have the last day’s stories shown? just in a couple of minutes. It will be much better than the usual “yeah, I am still working on that… Good progress… yeah no impediments.”

I will try it. Let’s see how it goes. If it messes up everything I’ll come back and keep you updated and I will finally understand why they didn’t call me to be one of the 17 who created the Manifesto for Agile Software Development.

9.4.13

Testing in Team Foundation Service 2012

The Update 2 of the TFS was released on the 4th of April and some new ALM features were added. I liked the new testing part and in this post I am going to go through the process of creating a new test plan, do some testing and adding a bug to the project.

Microsoft is doing a great effort catching up with testing and even though it’s still really hard for SharePoint development we should also do it because the benefits in general and the peace of mind it brings in particular are worth it.

Before we start we will need to download the Visual Studio Test Professional once we have it we are ready to go. Downloading this piece of software just for allowing you to create tests in the web is definitely overkilling but it’s the way it is…

Your computer will need to be restarted, bear this in mind too.

Creating a Test Plan:
Before we start we need a plan…

The way of creating a plan is perfectly explained here, there’s no need for me to repeat the steps basically you need to open the Microsoft Test Manager, Connect to your TFS and create a plan.

You are ready to start!
Creating a New Test:
Piece of cake.

Go to your TFS web and then to the TEST tab. Click New.
You’ll have to name it and assign, so far so good.

image

Then I will add some steps.

image

Adding steps is really easy and fast. I have added the last one to make sure we can find something that fails.
Then we click Save and Close.

Run the Tests:
That was easy!, now the hard part.

In order to run the test we just have to click the run button, make sure you allow the site to open pop up windows, otherwise it won’t work.

image

If the test is not applicable or is on hold we can mark it so in the menu at the top right corner:

image

If it needs to be tested we have to manually check that every step works. After testing something we will click the green check if it works or the red x if it doesn't.

If any of the steps doesn't work we will be asked to add a commentary about it.

image

Then we can create a bug right there or save and close.

If we click on the create a bug button it automatically will show us a bug form with the status of the steps on the test case to facilitate the description of the issue. We will have to name the bug, assign it and we will be able to give it a severity value.

image

After we save the test case it will be shown as Failed in the test plan.

image

I’ll suppose the bug has been fixed and run the test again.

Good now every step worked and the test is marked as passed in the test plan.

image

Conclusion:
It was fast, nice and easy.

I don’t know to what point this will be an able substitute to the infrastructure we have already in place… but no doubt it looks nicer.

8.4.13

SharePoint List Names Localization

You can’t call a list by its internal name by default in SharePoint or I don’t know how, you have to use the display name or the GUID and that’s an issue when it comes to localization.

In order to solve this issue I have created a couple of wrappers based on the URL. The idea is simple, I use a set of constants to store the “internal names” of the lists and always retrieve them using this constants. So far it has worked fine.
/// <summary>
/// Returns null if the list is not found
/// </summary>
public static SPList GetListByInternalName(this SPListCollection Lists, string InternalName)
{
    if (string.IsNullOrEmpty(InternalName)) return null;

    Guid ListGuid = Lists.GetListIdByInternalName(InternalName);

    if (ListGuid != Guid.Empty)
        return Lists[ListGuid];

    return null;
}

/// <summary>
/// Returns the UIDC of the list
/// </summary>
static Guid GetListIdByInternalName(this SPListCollection Lists, string InternalName)
{
    if (string.IsNullOrEmpty(InternalName)) return Guid.Empty;

    foreach (SPList list in Lists)
        if (list.GetInternalName().ToLower() == InternalName.ToLower())
            return list.ID;

    return Guid.Empty;
}

/// <summary>
/// Gets the Url of the list. That's what we consider its internal name
/// </summary>
public static string GetInternalName(this SPList list)
{
    string Url = list.RootFolder.ServerRelativeUrl;

    string[] split = Url.Split('/');

    return split[split.Length - 1];
}

After this the display name of the list is not relevant any more and localization is simple or, at least, a bit simpler.

7.4.13

Loving my Lumia 920

I was told that WP8 is almost like WP7, same look and feel, same tiles, same functionality… boring. I had my new phone on Wednesday and I tell you, NOT AT ALL.

The push notifications is now working as it should, every time something happens I get a notification. With WP7 this was true only for the email.

The mini tiles are a fantastic way of being updated on all your email accounts and apps without having to scroll. gMail, Outlook, 2 inboxes linked in one for the company product’s emails, my company email, phone calls, SMS, Whatsapp, Me and the calendar all there. It’s there. When I look to my screen I can see all at once.

The dynamic lock screen is also great, I have installed the free Accuweather app and it updates the lock screen with information. I am so relieved now that I don’t need to look at the window to see if it’s sunny…

Office, OneNote, Photos, everything is now connected to SkyDrive seamlessly and works as expected.

Skype integration is really good. For the first time I have been able to make and receive calls seamlessly with the mobile phone and the quality of the sound is as good as with normal calls (both over 3G and of course WiFi)

And finally the Nokia Apps. City lens made me feel for for the first time what augmented reality is, I had seen it but never experienced it. All the Here apps are great, very well finished, I have used mostly HERE Transit. And of course Nokia Music, I don’t know yet how it works but so far I have been able to listen to a lot of good music of my choice completely for free.

It could be that we are in that phase of the relationship but I was expecting a dull, more of the same phone and I found a very exciting device that really improves the productivity and has a big screen for games.

The 5 Most stupid (and common) errors of SharePoint developers

If you are a seasoned developer you'll probably agree, if you are not so experienced you'd have probably suffered at lest one on the last week and lost an hour before figuring out what was happening.

Updating the wrong web
You are changing something, the code works, no exception is triggered but you go to the SharePoint site and nothing has changed... Check the web you are pointing at, you might be updating the wrong one.

Wrong field name
Your CAML query is fine but you get an exception saying that the list's fields are wrong. Go to the list definition and to the field definition, there in the URL you have the field internal name as SharePoint expects it, yeah you got it wrong. I like to have them as constants somewhere, spell them one properly and let the intellisense remind you of the name for the rest of the project's life.

Wrong list
The item you are trying to create doesn't appear in the list or it complains about the field names and they are right... Check that you are connected to the right list. (I also have constants for the list names)

Wrong DLL
You change something in your code and still you get the same error. Depending on what you are doing you might need to deploy the DLL to the GAC or the BIN folder, the right DLL, and then reset the right service, sometimes it's not the IIS. Try changing a string or something very obvious to spot so you are certain the code you are running is the code you want to run.

My web services won't connect to the server
You need to create a clientaccesspolicy.xml file or a crossdomain.xml and put in in the web application's inetpub folder, so simple and yet so annoying.

[BONUS] The changes I am making do nor appear in SharePoint
Right list, right web, right fields, right everything and yet nothing happens when I change the object... You forgot to update it. And this goes for a lot of different items in SharePoint, not only list items. In doubt I always try to add a .Update after modifiying. Beware, the updates are usually slow and should be kept to a minimum.

5.4.13

SharePoint Transparent Data Encryption

SharePoint implements security at a user level, if you want to access a file and you don’t have the required permissions you will be blocked and that’s enough for the vast majority of companies, but…

- What happens if someone gains access to your database server?
+ They will be easily able to download the content database, connect it to one of their SharePoint servers and access everything even the branding.
- What can I do about it as a SharePoint developer?
+ Nothing.
- OMG! OMG!
+ Chill out SharePoint guy, if you want to encrypt the databases ask the database guy.

Yes, database administrators can encrypt whole instances of SQL using TDE. And it looks even easy although I’m happy I don’t have to do it.

- But encrypting everything will affect the performance of my farm!
+ Microsoft reckons it will be only a 3 – 5% increase.

You can get more info about this topic here and here (i.e.).

4.4.13

Dynamic Lock Object Provider

Sometimes you need to create a lock object dynamically, like in the post I wrote yesterday.

When you are retrieving a property from the property bag you don’t want to block the all the property bags for all the webs, you just want to make sure the property you are looking for has not changed since you started the lock and that needs a dynamic way of creating locks.

I started this class with a dictionary but after using it for a while I realized that dictionaries are not thread safe for reading… so I changed it to a hash table.
public class Locks
{
    /// <summary>
    /// This hashtable will provide a lock for every different key in the cache. Hashtables are threadsafe for one writer many readers scenario
    /// </summary>
    private static Hashtable LocksCollection = new Hashtable();

    public static object GetLock(string Key, params object[] args)
    {
        if (args!=null)
            Key = string.Format(Key, args);
           
        if (!LocksCollection.ContainsKey(Key))
        {
            lock (LocksCollection)
                if (!LocksCollection.ContainsKey(Key))
                    LocksCollection.Add(Key, new object());
        }

        return LocksCollection[Key];
    }
}

With this new addition in the method for setting web properties we just lock that web property bag leaving the other webs in the server unloked. It looks like this:
/// <summary>
/// Sets a web property to a given value. This method is thread safe.
/// </summary>
/// <param name="Key">The Key for the web property (case insensitive)</param>
/// <param name="Value">Value to set the property to</param>
public static void SetWebProperty(this SPWeb Web, string Key, string Value)
{
    Key = Key.ToLower();
    lock (Locks.GetLock("WebPropertyWriteLock - {0}", Web.ID))
    {
        if (GetWebPropertyThreadSafe(Web, Key) != Value)
        {
            Web.Properties[Key] = Value;

            Web.Properties.Update();
        }
    }
}

3.4.13

Sharing Data Between Web Front Ends With Thread Safe Web Properties

Remember back in the old days when you only had one frontend? Remember when you thought that was complex?

Times have changed and now for almost every production environment you have several frontends and that’s good, the performance is boosted and with SharePoint it’s simple to add one more if you feel you are falling short but this has also taken some tasks to a new level of complexity.

There is software out there like AppFabric that will allow you to persist data and share it between all your servers and they are good… but would you install and configure them everywhere just for sharing a 10 character string?

Well you don’t have to. You can use SharePoint Web Properties to do so. They are fast to write, to read, to code and you don’t need to configure anything.

SharePoint Web Properties are a bit tricky to use though, just a bit, particularly when you are using them in a multithreaded process but I hope these wrappers save you some time. (I have used them in a couple scenarios and work as expected but I can’t assure they are completely safe.)
static object PropertyWriteLock = new object();

/// <summary>
/// Sets a web property to a given value. This method is thread safe.
/// </summary>
/// <param name="Key">The Key for the web property (case insensitive)</param>
/// <param name="Value">Value to set the property to</param>
public static void SetWebProperty(this SPWeb Web, string Key, string Value)
{
    Key = Key.ToLower();
    lock (PropertyWriteLock) //It's better to have a lock just for the key we are working with. I'll post the trick maybe tomorrow.
    {
        if (GetWebPropertyThreadSafe(Web, Key) != Value)
        {
            Web.Properties[Key] = Value;

            Web.Properties.Update();
        }
    }
}

/// <summary>
/// Returns the web property with the given key. This method is thread safe.
/// </summary>
/// <param name="Key">The Key for the web property (case insensitive)</param>
/// <returns>Returns null if not found</returns>
public static string GetWebPropertyThreadSafe(this SPWeb Web, string Key)
{
    Key = Key.ToLower();
    using (SPSite site = new SPSite(Web.Site.ID))
    {
        using (SPWeb newWeb = site.OpenWeb(Web.ID))
        {
            return newWeb.GetWebProperty(Key);
        }
    }
}

/// <summary>
/// Returns the web property with the given key.
/// </summary>
/// <param name="Key">The Key for the web property (case insensitive)</param>
/// <returns>Returns null if not found</returns>
public static string GetWebProperty(this SPWeb Web, string Key)
{
    Key = Key.ToLower();

    return Web.Properties[Key];
}

/// <summary>
/// Removes the web property from the web
/// </summary>
/// <param name="Key">The Key for the web property (case insensitive)</param>
/// <remarks>The web property will remain there but set to null.</remarks>
public static void RemoveWebProperty(this SPWeb Web, string Key)
{
    if (Web.Properties.ContainsKey(Key))
        Web.Properties.Remove(Key);

    Web.Properties.Update();

    if (Web.AllProperties.ContainsKey(Key))
        Web.AllProperties.Remove(Key);

    Web.Update();
}

They have simplified some parts of my application a lot and, I’ll  say that again, they are fast. Give them a go and let me know your thoughts.

2.4.13

Iterating Lists in Small Chunks And Performance

A couple of months ago I created a post about it and in the conclusion I stated that it would probably be faster to act on the whole list by chunks than it is using the big query… It’s not.

I have done the test with a list of 100K items so you don’t have to and, depending on the size of the chunks it can take slightly longer or a lot longer.

If you are using a very small RowLimit (say 5) the number of times you have to perform the query in order to retrieve the whole list will make the process take twice as long or even longer. If you are using a bigger number (like 1000) the performance will be similar to the one achieved retrieving the whole list and processing it in just one query.

In summary, if you are looking for a small number of values and they are likely to be found in the first iterations of the paginated query then the performance boost will be obvious, in some cases I have noticed an average improvement of up to a 95%, but bear in mind that if you are using very small chunks and you are likely to go through the whole list to get the values back your query will be slower, up to an 400% slower or even more.

1.4.13

The Ultimate Blog Challenge

April fools!

Actually it is not a joke. I want to post more and this is a great excuse.

Going from my usual one post per month to the required one post per day sounds a bit daunting a lot, actually, but it also looks like a very interesting experiment and a great challenge.

Most of the times I don’t get to post because I think a bit of code is not interesting enough to be published, discussed or even commented and I am probably right, but in the other hand sometimes there are posts that I find not so good and they end up reporting the most visits to the blog.

The Ultimate Blog Challenge will be the perfect alibi to start posting more and to see if it improves the blog or not.

Let the posting begin.

5.3.13

I have been chosen as one of the MAPs for 2013

Microsoft Active Professional 2013
Most of the time I think I am shouting in an empty hall when I publish new posts.

When I feel like that, I still post what seems interesting to me when I get the time to, which is almost never because sometimes I like to read my blog and remember the way I have gone through.

It is so gratifying to see that someone out there cares.

Coding, exploring new things and writing are some things that I love but, even though, sometimes it is becomes hard and tiresome guess why I call my blog Developing is Painful...

Getting this recognition was completely unexpected, a great surprise, and it really pushes me to keep on spreading the word and in contact with the new technologies.



Thanks for noticing Microsoft.

4.2.13

Iterating Big SharePoint Lists In Small Chunks

Why would anyone want to iterate SharePoint lists in small chunks? because of two main reasons, the first one is because if the results of one of your queries is too big (say 2000 items) it gets really slow to work with it and second because you might just need the top 20 items that fulfil a condition and that condition is too complex to be represented in the first CAML query.

I am using query pagination to do it and it’s about 75% faster than processing all of the items in the first query (this depends on the amount of data you have in the list)

It basically goes as follows, you set the RowLimit to a sensible number (you also want to keep the number of round trips to a minimum) then you query the SharePoint List until the number of items that fulfil the condition hits the target or you run out of items. (I still haven’t tested if it’s faster to go through every item in a big list using this method or retrieving all at once but something tells me this will be faster)

/// <summary>
/// Returns the top MaxResults items in the query
/// </summary>
internal static List<SPListItem> GetTopItems(SPWeb web, uint MaxResults = 0)
{
    SPList ListToQuery = web.Lists.GetListByInternalName("ListInternalName");


    List<SPListItem> result = new List<SPListItem>();

    SPQuery query = new SPQuery();
    query.Query = string.Format(@"<Where><Eq><FieldRef Name='State' /><Value Type='Choice'>Live</Value></Eq></Where>");
    query.ViewAttributes = "Scope='RecursiveAll'";
    query.RowLimit = MaxResults + 1;

    SPListItemCollection QueryResults;
    string PagingInfo = string.Empty;

    while (result.Count < MaxResults && PagingInfo != null)
    {
        //We set the beginning of the query to the last item in the previous page
        query.ListItemCollectionPosition = new SPListItemCollectionPosition(PagingInfo);

        //We get the page of items
        QueryResults = ListToQuery.GetItems(query);

        foreach (SPListItem item in QueryResults)
        {
            //We process the items and add them to the result list
            if (ComplexProcessing(item))
                result.Add(item);
        }

        //We set PagingInfo to the last item retrieved in the query
        if (QueryResults.ListItemCollectionPosition != null)
            PagingInfo = QueryResults.ListItemCollectionPosition.PagingInfo.ToStringSafe();
        else //PagingInfo will be null if we reach the end of the pagination
            PagingInfo = null;
    }

    return result;
}

This approach to paginated CAML queries have improved the performance of my site hugely, I hope it serves you as well.