Migrating SharePoint Users to a New Domain

I have been dreading this type of migration for years... so many years that I already had planned a couple of ways of solving the issue. It finally happened.

Scenario:

Someone decides we need to change the farm from one environment to a new one completely different with a new AD and in a new city.

Well, let's get to it. We have created a new SharePoint farm in the new environment and we have backed up and restored the content databases. We have manually changed the admin of the site collection to the new admin in the new AD in the new farm and we can access the site and see the data. Fantastic.

Fantastic?

The users in the list items are the users from the old farm. And we have several lists with a lot of user fields. And some of our lists have tens or hundreds of thousands of rows. Changing them manually is not an option.

First idea: Go refined and try stsadm -o migrateuser:

Ohh so easy... we change the login name of the user to something else and we are good because the user IDs are still the same... NO.

This is a new domain and we don't have access to the old domain users so the migrateuser parameter throws a nice User not found error.

Second idea: Go berserk and change the strings in the list items

And that worked. Oh the beauty of a simple idea. The process is pure brute force... beautiful in its barbarity... If you have read up to here you are probably desperate for a solution.

Step One:
Get all of the users from the old farm in an XML file or something really high tech (a csv could work too).

static void Main(string[] args)
{
    using (SPSite site = new SPSite(args[0]))
    {
        using (SPWeb web = site.OpenWeb())
        {
            XElement users = new XElement("Users");

            foreach (SPUser user in web.SiteUsers)
            {
                XElement xmlUser = new XElement("User");
                xmlUser.Add(new XAttribute("Name", user.Name));
                xmlUser.Add(new XAttribute("LoginName", user.LoginName));
                xmlUser.Add(new XAttribute("ID", user.ID));

                users.Add(xmlUser);
            }

            users.Save("SiteUsers.xml");
        }
   
    }

    Console.WriteLine("\nProcess finished...");
    Console.ReadLine();
}

Step Two:
Make sure all the users you need are in the new AD. As you have a list in XML you can pass it to someone with privileges in the AD.

Step Three:
Ensure the users in SharePoint, add them to a group with reading permissions and then iterate through all the items in the list changing the users from the old domain to the users in the new one.

static void Main(string[] args)
{
    XElement users = XElement.Load("SiteUsers.xml");
    string newDomain = "XXXXXXXX";

    string ListName = string.Empty;
    if (args.Length == 2) ListName = "Stratex Framework";
    else ListName = args[2];

    //Args are SiteUrl VisitorsGroup ListName
    using (SPSite site = new SPSite(args[0]))
    {
        using (SPWeb web = site.OpenWeb())
        {
            SPList listToUpdate = web.Lists[ListName];

            Dictionary<string, SPUser> NewUsers = new Dictionary<string, SPUser>();

            foreach (XElement user in users.Descendants("User"))
            {
                string LoginName = FixDomain(user.Attribute("LoginName").Value, newDomain);
                SPUser spUser = null;
                try
                {
                    //We try to ensure all the users from the XML file. We'll probably need them
                    spUser = web.EnsureUser(LoginName);
                }
                catch
                { Logger.WriteLine("The user {0} could not be found.", LoginName); }

                if (spUser != null)
                {
                    SPGroup viewers = web.Groups[args[1]];

                    viewers.AddUser(spUser);
                    //Finally we add them to a group with read permissions
                    //We can worry about restricting this further after the migration

                    if (!NewUsers.ContainsKey(LoginName)) NewUsers.Add(user.Attribute("ID").Value, spUser);
                }
            }

            web.Update();


            UpdateUsersInList(listToUpdate, NewUsers);
        }

    }

    Logger.WriteLine("\nProcess finished...");
    Console.ReadLine();
}

private static void UpdateUsersInList(SPList list, Dictionary<string, SPUser> NewUsers)
{
    int itemsInList = list.ItemCount;
    Logger.WriteLine("Updating users at {0}. {1} items.", list.Title, itemsInList.ToString());

    SPQuery qry = new SPQuery();
    qry.ViewAttributes = "Scope=\"RecursiveAll\"";
    SPListItemCollection allItems = list.GetItems(qry);
    int count = 0;

    UpdateCount(count++, itemsInList);

    foreach (SPListItem item in allItems)
    {
        try
        {
            bool changed = false;
            SPFieldCollection allFields;
            //If the item has content type it has usualy less fields
            if (item.ContentType == null)
                allFields = item.Fields;
            else
                allFields = item.ContentType.Fields;

            foreach (SPField field in allFields)
            {
                if (field is SPFieldUser)
                    changed = ChangeUserToNewDomain(item, field, NewUsers) || changed;
            }

            changed = ChangeUserToNewDomain(item, item.Fields.GetFieldByInternalName("Author"), NewUsers) || changed;
            changed = ChangeUserToNewDomain(item, item.Fields.GetFieldByInternalName("Editor"), NewUsers) || changed;

            if (changed) item.SystemUpdate(false); //if the item has not been changed we won't update it to save time
        }
        catch (Exception ex) { Logger.WriteLine("Failed to update item {0}. Exception {1}", item.Title, ex.Message); }

        UpdateCount(count++, itemsInList);
    }

    UpdateCount(count++, 0);
}

private static bool ChangeUserToNewDomain(SPListItem item, SPField field, Dictionary<string, SPUser> NewUsers)
{
    bool changed = false;
    string fieldContent = item[field.InternalName] == null ? null : item[field.InternalName].ToString();

    if (string.IsNullOrEmpty(fieldContent)) return false;

    List<string> oldUserIds = GetUserIDs(fieldContent.Split(new string[] { ";#" }, StringSplitOptions.RemoveEmptyEntries));

    if (oldUserIds.Count == 1)
    {   //The field has only one user in it
        SPFieldUserValue foundUser = FindUser(NewUsers, oldUserIds[0]);

        if (foundUser != null)
        {
            item[field.InternalName] = foundUser;
            changed = true;
        }
    }
    else if (oldUserIds.Count > 1)
    {   //The field has several users in it
        SPFieldUserValueCollection usersInField = new SPFieldUserValueCollection();
        foreach (string oldUser in oldUserIds)
        {
            SPFieldUserValue foundUser = FindUser(NewUsers, oldUser);

            if (foundUser != null)
                usersInField.Add(foundUser);
        }

        if (usersInField.Count > 0)
        {
            item[field.InternalName] = usersInField;
            changed = true;
        }
    }
            
            
    return changed;
}

private static List<string> GetUserIDs(string[] UserTokens)
{   //We do not care about the login name. The ID is gold
    List<string> result = new List<string>();

    if (UserTokens.Length > 0)
    {
        for (int i = 0; i < UserTokens.Length; i++)
        {
            int id;

            if (i % 2 == 0 && int.TryParse(UserTokens[i], out id))
                result.Add(id.ToString());
        }
    }

    return result;
}

private static SPFieldUserValue FindUser(Dictionary<string, SPUser> NewUsers, string oldUser)
{
    SPUser foundUser = null;

    if (NewUsers.ContainsKey(oldUser)) foundUser = NewUsers[oldUser];
    else
    {
        //If we can't find the ID of the user we will still try with the login or even with the Display Name
        foreach (SPUser newUser in NewUsers.Values)
        {
            if (newUser.Name == oldUser || newUser.LoginName == oldUser) { foundUser = newUser; break; }
        }
    }

    if (foundUser != null)
        return new SPFieldUserValue(foundUser.ParentWeb, foundUser.ID, foundUser.Name);
    else
        return null;
}

private static string FixDomain(string loginName, string newDomain)
{
    //Here we change the users from XXXXX\\User to YYYYY\\User
    //The source domain was claim based
    if (loginName.Contains("|")) loginName = loginName.Split('|')[1];

    string[] tokens = loginName.Split('\\');

    tokens[0] = newDomain;

    return string.Join("\\", tokens);
}

private static void UpdateCount(int currentItem, int itemsInList)
{
    int percentage;

    if (currentItem == 0) percentage = 0;
    else if (itemsInList == 0) percentage = 100;
    else
    {
        //We will only change the value every 10 times to make the process faster.
        if (currentItem % 10 != 0) return;
        percentage = currentItem * 100 / itemsInList;
    }
    Console.Write("\r");
    if (percentage >= 0 && percentage < 10)
        Console.Write("  ");
    else if (percentage >= 10 && percentage < 100)
        Console.Write(" ");

    Console.Write("{0}%", percentage);
}

This is a first prototype that has worked as expected but it's not fully tested (by far) if you need it you can use it as a base to develop your own tool.

The one who possesses the strings has the power.

No more recursive functions to define CAML Queries thanks to Camlex

Some times you have a random number of conditions to check in a CAML query and in those cases I used to define the queries using a random recursive function that I usually have to debug a few times works perfectly on the first go.

The code for those queries would be something like this (and this is a simple one):

public List<string> GetSomeInfo(string fieldsToSearch, string contentTypesToSearch)
{
    ...

    var queryval = string.Empty;
    if (contentTypesToSearch.IsNullOrEmpty())
        queryval = string.Format("<Where>" + GenerateFieldsQuery(fieldsToSearch.Split(','), 0) + "</Where>", text);
    else
        queryval = string.Format("<Where><And>" + GenerateCTypesQuery(contentTypesToSearch.Split(','), 0) + GenerateFieldsQuery(fieldsToSearch.Split(','), 0) + "</And></Where>", text);

    var scope = "Scope=\"RecursiveAll\"";

    ...
}

private static string GenerateFieldsQuery(string[] fields, int index)
{
    if (fields.Length == 0) return string.Empty;

    if (fields.Length == index + 1)
        return "<Contains><FieldRef Name='" + fields[index] + "' /><Value Type='Text'>{0}</Value></Contains>";

    return "<Or><Contains><FieldRef Name='" + fields[index] + "' /><Value Type='Text'>{0}</Value></Contains>" + GenerateFieldsQuery(fields, ++index) + "</Or>";
}

private static string GenerateCTypesQuery(string[] cTypes, int index)
{
    if (cTypes.Length == 0) return string.Empty;

    if (cTypes.Length == index + 1)
        return "<Eq><FieldRef Name='ContentType' /><Value Type='Choice'>" + cTypes[index] + "</Value></Eq>";

    return "<Or><Eq><FieldRef Name='ContentType' /><Value Type='Choice'>" + cTypes[index] + "</Value></Eq>" + GenerateCTypesQuery(cTypes, ++index) + "</Or>";
}

That was until now... Thanks to Camlex (and thanks to Luis for showing it to me), that code can be written like this:

public List<string> GetSomeInfo(string fieldsToSearch, string contentTypesToSearch)
{
    ...

    var queryVal = string.Empty;
    var fieldExtensions = new List<Expression<Func<SPListItem, bool>>>();
    var cTypeExtensions = new List<Expression<Func<SPListItem, bool>>>();

    if (!contentTypesToSearch.IsNullOrEmpty())
    {
        foreach (var cType in contentTypesToSearch.Split(','))
            cTypeExtensions.Add(x => (string)x["ContentType"] == cType);
    }

    foreach (var field in fieldsToSearch.Split(','))
        fieldExtensions.Add(x => ((string)x[field]).Contains(text));

    var expressions = new List<Expression<Func<SPListItem, bool>>>();
    expressions.Add(ExpressionsHelper.CombineOr(cTypeExtensions));
    expressions.Add(ExpressionsHelper.CombineOr(fieldExtensions));

    queryVal = Camlex.Query().WhereAll(expressions).ToString();

    ...
}

I'll miss the recursive methods though... they made me feel special...

Get the fields available in an SPListItem

When you retrieve the items from a CAML query with the ViewFields parameters set there’s no way (or at least i don’t know it) to find out which fields you have available and populated with data.

The usual way of getting the list of fields from the content type doesn’t work because the content type is null in that kind of items…

The answer, my friend, is sitting in the Xml.

static List<string> ExcludedFields = new List<string> { "z", "ows_ServerRedirected", "ows_FileRef", "ows_PermMask", "ows_FSObjType", "ows__Level", "ows__ModerationStatus" };
/// <summary>
/// Gets a dictionary with the available fields and its values.
/// </summary>
public static Dictionary<string, string> getAvailableFields(this SPListItem item)
{
    XElement row = XElement.Parse(item.Xml);
    Dictionary<string, string> Fields = new Dictionary<string, string>();

    foreach (XAttribute field in row.Attributes())
    {
        if (!ExcludedFields.Contains(field.Name.LocalName))
            Fields.Add(field.Name.LocalName.Substring(4), field.Value);
    }

    return Fields;
}

Using this I have been able to reduce the traffic on a web service method in a 65%.

Scopes in a CAML Query

I have been working for quite a while now with CAML queries and the scope is always something very important to bear in mind. How many times my queries returned nothing when I was sure they should bring back something…

Basically we have two modifiers Recursive and All and nothing, could we call nothing a modifier? All will bring back folders and files. Recursive will repeat the query in all the folders under the one we are working with.

If you don’t set the scope to All it will only bring files. If you don’t set it to recursive it will only retrieve items from the folder you are at. There are not that many variants so let’s make an example of each.

Let’s imagine we have a SharePoint folder like this one and we want to query it:



I am not good at paint, I know but what I want to show here is a tree where we have a Root folder (the root of the queries) and two sub-folders with files. For each scope possible I’ll highlight what you can expect to retrieve.

Just before we start, allow me to remind you that the scope is set in the property ViewAttributes of the SPQuery item.

To add a bit more of clarity I have also painted the levels:

The green line marks what’s inside the root folder, the blue line marks the contents of SubFolder1 and the red line SubFolder2.

ViewAttributes left by default:


This is just the files under the root folder.

ViewAttributes = "Scope='Recursive'"



This means all the files in all the folders.

ViewAttributes = "Scope='All'"



This scope will bring folders and files under root.

ViewAttributes = "Scope='RecursiveAll'"

And finally with RecursiveAll you can bring back everything under the root entity.

Good luck with your queries.

How Much Are you Bringing Back In Your CAML Queries?

I don't know you but I am bringing back too much.

This is one of those things you don't notice until it's too late. I thought just adding very strict filters to the CAML Queries to bring back just the items you needed was enough but there's one more thing you can do.

You can restrict the fields you are retrieving... and you should.

The SQL Queries that SharePoint generates for retrieving the items when you let the ViewFields parameter of the CAML empty almost doubles the complexity of the one it generates when you specify the fields you want to query, or use a view.

Two simple lines like these:

query.ViewFields = "<FieldRef Name=\"Value\" />";
query.ViewFieldsOnly = true;

Can make your query better.

Who could resist to do it right when it's this simple?

Disposing every SPWeb you use... Is it necessary?

I am becoming a bit paranoid about disposing lately. In example.

Until now I would have used things like Web.ParentWeb and then forgotten about them. As I was not "Opening" them manually I was not disposing them either.

Now I usually do things like this:

using (SPWeb web = anotherWeb.ParentWeb)

{

      //Do Whatever you need here with the parent web

 }

I haven't had the time to check whether this is reducing the memory leaks or just making my application more unstable (if I close the web while I am using it somewhere else it could cause an exception it the other part of the code)

Do you think when you dispose a web it automatically disposes every parent web automatically?
It could be...

I need to test this and once I do it I'll post about it.

SharePoint List Names Localization

You can’t call a list by its internal name by default in SharePoint or I don’t know how, you have to use the display name or the GUID and that’s an issue when it comes to localization.

In order to solve this issue I have created a couple of wrappers based on the URL. The idea is simple, I use a set of constants to store the “internal names” of the lists and always retrieve them using this constants. So far it has worked fine.

/// <summary>
/// Returns null if the list is not found
/// </summary>
public static SPList GetListByInternalName(this SPListCollection Lists, string InternalName)
{
    if (string.IsNullOrEmpty(InternalName)) return null;

    Guid ListGuid = Lists.GetListIdByInternalName(InternalName);

    if (ListGuid != Guid.Empty)
        return Lists[ListGuid];

    return null;
}

/// <summary>
/// Returns the UIDC of the list
/// </summary>
static Guid GetListIdByInternalName(this SPListCollection Lists, string InternalName)
{
    if (string.IsNullOrEmpty(InternalName)) return Guid.Empty;

    foreach (SPList list in Lists)
        if (list.GetInternalName().ToLower() == InternalName.ToLower())
            return list.ID;

    return Guid.Empty;
}

/// <summary>
/// Gets the Url of the list. That's what we consider its internal name
/// </summary>
public static string GetInternalName(this SPList list)
{
    string Url = list.RootFolder.ServerRelativeUrl;

    string[] split = Url.Split('/');

    return split[split.Length - 1];
}

After this the display name of the list is not relevant any more and localization is simple or, at least, a bit simpler.

The 5 Most stupid (and common) errors of SharePoint developers

If you are a seasoned developer you'll probably agree, if you are not so experienced you'd have probably suffered at lest one on the last week and lost an hour before figuring out what was happening.

Updating the wrong web
You are changing something, the code works, no exception is triggered but you go to the SharePoint site and nothing has changed... Check the web you are pointing at, you might be updating the wrong one.

Wrong field name
Your CAML query is fine but you get an exception saying that the list's fields are wrong. Go to the list definition and to the field definition, there in the URL you have the field internal name as SharePoint expects it, yeah you got it wrong. I like to have them as constants somewhere, spell them one properly and let the intellisense remind you of the name for the rest of the project's life.

Wrong list
The item you are trying to create doesn't appear in the list or it complains about the field names and they are right... Check that you are connected to the right list. (I also have constants for the list names)

Wrong DLL
You change something in your code and still you get the same error. Depending on what you are doing you might need to deploy the DLL to the GAC or the BIN folder, the right DLL, and then reset the right service, sometimes it's not the IIS. Try changing a string or something very obvious to spot so you are certain the code you are running is the code you want to run.

My web services won't connect to the server
You need to create a clientaccesspolicy.xml file or a crossdomain.xml and put in in the web application's inetpub folder, so simple and yet so annoying.

[BONUS] The changes I am making do nor appear in SharePoint
Right list, right web, right fields, right everything and yet nothing happens when I change the object... You forgot to update it. And this goes for a lot of different items in SharePoint, not only list items. In doubt I always try to add a .Update after modifiying. Beware, the updates are usually slow and should be kept to a minimum.

Iterating Lists in Small Chunks And Performance

A couple of months ago I created a post about it and in the conclusion I stated that it would probably be faster to act on the whole list by chunks than it is using the big query… It’s not.

I have done the test with a list of 100K items so you don’t have to and, depending on the size of the chunks it can take slightly longer or a lot longer.

If you are using a very small RowLimit (say 5) the number of times you have to perform the query in order to retrieve the whole list will make the process take twice as long or even longer. If you are using a bigger number (like 1000) the performance will be similar to the one achieved retrieving the whole list and processing it in just one query.

In summary, if you are looking for a small number of values and they are likely to be found in the first iterations of the paginated query then the performance boost will be obvious, in some cases I have noticed an average improvement of up to a 95%, but bear in mind that if you are using very small chunks and you are likely to go through the whole list to get the values back your query will be slower, up to an 400% slower or even more.

Iterating Big SharePoint Lists In Small Chunks

Why would anyone want to iterate SharePoint lists in small chunks? because of two main reasons, the first one is because if the results of one of your queries is too big (say 2000 items) it gets really slow to work with it and second because you might just need the top 20 items that fulfil a condition and that condition is too complex to be represented in the first CAML query.

I am using query pagination to do it and it’s about 75% faster than processing all of the items in the first query (this depends on the amount of data you have in the list)

It basically goes as follows, you set the RowLimit to a sensible number (you also want to keep the number of round trips to a minimum) then you query the SharePoint List until the number of items that fulfil the condition hits the target or you run out of items. (I still haven’t tested if it’s faster to go through every item in a big list using this method or retrieving all at once but something tells me this will be faster)

/// <summary>
/// Returns the top MaxResults items in the query
/// </summary>
internal static List<SPListItem> GetTopItems(SPWeb web, uint MaxResults = 0)
{
    SPList ListToQuery = web.Lists.GetListByInternalName("ListInternalName");


    List<SPListItem> result = new List<SPListItem>();

    SPQuery query = new SPQuery();
    query.Query = string.Format(@"<Where><Eq><FieldRef Name='State' /><Value Type='Choice'>Live</Value></Eq></Where>");
    query.ViewAttributes = "Scope='RecursiveAll'";
    query.RowLimit = MaxResults + 1;

    SPListItemCollection QueryResults;
    string PagingInfo = string.Empty;

    while (result.Count < MaxResults && PagingInfo != null)
    {
        //We set the beginning of the query to the last item in the previous page
        query.ListItemCollectionPosition = new SPListItemCollectionPosition(PagingInfo);

        //We get the page of items
        QueryResults = ListToQuery.GetItems(query);

        foreach (SPListItem item in QueryResults)
        {
            //We process the items and add them to the result list
            if (ComplexProcessing(item))
                result.Add(item);
        }

        //We set PagingInfo to the last item retrieved in the query
        if (QueryResults.ListItemCollectionPosition != null)
            PagingInfo = QueryResults.ListItemCollectionPosition.PagingInfo.ToStringSafe();
        else //PagingInfo will be null if we reach the end of the pagination
            PagingInfo = null;
    }

    return result;
}

This approach to paginated CAML queries have improved the performance of my site hugely, I hope it serves you as well.

Restarting SharePoint 2010 Timer Service Programmatically

I need to restart the Timer Service from an application and it took me some time to find the answer. It’s as easy as one could think.

Anyway here's the code:

/// 

/// Stops the SharePoint timer.
/// 
public static void TimerStop()
{
    ServiceController timerService = new ServiceController(Constants.SPTimerName);

    if (timerService.Status == ServiceControllerStatus.Running)
    {
        timerService.Stop();
        timerService.WaitForStatus(ServiceControllerStatus.Stopped, Constants.WaitingTimeout);
    }
}

/// 

/// Starts the SharePoint timer.
/// 
public static void TimerStart()
{
    ServiceController timerService = new ServiceController(Constants.SPTimerName);

    if (timerService.Status == ServiceControllerStatus.Stopped)
    {
        timerService.Start();
        timerService.WaitForStatus(ServiceControllerStatus.Running, Constants.WaitingTimeout);
    }
}

And by the way the constants are:

public static TimeSpan WaitingTimeout = new TimeSpan(0, 1, 30);
public static string SPTimerName = "SharePoint 2010 Timer";

Hiding Edit in Datasheet in the Actions Menu with javascript code

Coming back after all this time to write such an ugly post? That’s exactly my style…
Well, the title of the post explains very well what was the objective and everybody knows that this is not the best way, but editing the permissions was causing issues somewhere else so we decided to do this.
This is the enemy:

And in order to hide it we added a Content Editor Web Part and then clicked in the Source Editor Button:

There we added this script:

<script type="text/javascript" >
var allMenuItems = document.getElementsByTagName('ie:menuitem'); 
for(var i = 0; i < allMenuItems.length; i++ )   
{
 try
        {
         if (allMenuItems[i].text.toLowerCase() == "edit in datasheet")
         {
   var parentNodeOfMenuItem = allMenuItems[i].parentNode;  
                 parentNodeOfMenuItem.removeChild(allMenuItems[i]);                                
  }
 }
 catch(err)
 {}

} 
</script>

It could look a bit weird, but SharePoint won’t mind. Then we should click Save.

After that, in the Appearance group of the web part we could change the parameter Chrome Type to “None” (Or Layout –> Hidden) to make the CEWP invisible to the users.
And that’s all.

Enjoy!

Extension Methods for Null Objects

Is it possible to use an extension method on something that is null?

How beautiful would it be to do something like:

if (!MyString.IsNull()) return MyString.ToString();

I read somewhere a couple of years ago that would be impossible because if the object was null you’d not be allowed to call the method or something like that… But in my head it makes perfect sense.

So I have been refraining myself of doing this for a long time but today I felt brave enough.

The Method Extensions:

public static bool IsNull(this string str)
{
    return str == null;
}

public static string ToStringSafe(this object obj)
{
    return (obj ?? string.Empty).ToString();
}

The main method obviously every good test program should be a console application:

static void Main(string[] args)
{
    string str = null;

    if (str.IsNull())
        Console.WriteLine("It was null...");
    else
        Console.WriteLine("It wasn't null...");

    Console.WriteLine(str.ToStringSafe());

    Console.ReadKey();
}

And it works! This just opens a new dimension to my spaghetti recipes.

The first useful example that comes to my mind apart from the ToStringSafe and ToInt, ToDateTime... Look at me, I can't stop! is:

/// 

/// Disposes the object if it's not null.
/// 
public static void DisposeSafe(this IDisposable DisposableObject)
{
    if (DisposableObject != null)
        DisposableObject.Dispose();
}

With this method I won’t have to worry if the object is null or not. Everything is disposed properly!

It frightens me sometimes to get so excited about this things…

Filter Provider Web Part with a Silverlight Tree in a ModalDialog window

I started with this a couple of days ago… I didn’t want to do it, because I knew it was going to be painful… But they made me… An then I thought it would be a perfect post for the blog because of the painful.
What we wanted to achieve was to filter the items in a List View Web Part by Entity. We have an entity hierarchy, so we thought it would be a good idea to have a web part showing the hierarchy as a tree.

Alright so we need web part with a TreeView that is able to send the selected entity as a filter to a LVWP. Great. I built it… And no-one liked it. They wanted it to be Silverlight, and not only that, they wanted it to be in a pop up window. Yes, they got me. I have never done anything like that. But hey, where’s your spirit of adventure?

It is a lot of code, most of it ugly. So I’ll only post the most tricky parts (basically the parts related to the communication of the pages and the Silverlight) and the URL’s from where I got the ideas.

The firs thing is to get a proper Filter Provider Web Part working. To do so, I followed the instructions from here. My first try was with IWebPartRow, but it wasn’t what I was looking for, so I went for ITransformableFilterValues. It’s pretty straightforward so I won’t comment anything here.
The second thing is to create the PopUp Window. After googling a while I found this post a good point to start. My code in the web part ended like this:

        protected override void OnLoad(EventArgs e)
        {
            if (Page.IsPostBack)
            {
                if (!string.IsNullOrEmpty(GetFormValue("HiddenEntityName")))
                {
                    Page.Session["SelectedEntityName"] = Page.Request.Form["HiddenEntityName"];
                    Page.Session["SelectedEntityID"] = Page.Request.Form["HiddenEntityID"];

                    RenderHeader();

                    //SelectedEntityText.Text = string.Format("{0}  ", Page.Request.Form["HiddenEntityName"], Page.Request.Form["HiddenEntityID"]);
                }
                else
                    RenderHeader();
            }

            string CurrentWeb = SPContext.Current.Web.Url;
            string height = "500";
            string width = "500";
            string page = "/_layouts/stratex/EntityTree.aspx";

            // use next line for direct with  between  and  
            string scrp = @"
                            ";

            Type t = this.GetType();
            if (!Page.ClientScript.IsClientScriptBlockRegistered(t, "bindWebserviceToAutocomplete"))
                Page.ClientScript.RegisterClientScriptBlock(t, "bindWebserviceToAutocomplete", scrp);
        }

I had problems retrieving the values from the ModalDialog. I was not able to find the controls in the web part because the IDs and the Titles of the were being dynamically. The trick I used to fix this was to create two hidden fields in the CreateChildControls:

        protected override void CreateChildControls()
        {
            base.CreateChildControls();

            ...

            Page.ClientScript.RegisterHiddenField("HiddenEntityName", "");
            Page.ClientScript.RegisterHiddenField("HiddenEntityID", "");

            ...
        }

Hey Chan, have you noticed that you could use the session and forget about the HiddenFields? Don’t ask. I’m warning you…

The next part is to get the new aspx page for the modal dialog. I created a new directory in the layouts folder and created my page there. It looks like this:

<%@ Page Language="C#" Inherits="StratExFramework.EntityTree,StratExFramework,Version=2.2.0.0,Culture=neutral,PublicKeyToken=311246df7412ca98" %>

<html>
<head>
<title>Select Entity for filtering</title>
<script type='text/javascript'>
    function PassParameterAndClose(EntityName, EntityID) {

        window.returnValue = new Array( EntityName, EntityID) ;

        var version = parseFloat(navigator.appVersion.split('MSIE')[1]);
        if (version >= 7) 
            { window.open('', '_parent', ''); }
        else
            { window.opener = self; }

        window.close();
    }
</script>
</head>
<body></body>
</html>

I have also created a code behind class as you can see in the first line of the aspx… :

    public class EntityTree : WebPartPage
    {
        string CurrentWeb;
        string SelectedEntityID;

        protected void Page_Load(object sender, System.EventArgs e)
        {
            CurrentWeb = Request.Params["CurrentWeb"];
            SelectedEntityID = Request.Params["SelectedEntityID"];
        }

        protected override void CreateChildControls()
        {
            base.CreateChildControls();

            string Source = CurrentWeb + "/Lists/XAPLibrary/SilverlightEntityTreeSelector.xap";
            string SilverlightHeight = "515";
            string SilverlightWidth = "500";

            LiteralControl obj = new LiteralControl();
            obj.Text = "<object id='silverlightHost' style='height: " + SilverlightHeight + "; width: " + SilverlightWidth + @"; margin: 0; padding: 0;' data='data:application/x-silverlight-2,' type='application/x-silverlight-2'>
                            <param name='Source' value='" + Source + @"' />
                            <param name='MinRuntimeVersion' value='3.0.40624.0' />
                            <param name='Background' value='#FFFFFFFF' />
                            <param name='initParams' value='" +
                                string.Format("{0}={1}", "site", HttpUtility.UrlEncode(CurrentWeb)) +
                                string.Format(", {0}={1}", "selectedentityid", HttpUtility.UrlEncode(SelectedEntityID)) +
                                @"' />
                            </object>";
            this.Controls.Add(obj);
        }

        public override void VerifyRenderingInServerForm(Control control)
        {
            return;
        }
    }

What I am doing here is getting the parameters and passing them to the Silverlight. I use the CurrentWeb to tell the web services I use in the Silverlight what’s the context and the SelectedEntityID to highlight the previously selected Entity in case it’s not null. This is the code of the Silverlight:

namespace SilverlightEntityTreeSelector
{
    public partial class MainPage : UserControl
    {
        public MainPage()
        {
            InitializeComponent();

            Tree.PropertyChanged += new System.ComponentModel.PropertyChangedEventHandler(Tree_PropertyChanged);

            Tree.Show(string.Empty);
        }

        void Tree_PropertyChanged(object sender, System.ComponentModel.PropertyChangedEventArgs e)
        {
            if (e.PropertyName == "SelectedEntityID")
                HtmlPage.Window.Invoke("PassParameterAndClose", Tree.SelectedEntityName, Tree.SelectedEntityID);
            else if (e.PropertyName == "TreeLoaded")
                if (Application.Current.Resources.Contains("selectedentityid"))
                    if (!string.IsNullOrEmpty(Application.Current.Resources["selectedentityid"] as string))
                        Tree.ChangeSelectedItemTo(Application.Current.Resources["selectedentityid"] as string);
        }
    }
}

Now we have all the components.

The way it works is: You chose an entity in the Silverlight tree, then the Silverlight calls the javascript function in the modal window that returns the parameters to the parent web part window and closes the ModalDialog. And then the web part sends the filter to the List View Web Part.


If you try to use this code you’ll be missing some methods, but what I wanted to share here is the solution I've used mainly because I don’t want to think about this again if I’m asked to do something similar in the future.

I would love to share some pics but the web parts haven’t been retouched by the skilled hand of Adam, our designer, and your eyes could pop out of their sockets.

Cloning a SharePoint Virtual Machine And Changing its Name in the process

The first problem I found was that after copying the virtual machine is that I couldn’t add it to my a new VM because the UUID in the VHD was the same as in the previous VM… I did this:

VBoxManage.exe internalcommands sethduuid g:\StratexPoint\StratexPointBigger.vhd

And then it worked. But now we have the tricky part. We have to change the name of the SharePoint WFE.

After we change the name of the server we try to connect to the central administration, but we get “Cannot connect to the configuration database.” so we ran the SharePoint Products and Technologies Configuration Wizard.

It complains that it can’t detect whether we are connected to a farm or not… OK. Then it disconnects the server from the farm and closes.

I must insist, my dear. I run the wizard again and select create a new farm. It tries to connect to the database server that happens to be the same server we have just changed the name…

SQL is not so delicate about names and it works.

Now we have to create a new Config_Database, ok, ok, and it looks like it’s going to work… but nooo.

System.Security.Principal.IdentityNotMappedException: Some or all identity references could not be translated.

After checking this post I changed the username of the identity of the SharePoint Central Administration V3 application and it worked .. but failed after a while, so I deleted the site, the directory and the application pool and tried again.

 

And It worked!

After that I started the services, created an 80 web application and attached the old content database to it.

 

Who said it was impossible?

HTTP 500 (No authority could be contacted for authentication.) in SharePoint

I was getting an HTTP error 500 after restoring my SharePoint development server in a different network (at home). The thing was a bit curious because I could see the SharePoint site from the virtual server, but not from my physical server and yes, I checked the IPs and they were OK. The first thing I did was to configure the IE9 to show me the HTTP errors the way I like them, the “unfriendly” way.

To do so I went to Tools –> Internet Options –> Advanced, and there in the tree below Browsing I unchecked the “Show friendly HTTP error messages”.

After that I could see that the problem I was having when connecting from outside was “No authority could be contacted for authentication.” Here I saw that the easiest way to fix the problem was to remove the server from the domain and then add it again.

And it worked... Finally…

Hiding List View web parts when they are empty

Today I have been required to hide the List View web parts of the solution when they are empty… Even though I don’t like javascript, it was the obvious way to go so I asked Google and it sent me to this post.

I copied the code to a Content Editor Web Part and it worked more or less, but when I tried to hide the column headers and adding a message instead it was only working for two of my the three listview web parts in the page… After a bit more of googling I changed it to:

<script type="text/javascript">
 function HideEmptyWebParts()
 {
   var itemsfound = new Array;
   var elements = document.getElementsByTagName('*');
   for(var i=0;i<elements.length;i++)
   {
      if(elements[i].className == 'ms-vb')
      {
         itemsfound.push(elements[i]);
      }
   }
  
   for (var i=0;i<itemsfound.length;i++)
   {
       if (itemsfound[i].innerHTML.indexOf("There are no items to show in this view of the")>-1)
       {
                itemsfound[i].parentNode.parentNode.parentNode.parentNode.innerHTML="<div class='ms-vb'>Not enough data to produce a dashboard.</div>";
         
       }
   }
 }

_spBodyOnLoadFunctionNames.push("HideEmptyWebParts")
 
</script>

And after that it worked as expected.

Not SharePoint nor SQL Servers are working on the Azure VM Role

As you know, if you read me, I have been trying to be accepted in the Azure VM Role beta program, well, I have finally got a meeting with the guys that could grant me access and they've asked me the most feared question... What do you want it for?

My idea was to set up a farm in the cloud with SQL and SharePoint so I could use SSIS, our BI cube, Reporting services and SharePoint.

Well, this Microsoft guys say that "At the moment that's impossible"... Looks like there are connectivity problems between the machines in VM Role so not SharePoint nor SQL are working properly.

The advice they have for me is to set everything up on Office 365. But, what happens to my BI cube and my reporting services if I can't use web services from the Office 365's sandbox? Hmmm they won't work, so I'll just have half of my solution working and the other half sadly forgotten in the recesses of my hard disk.

The solution they provide... wait until summer.

I'll try to convince my boss to give me six months off, but if this doesn't work I am going to be forced to look for another place to set up my farm on.

Oh and, in case someone doubts it, I wasn't granted the access to the beta program.

And in the meantime Google is giving away laptops for the developers to test their new Chrome OS...

Copying ListViewWebparts between sites with their own custom views

In a new example of what I call "Guerrilla Coding" I had a couple of hours to develop a little program to copy hundreds of webpart pages full of ListViewWebparts, each one with its own custom view. I don't know if you feel the same, but I start to tremble every time I have to touch anything related to the ListViewWebpart

The ListViewWebparts create in the parent list a hidden view when we customize their view. This means that if you want to copy the webparts to a new site you'll have to copy the views from the parent list to its destination equivalent, I don't think I'm explaining my point well... I'd better paste the code...

What I do in this code snippet is copying the webparts from every origin file to the destination one giving them a special treatment if they are LisvViewWebpart which is enough for the application I need.

private static void CopyWebParts(SPFile origFile, SPFile destFile)
        {
            SPLimitedWebPartManager origWpm = origFile.GetLimitedWebPartManager(PersonalizationScope.Shared);
            SPLimitedWebPartManager destWpm = destFile.GetLimitedWebPartManager(PersonalizationScope.Shared);

            DeleteWebparts(destWpm);

            foreach (Microsoft.SharePoint.WebPartPages.WebPart webpart in origWpm.WebParts)
            {
                Microsoft.SharePoint.WebPartPages.WebPart newWp;

                if (webpart is ListViewWebPart)
                    newWp = UpdateListViewWebPart(webpart as ListViewWebPart, origFile.GetWeb(), destFile.GetWeb());
                else
                    newWp = webpart;

                destWpm.AddWebPart(newWp, Common.ConvertToString(newWp.ZoneID), newWp.ZoneIndex);
            }

            try
            {
                destFile.Publish("Added Web Parts");
                destFile.Approve("Web Part additions approved");
            }
            catch { }
        }

        private static ListViewWebPart UpdateListViewWebPart(ListViewWebPart webpart, SPWeb origWeb, SPWeb destWeb)
        {
            ListViewWebPart newWebPart = webpart; //yeah I know.

            string oldListName = webpart.ListName;
            string oldViewGuid = webpart.ViewGuid;

            SPList OrigList = origWeb.Lists.GetList(new Guid(oldListName), true);
            
            SPList DestList = destWeb.Lists[OrigList.Title];

            SPView OrigView = OrigList.GetView(new Guid(oldViewGuid));
            SPView DestView = DestList.Views.Add(string.Empty, OrigView.ViewFields.ToStringCollection(), OrigView.Query,
                OrigView.RowLimit, OrigView.Paged, OrigView.DefaultView);
            DestView.Hidden = OrigView.Hidden;
            DestView.Scope = OrigView.Scope;
            DestView.ApplyStyle(destWeb.ViewStyles.StyleByID(Convert.ToInt32(OrigView.StyleID)));

            DestView.Update();

            newWebPart.ViewGuid = DestView.ID.ToString("B").ToUpper();
            newWebPart.ListName = destWeb.Lists[OrigList.Title].ID.ToString("B").ToUpper();


            return newWebPart;
        }

And against all odds it worked, I hope I don't have to deal with this webparts for a while.

SharePoint and New Relic's RPM

I have just installed, in one of my demo servers, a copy of New Relic's RPM just to see by myself how it works.

Unbelievably it has installed without issues and it has started to work not a very common thing in the world of SharePoint.

After installing it,the free version, of course, they upgraded my licence to a gold one for 8 days to give me time to test the product in all its plenitude so a couple of minutes after deploying in my server I had a whole lot of statistics and graphs showing me once more that my web services son lentos and the response times of MOSS just after it wakes up are not the best in the world.

I have really liked the charts about the system performance. Never before was so easy for an employee to prove the server slow and that it should be improved and where (SQL, RAM or CPU they forgot about the HD)

To whom may be still dubious about trying it I can tell that (in my environment) SharePoint, the web services, the reporting services and anything else is working perfectly, or at least as perfectly as it was before the installation I've just found an error in a report thanks to RPM, the perfect excuse to bother the BI guy

Well, as I am more into complaining than into praising I'll stop talking about this product.

In other time I've had huge public servers under my boot, but now I see myself clicking fast and repeatedly to fake server traffic...

The less hair, the less power, it has always been that way.