Using CSOM with PowerShell, some things I came across

Having done some work with CSOM and PowerShell, I have come across some interesting things. Here I will describe some of the things I think that might be interesting for others.

Working with the PowerShell ISE

So probably you would eventually want to debug you PowerShell and CSOM code at some point in time, the tool I prefer for developing and debugging PowerShell is the ISE (Integrated Scripting Environment) that comes with PowerShell/Windows. Others might prefer PowerGUI from Dell. I have used both, but PowerGUI loads a bit slowly, so I (slightly) prefer the ISE. But using the ISE I have found that debugging CSOM PowerShell sometimes the ISE messes up things. I noticed that my object was null after an ExecuteQuery. After checking the code a few times, I concluded that there was nothing wrong with the code. Then using Fiddler I found that the ExecuteQuery did return the data I was expecting. So that's when I knew that the ISE in debug mode had messed up things (this doesn't occur with stepping through every ExecuteQuery). Setting the breakpoint after the offending ExecuteQuery fixed things.

The thing to learn from this, is that when you're using the ISE to debug PowerShell CSOM code, that you sometimes will have to change your breakpoints when these unexpected issues occur. Firstly setting breakpoint to check the code upto the 'offending' ExecuteQuery to check this code, and the next run setting the breakpoint after the ExecuteQuery, letting the script run straight to this point, to check the rest of the code.

Not able to use includes/lambda expressions in the Load method

As you can't use lambda expressions in PowerShell, you can't use statements like $context.Load($web.Lists, [include properties]). Most of the time this shouldn't be an issue, and the properties will be loaded in PowerShell, but just sometimes you would really want this. I came across such a situation when checking if a webpart with a specific title was already provisioned on the page. There I needed the title of each webpart from the LimitedWebPartManager object WebPartDefinitions collection. So normally in C# CSOM you would do something like:

var lwpm = file.GetLimitedWebPartManager(PersonalizationScope.Shared);
// loads only the title, use w => w.WebPart to load all properties of the .WebPart object
context.Load(lwpm.WebParts, l => l.Include(w => w.WebPart.Title));
context.ExecuteQuery();

// Or you can do a direct check with lambda expression, and after ExecuteQuery check if lwpm.WebParts.Count != 0
// This does not load the .WebPart object, not even the title
// context.Load(lwpm.WebParts, l => l.Where(w => w.WebPart.Title == "Documents"));
    

Then you can check if there is a webpart with the specified title. In PowerShell you don't have this option, but as a work-a-round I have used the power of PowerShell to get the webpart objects in the collection, this will also include the title. See code below.

$lwpm = $file.GetLimitedWebPartManager([Microsoft.SharePoint.Client.WebParts.PersonalizationScope]::Shared)
$context.Load($lwpm.WebParts)
$context.ExecuteQuery()
$lwpm.WebParts | ForEach-Object { $context.Load($_.WebPart) }
$context.ExecuteQuery()
    

This is somewhat less efficient, as you need two ExecuteQuery calls, and retrieve more data, but still this is very readable and understandable code, and it gets the job done.

By the way, I use a somewhat similar solution for selecting a specific list, where in C# you could use:

context.Load(rootWeb.Lists, col => col.Where(l => l.Title == "Documents"));
context.ExecuteQuery();
var list = rootWeb.Lists.FirstOrDefault();
    

Then in PowerShell I use:

$context.Load($rootWeb.Lists)
$context.ExecuteQuery()
$list = $rootWeb.Lists | Where-Object { $_.Title -eq "Documents" }
    

For the purists, I did find a great post by Gary Lapointe: Loading Specific Values Using Lambda Expressions and the SharePoint CSOM API with Windows PowerShell. He made the effort to do all the underlying plumbing to break down the lambda expressions into plain .Net code to enable loading specific properties using PowerShell.

So these are the things I thought might interest you. If I come across other interesting bits specific to PowerShell with CSOM, I will add them to this page.

Using ExecuteQueryWithIncrementalRetry with PowerShell CSOM

Today I was writing a CSOM PowerShell script that could possibly have an impact on the throttling in SharePoint, so I looked for a port of the OfficeDev PnP Core.Throttling implementation of ExecuteQueryWithIncrementalRetry. As I couldn't find one, I created a port myself. Just to be on the save side with my script, as for now I did not experience any throttling issues. Below is the code for the PowerShell port:

# Port of the ExecuteQueryWithIncrementalRetry from the OfficeDev PnP 
function ExecuteQueryWithIncrementalRetry([Microsoft.SharePoint.Client.ClientContext] $context, $retryCount, $delay)
{
    if ($retryCount -eq $null) { $retryCount = 5 } # default to 5
    if ($delay -eq $null) { $delay = 500 } # default to 500
    $retryAttempts = 0
    $backoffInterval = $delay
    if ($retryCount -le 0)
    {
        throw New-Object ArgumentException("Provide a retry count greater than zero.")
    }

    if ($delay -le 0)
    {
        throw New-Object ArgumentException("Provide a delay greater than zero.")
    }

    # Do while retry attempt is less than retry count
    while ($retryAttempts -lt $retryCount)
    {
        try
        {
            $context.ExecuteQuery()
            return
        }
        catch [System.Net.WebException]
        {
            $response = [System.Net.HttpWebResponse]$_.Exception.Response
            # Check if request was throttled - http status code 429
            # Check is request failed due to server unavailable - http status code 503
            if ($response -ne $null -and ($response.StatusCode -eq 429 -or $response.StatusCode -eq 503))
            {
                # Output status to console. Should be changed as Debug.WriteLine for production usage.
                Write-Host "CSOM request frequency exceeded usage limits. Sleeping for $backoffInterval seconds before retrying."

                # Add delay for retry
                Start-Sleep -m $backoffInterval

                # Add to retry count and increase delay.
                $retryAttempts++
                $backoffInterval = $backoffInterval * 2
            }
            else
            {
                throw
            }
        }
    }
    throw New-Object Exception("Maximum retry attempts $retryCount, has be attempted.")
}
    

You can just add this to your PowerShell script, and use this like:

ExecuteQueryWithIncrementalRetry $context
        
# or with specifying a different retryCount and delay:

ExecuteQueryWithIncrementalRetry $context 8 800
    

One other PowerShell thing that I should have mentioned earlier, is the availability of the PowerShell Tools for Visual Studio 2015, also available for VS2012 and VS2013, created by Adam R Driscoll. This tool also allows for debugging. Still, I do tend to use the ISE most often, but it is an other great tool for your toolbox.