Restore default Site Columns and Content Types in SharePoint Online / Office365

Through my tenure as a SharePoint Administrator focusing on SharePoint migrations, I feel like I somehow run into a new and unique issue each time I have a new engagement. For this specific issue, I am migrating my client from an on-premise SharePoint 2007 environment to an ADFS enabled SharePoint Online / Office365 tenant. This is obviously a big jump, and is going to offer a lot of great new features for my client. As an end-user, this migration will be awesome, but as an administrator, I cringe at the thought of performing a jump straight from 2007 to SPO.

In order to make things work, we had to leverage a migration tool – specifically Metalogix Content Matrix. We used this tool to migrate mostly content-only from the source to the target. We skipped all custom InfoPath forms, workflows, etc. to prevent any conflicts or degraded functionality that may be replaced by upgraded modules.

Using Content Matrix to perform this content-only migration, we found that Site Columns associated with Custom InfoPath forms were still being migrated to the target. To make it better, my client had created some InfoPath forms with friendly names which were the same as system default Content Type Site Columns. Content Matrix addressed this by migrating the custom InfoPath form Column, and having this column overwrite the system default. In our case specifically, we saw the First Name and E-Mail Address fields (seen often in Contact List lists) to not be viewable. The column would show in List Settings, but you could not select it to review the column settings or migrate the content into these disabled columns.

This placed a need on how to recreate the out of the box Site Columns and Content Types to restore them to what they would be on a fresh Site Collection. With this being SharePoint Online, we have many more limitations on how to do this, but I was able to find a fix.

Using the SharePoint Online Client Browser (HERE), I was able to review all Site Collections Features (including the specific hidden features). The features I was looking for were:

  • Standard Column Definitions
  • Standard Content Type Definitions

Once finding the GUIDs for the above features, I was able to use the manage-features.ps1 script for SharePoint Online (HERE). I ran through and disabled both of the above features, and then re-enabled them, and the Site Columns were restored and functionality of the affected lists were restored.


SharePoint 2007 File Inventory

Once again, I am reminded of how much I can appreciate the PowerShell snap-ins for SharePoint after the 2010 release. Attempting to iterate through a farm or script out actions using only STSADM commands and direct access to the SharePoint assemblies ([System.Reflection.Assembly]::LoadWithPartialName(“Microsoft.SharePoint”)) is much more challenging than using the built-in PowerShell capabilities of the later versions (even SharePoint Online with it’s own quirks).

I have a client who is migrating from an on-premise SharePoint 2007 environment to SharePoint Online. Yes – big jump, and yes – there will be some headaches. So many features and functions of SharePoint 2007 simply will not migrate and/or be mappable to equivalent SharePoint Online features due to the sheer difference in the way each of the platforms are laid out. Due to this, identifying some of these issues proactively before a migration rather than reacting to broken page layouts, missing site feature dependencies, and other prerequisites is greatly beneficial. To make matters a step more complicated, the client had elected to engage in a branding effort to give their SharePoint Online instance a fresh new look that resulted in a site that really didn’t look like SharePoint at all in the end.

So how do you migrate a bunch of SharePoint 2007 ASPX pages, based on deprecated site templates and layouts, into a branded SharePoint Online instance which utilizes a specific page layout for branding and the site home pages?

You don’t.

Lucky this client didn’t have very customized site home pages or a large extent of customized pages throughout their farm, so efforts to re-create these pages into branded SPO pages manually was feasible. In order to prepare for this work and delegate out this ASPX page recreation to site owners, the client wanted to have an inventory of all ASPX pages in their farm currently, with location. Per my normal consulting habits, I was confident in telling the client we could figure out a way to give them this report to help them plan this work out, yet I was not 100% sure how I would do it at that time. Now was time to search for how to get a SharePoint 2007 file inventory, and then strip out everything besides the ASPX pages from the list.

Through some research online, it was easy to find all arrows pointing to Gary LaPointe. The SharePoint MVP is THE man when it comes to STSADM commands. His website has WSP solutions for installing snap-ins with SharePoint 2010+ PowerShell commands, as well as some scripts here. Lucky for me, he had one script on his website which iterated through every web application in the local farm, and drilled down through each Site Collection, subsite, and document library recursively and output the results to either a grid view or a CSV file. View the script below, as well as a link to Gary’s personal blog which I would highly recommend to any SharePoint consultant or legacy farm administrator

View the script on his site HERE, as well as the downloads page which features a large library of tools and scripts for completing unique tasks that aren’t quite perfectly mapped out in OoB cmdlets.

SharePoint 2010 vs PowerShell 3.0 CLR

Microsoft SharePoint 2010 is not supported with version 4.0+ of the Microsoft .NET Framework. This issue can come into play if you happen to install the Windows Management Framework 3.0+, .NET Framework 4.0+, or PowerShell v3.0. All of these items run on a CLR (Common Language Runtime) of 4.0+, the default that comes with these applications. When loading the SharePoint 2010 Management Shell, you may see one of the following:

The local farm is not accessible. Cmdlets with FeatureDependencyID are not registered


Microsoft SharePoint is not supported with version 4.0.x

To address this, there are a couple of workarounds:

Force the SharePoint Management Shell to use PowerShell Version 2

Forcing the shell to run on PSv2 is a quick workaround that requires no downtime, restarts, or interruptions in services.

    1.  Right click the icon for the SharePoint Management Shell, click Properties
    2. Click the shortcut tab in the properties window
    3. Input the following into the target value:

This will force the Shell to open in PowerShell v2.0, running the CLR 2.0.X. To verify this, open the SharePoint Management Shell, and type $psversiontable. This will show you the version of CLR and PowerShell that the current session is utilizing.

Force PowerShell to globally use a different CLR

  1. Create or edit a file at $pshome\powershell.exe.config
  2. Insert or edit the contents to include the line for the CLR 2.0.X support, ensuring that this is before the 4.0 callout (the order of the supported run times shows how the system will favor each).
  3. Save the file and reopen any previous PowerShell sessions
  4. Test this by opening PowerShell and running the $psversiontable command

Azure ACS Namespace authentication with SharePoint 2013


This document will be used to outline the process for implementing Azure’s ACS namespace as a trusted identity provider to an on-premise SharePoint environment, utilizing external identity providers such as Facebook, Yahoo, or Windows Live ID to pass manipulated claims as a standardized trusted token to SharePoint

Preface, Considerations, Prerequisites

  • Azure ACS authentication with SharePoint is a free service as of 12/2014. The service requires a subscription to Azure, but costs no money. A $0 spending limit can be set to prevent the possibility of any charges coming through from this account.
  • The Azure ACS system can utilize any Identity Provider that can export an OpenID claim
  • SharePoint 2013 does not support SAML 2.0, SAML 1.1 must be used
  • As of 12/2014, Windows Live ID can only export a nameidentifier claim, but this can be mapped to an email address claim to use as a UPN for login
  • An x.509 SSL certificate needs to be uploaded to Azure to sign tokens, and to explicitly trust in SharePoint


Azure ACS – ACS (access control system) is a free feature of Azure that allows the developer to create a namespace that can tie external Identity Providers incoming tokens to a single claim to a web application
Azure Namespace – the item in Azure that provides the URL for all Identity Providers, Relying Party Applications, Rule Group Settings, and Certificates and Keys to be trusted
Identity Provider – an external service, such as Yahoo!, Facebook, or Windows Live ID, that provides the inherently trusted authentication to SharePoint
Relying Party Application – these are the web applications that are connected to the namespace that receive the token that is output from the Azure ACS namespace. In this walkthrough, SharePoint 2013 is our Relying Party Application
Rule Groups – Rule groups are the list of incoming claims tied to claim issuers/identity providers, how they may be mapped when exported, as well as any customizations that need to happen for the token itself

Azure Setup

Creating the Namespace
  1. Login to the Azure management portal at https://manage.windowsazure.com with the Azure account created prior to this work. Note: Ensure that this account has a subscription set up prior to continuing any further, any work created with a trial account will expire once the trial is over, and are unable to be transferred to a setup account
  2. Once logged into the management console, click the large NEW button at the bottom left of the screen to begin the process of creating a namespace
  3. Azure1

  4. Navigate App Services > Active Directory > Access Control > Quick Create:
  5. Azure2

  6. Give the namespace a name that makes sense for the authentication set up (this won’t be an actively used URL by an end-user). Select your region, and the subscription that has been set up (Pay-As-You-Go, or another valid subscription, NOT free trial). The namespace status will change from creating to active:
  7. Azure3

  8. Click on the newly created namespace, and click the manage button on the bottom ribbon, this will take you to the Silverlight based Windows Azure Platform Access Control Service control panel:
  9. Azure4

  10. Utilize the Getting Started steps that appear on the homepage of this control panel for much of the work. You will now want to set up the Identity Providers:
Making Windows Live ID Work for SharePoint 2013 Authentication

Since the Windows Live ID Identity Provider is a preset default in the Azure ACS namespace, I will use it as a full example on configuring the claim issuer for authentication in SharePoint, and will later lead into how to configure additional Identity Providers.

Configure Relying Party Applications
  1. Navigate to the Relying party applications management by clicking the URL on the left-hand side of the ACS administrative panel
  2. Click the Add URL at the top left of the Relying Party Applications table
  3. On the configuration page, enter in the following information:
    Name: a display name such as “SharePoint2013” is fine for this as it is simply a display name for SharePoint’s trust with ACS
    Realm: the realm will be equal the resolvable url for your SharePoint web application (https://winchestertonfieldville.ia.gov/) would work fine if this is your SharePoint web application URL
    Return URL: this is the URL in which ACS will return tokens to SharePoint. This will always be the same URL as the realm defined above, with “/_trust” appended to the end (https://winchestertonfieldville.ia.gov/_trust)
    Error URL (optional): This is the URL for an optional error page in case authentication fails
    Token Format: Choose SAML 1.1, SharePoint 2013 is NOT compatible with SAML 2.0
    Token lifetime (secs): This value represents the seconds that the token is valid before expiring, or how long the user will be able to hit the realm URL repeat times and be auto-authenticated back in to the environment. The default value of 600 seconds is typically on the low side. Set this value to 3600 seconds (equal to the default value in ADFS).
    Identity Providers: the list of Identity Providers that you would like to have as optional authentication methods when users hit the realm’s login page. Each Identity Provider will show in the login drop down
    Rule Groups: leave the check box to create a new rule group as we will be doing this in the next step
    Token Signing: Choose whether to Use service namespace certificate or Use a dedicated certificate. For the purposes of this walk through, I would recommend choosing Use a dedicated server and upload the X.509 certificate .pxf file to use for when signing. Browse to the file, upload, and input the password that was created when it was exported from the server.
  4. Click Save
Configure Rule Groups

The rule groups are the configuration for what claims Azure will receive from the Identity Providers, as well as how they will map these claims to the output token.

  1. Navigate to the Rule Groups page by clicking the URL on the left-hand navigation
  2. Click the “Default Rule Group for <RelyingPartyApplication>” that was automatically created when we set up the Relying Party Application
  3. On the Edit Rule Group page, feel free to edit the name for the rule group
  4. In the Rules table, click the Generate URL. You will notice that the Rules table currently shows “No rules have been added. Click Generate to generate rules automatically, or click Add to add rules manually.” Note – If additional Identity Providers are added at a later time, use this option to generate new rules for the new identity providers
  5. On the following page, click the checkbox next to Windows Live ID, and click Generate
  6. You will then see the that the nameidentifier output claim from Windows Live ID is listed as a rule. Click on the nameidentifier URL.
  7. On the resulting page, you can see how to configure the input/ouput of this claim for how ACS will prep it for the output token that will be trusted by SharePoint. By default, nameidentifier is not a usable claim for a login in SharePoint. To address this, we will map the output claim type to email address.
  8. For everything under the If section, leave default
  9. For the “Then” section, change the select type dropdown from http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier to http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress to map the claim from the nameidentifier URI to a usable email address login
  10. You can leave all other items as default, or add information under “Rule Information” that will explain that we are mapping the Windows Live ID nameidentifier claim to an emailaddress output claim
Upload Token Signing Certificate

Azure ACS uses an X.509 certificate to digitally sign the tokens it generates. You can also increase the functionality of this certificate to include encryption/decryption of the tokens themselves. Many users will use the makecert utility to create a self-signed certificate for testing purposes, but do not do this in a production environment.

  1. Click the Certificates and Keys URL on the lefthand side of the Azure ACS configuration page
  2. Click the Add URL at the top of the Token Signing table
  3. On the “Add Token-Signing Certificate or Key” page, select the correct Relying Party Application in the dropdown.
  4. Select X.509 Certificate for type
  5. Click Choose file and upload the .pfx file. Note: On this page, it also shows how to use the MakeCert utility from the Microsoft SDK to make this certificate.
  6. Click Save

SharePoint 2013 Configuration

Within SharePoint, we will move forward in this walkthrough with the assumption that the Web Application within SharePoint 2013 has already been created with a friendly URL assigned to it.

  1. Log in to the Central Admin server
  2. Open a PowerShell ISE session as an administrator and run the following:
  3. I would recommend running the above code line by line to ensure you can identify any errors as they happen, rather than an error that you can’t attribute to a single line of code when bulk running the block
  4. Open Central Administration and click Manage Web Applications
  5. Click the web application that we have been using thus far, and click Authentication Providers in the ribbon.
  6. In the appropriate zone, you should be able to see empty check boxes for the newly created Trusted Identity Providers that we have set up. Check the boxes next to Windows Live ID
  7. Press Save
  8. SharePoint now knows to trust any SAML1.1 tokens that come from the specific Azure ACS namespace that we have created, and knows how to parse the claims within the token.

You are now complete! You now should be able to hit the SharePoint Web Application URL, be redirected to an Azure ACS hosted login page, be able to choose between windows authentication or Windows Live ID, be redirected to the Windows Live ID login page to authenticate, and automatically be pushed back to SharePoint. As of our current configuration performed thus far, the users will hit a “You are not authorized to view this page” SharePoint page. For security, I would recommend enabling Access Request settings for the root Site Collection, and setting the recipient to a HA individual, or a distribution group to be able to triage requests from possible users as they hit your environment.

Powershell in MOSS 2007 – Site Maps

So you have a client that’s running MOSS 2007. With the features and benefits of SharePoint 2013, it’s easy to get quickly frustrated with this old version of SharePoint. Nothing irks me more than forgetting about the lack of PowerShell, each and every time, and spending time looking up STSADM equivalent cmdlets to achieve the same task.

Isn’t there some way I can PowerShell SharePoint 2007?

Yes, but it kind of sucks. I would always recommend using STSADM commands natively when possible, but there really isn’t a good alternative to PowerShell sometimes. Windows SharePoint Services 3.0 and SharePoint Server 2007 do no include built-in cmdlets, but we can do the following if PowerShell is at least installed on the server:

1. Run the following to set the execution policy to allow for scripts to be run locally (this may or may not be reversed/denied via group policy:

2. Load the SharePoint API into you script with the following:

3. Load the MOSS 2007 Script Collection from CodePlex onto the machine HERE

4. Create a new PS1 file with the following format:

The above will run the two referenced PS1 files and keep them in memory preemptively to calling the cmdlets in the script later, and when they are, the console knows where to look for them.

The following can be done via native STSADM to create the same type of output (Note: the enumallwebs command is an STSADM command that is only available in SP ’07 SP2 and above):

Both of these options clearly will work, but the output will be different and some may prefer to stick to STSADM when possible in the older environments.

In the end, this is simply a workaround that allows you to run a small sub-set of your PowerShell scripts that you know and love from SP’13 and SP’10 over a MOSS2007 farm when STSADM commands just fall short.

The Magic of Powershell – Email Spoofing

The magic of Powershell.

Something I’m learning more and more about as I spend more time in the IT world. Prior to my first technology job that I am in now, I had never even heard of this witchcraft. Automate everything? Sounds interesting.

But there’s also plenty of black, dark magic, and witchcraft that this Powershell stuff is capable of performing. A coworker of mine is somewhat of a Powershell enthusiast around the office. He is also a gremlin. Powershell pranks around the office have gained quite the reputation.

Leave your computer unlocked when you go to the bathroom? Expect to return to a laptop that will randomly speak dirty phrases to you randomly. Hasslehoff’d (yes it’s a verb) desktop backgrounds, spoof’d emails, and extra mouse dongles in your computer are quite the norm.

The extent to which I have been able to work with Powershell thus far in my career has been limited to Server 2012 commands – boring in comparison to it’s full capabilities. Today, I did have my first chance to exercise its capabilities.

While setting up this blog, another coworker of mine decided to exploit my lack of WordPress knowledge by installing a backdoor snippet of code so that he could have an admin log in and create users and edit my page. Very funny for all but me. Every time I made a change to my blog, he would change the theme, posts, add users, etc. I finally got past it but was still annoyed.

Introduce the Send-MailMessage cmdlet. This fun cmdlet enables a user to send emails to an email, and spoof it from another. All it takes to utilize this is knowledge of the mail server utilized by your domain’s email address.

Below is the snippet for how to use this cmdlet:


The above will send an email to georgesmith@company.com from georgesboss@company.com with the stated email subject line and body. This email will show up in George Smith’s email inbox appearing as if it came from his boss (sans signature). George would have no idea that this email didn’t come from his boss unless viewing the email properties which would show the original computer that sent the email.

Note: Spoofing emails has questionable ethical and moral implications when used in malicious ways. Use common sense for what is okay and always remember that even though IT may give you permissions to do things such as this, it doesn’t mean it’s always okay.