The blog of Wictor Wilén

  • The SharePoint Team is listening - make your voice heard

    Tags: SharePoint 2013, Office 365, Microsoft

    There’s a lot of stuff happening right now at Microsoft, they innovate, create great software and services, the new CEO accepts and wins almost all challenges and the SharePoint and Office team is listening! This is the Microsoft that I like and this is how I want Microsoft to continue to be. But Microsoft and the SharePoint team can’t just listen in blind – they listens to us out here in the real world, customers, clients etc. and we need to make our voice heard. This can be done in several ways, we can talk to our Microsoft representatives, we can whine on our blogs and on social networks OR we could make ourselves heard at UserVoice.

    Microsoft and different divisions and groups within Microsoft has started to use UserVoice pretty extensively lately. UserVoice is a great service where you can set up your own channel, listen for feedback and questions, and answer them and most importantly act upon them. Just recently the SharePoint team had a blog post called “UserVoice driving improvements to SharePoint API” which shows just this. From the feedback they received on UserVoice and with direct and indirect customer contact they’ve made some pretty significant improvements to the SharePoint API’s, such as JSON Light support and others. I really dig this!

    FeedbackSo, if you have a suggestion or improvement to Office, SharePoint and/or the Office 365 service then get your sorry behind over to UserVoice and make yourself heard. Read other peoples suggestions and vote on them. The more votes, the more likely the teams will pick up it.

    Here are some of the UserVoice channels that Microsoft and the different product groups use:

     

    One suggestion that I made the other day was a suggestion that the Office/SharePoint/Office 365 team has a publically posted “change log”. I would like them to post any changes done to the API, UI, CSS, JavaScript, PowerShell etc in a chronological change log, so we don’t have to read between the lines in KB articles etc. If you think that this is a great idea, then I invite you to vote on my suggestion here: http://officespdev.uservoice.com/forums/224641-general/suggestions/6297224-sharepoint-and-sharepoint-online-change-log

  • SharePoint 2013: Using the App Only policy and App Principals instead of username and password combos

    Tags: SharePoint 2013, Apps

    Introduction

    One of the blog post I receive the most e-mails about is the How to do active authentication to Office 365 and SharePoint Online. Some of the feedback I get is “thank you” etc. and some of them are people that are modifying it for all kind of things, especially when trying to do “background jobs” or integrations using user credentials. And this is what this post is all about. My standard answer to that question is that you do not want to use the methods in that post for those scenarios, nowadays, since you can actually build an App for that. And even better you can build an App with a so called App Only policy – which means that we do not need to have any user credentials or anything like that. So in order to save me some time in replying that and then another reply on the “how do I do that?” question I can from now on only link to this post :-)

    Scenario

    Before we dig down to the actual nuts and bolts, let’s have a scenario in front of us. Assume we have a SharePoint site containing a product list (or similar). We would like an application/app/service/background job (you name it) to read and potentially write to this list. This is a very common scenario and traditionally this was built as timer jobs in SharePoint, or scheduled tasks running as specific users or even with hardcoded username and passwords (yikes!). In this sample we will instead use an App Principal, this is very much like a user principal which you can give permissions to in SharePoint.

    In this simple example I will use a site containing a list of products (two columns; Title and Price) and then an ASP.NET MVC web site that will show that list of products. An no, I will not build a SharePoint App, but instead show how you can take advantage of App Principals to get the products from SharePoint without using any user credentials whatsoever.

    Create the App Principal

    Register an App The first thing we would like to do once we have created our site and the list of products is to generate an App Principal with access to read from this list of products. This is done by navigating to your site and then appending /_layouts/15/AppRegNew.aspx to the URL so that you end up on the “Register an App” page. On that page click on the two Generate buttons to generate an App Id and an App Secret, then provide a Title for your App and the domain (localhost in this demo) for your app.

    Once that is done you will get a confirmation and a summary of the app identifier created. Screenshot this and/or copy and paste it so that you have the details, you will need them later.

    Give the app some permissonsNext we need to grant permissions to this app. This is done by navigating to /_layouts/15/AppInv.aspx. On that page paste your App Id (generated previously) into the App Id text box and click on the Lookup button, this will retrieve the details of the app and give us the opportunity to give the app some permissions.

    Giving the app permissions is not for the faint hearted and the UI is horrible (just like basically any admin UI in SharePoint). You need to write a valid app permission request XML. In this case I only want read rights to a specific list, nothing more, and I also want the app to be able to retrieve this data without any user credentials (App Only). The resulting XML looks like below, and the key thing is the AllowAppOnlyPolicy attribute:

    <AppPermissionRequests AllowAppOnlyPolicy="true">
      <AppPermissionRequest
        Scope="http://sharepoint/content/sitecollection/web/list"
        Right="Read"/>
    </AppPermissionRequests>

    When you click the Create button you will be taken to the “Do you trust” page, where you should trust your app. Since we requested to be able read from a list, we must also choose which list that we should read from. Once we’ve done that we click “Trust it”. One crappy thing is that if you would like to modify the permissions later, you need to re-write the XML completely since the UI does not retrieve the current requested permissions, so you need to document and save your XML.

    Trust the app!

    If you don’t want to study the app permission request schema I recommend you to look at this MSDN page which will give you some more insights.

    (Optional) High-trust apps for on-premises usage

    If you’re now doing this for on-premises deployments and do not have configured your farm for low-trust apps (integration with ACS), then you also need to register a high-trust app. To not repeat myself, you can use the same set of PowerShell that I used in the SharePoint 2013 with SAML Claims and Provider Hosted Apps blog post. Just make sure that you generate a new Issuer Id (which is not the same as the Client Id you got above when registering the app.

    Building the ASP.NET MVC site

    Now it is time to start building the consumer of this products list, in this case an ASP.NET MVC site (but it could be basically anything). Fire up Visual Studio 2013, create a new ASP.NET Web Application, then choose MVC and configure it as you like.

    App for SharePoint Web Toolkit NuGet packageThe first thing we need to do is to download a very sweet NuGet package. It is called “App for SharePoint Web Toolkit”. This “little” package will install the SharePoint 2013 client assemblies and also add the TokenHelper.cs and SharePointContext.cs files that you should be familiar with from a normal Provider Hosted SharePoint App.

    Update the web.config

    Before we start writing the code to retrieve the products from the SharePoint list we need to add some plumbing into the web.config. Depending on if you’re using ACS or using the High-Trust approach you need to add different configuration. For a low-trust approach you only need the first two lines (ClientId and ClientSecret that you got from AppRegNew.aspx) and for the high-trust approach you need all settings below, except the ClientSecret, and you also need to have the pfx file of the certificate used for the high-trust configuration.

    <add key="ClientId" value="d0ee6f6d-1c39-44b2-a284-f3143e820ec2"/>
    <add key="ClientSecret" value="11Flt0LME9mnToNhRN5nmgeynAXW3nMBvJBrIBGlT/o="/>
    
    <add key="ClientSigningCertificatePath" value="C:\temp\AppOnlySite.pfx" />
    <add key="ClientSigningCertificatePassword" value="Pass@word1" />
    <add key="IssuerId" value="93d08816-eb35-4ed9-be90-ea352cc81130" />
    

    Modifying the Home Controller

    To read the stuff from our SharePoint list I modify the Index action of the Home Controller so that it looks like this:

    public async Task<ActionResult> Index()
    {            
        Uri site = new Uri("https://intranet.sp01.contoso.com/sites/productlist");
        string realm = TokenHelper.GetRealmFromTargetUrl(site);
    
        // For use with ACS/SPO
        //var tokenResponse = TokenHelper.GetAppOnlyAccessToken(TokenHelper.SharePointPrincipal, site.Authority, realm);
        //var accessToken = tokenResponse.AccessToken;
                
        // High Trust
        var accessToken = TokenHelper.GetS2SAccessTokenWithWindowsIdentity(site, null);
                
        UriBuilder builder = new UriBuilder(site);
        builder.Path += "/_api/lists/GetByTitle('Products')/Items";
        builder.Query = "$select=Title,Price";
    
        HttpClient client = new HttpClient();
        client.DefaultRequestHeaders.Add("Accept", "application/json;odata=verbose");
        client.DefaultRequestHeaders.Add("Authorization", "Bearer " + accessToken);
    
        var response = await client.GetAsync(builder.Uri);
    
        var result = JsonConvert.DeserializeObject<ProductResponse>(await response.Content.ReadAsStringAsync());
    
        return View(result.Data.Results);
    }

    The first thing I do is to modify the method signature (#1) and declare it with the async keyword, so that I later can use the await keyword for the asynchronous operations. Then I use the TokenHelper class to retrieve the realm (#4)and the access token (#8 || #11). As you can see there are two different ways to retrieve the access token for an App Only principal, one when using ACS and one when using the High-Trust approach.

    Once I have the access token I build the REST URL (#13-15) and then create the HttpClient that I will use to invoke the REST end-point. On line #18 I specify that I want the data back in JSON format and on line #19 I pass the access token for authorization.

    Once that is all done we only need to invoke our HTTP client and retrieve the result. Once we have the result I use the JsonConverter to deserialize the data into a nice object structure (ProductResponse). In order to more easily work with the data returned I created these POCOs:

    public class ProductResponse
    {
        [JsonProperty("d")]
        public ProductResult Data { get; set; }
    }
    public class ProductResult
    {
        [JsonProperty("results")]
        public IEnumerable<Product> Results { get; set; }
    }
    public class Product
    {
        [JsonProperty("Title")]
        public string Title { get; set; }
        [JsonProperty("Price")]
        public float Price { get; set; }
    }

    The JsonConverter converts my JSON data into this object structure and I can easily then retrieve the actual Product results and return it my View.

    Modifying the View

    To display the set of products I use the built-in WebGrid in my Index View:

    <div class="row">
        <div class="col-md-4">
        @{
            WebGrid grid = new WebGrid(this.Model);
            @grid.GetHtml();
        }  
        </div>    
    </div>

    This will result in a grid view of my SharePoint products list like this:

    Wow, this dude has some serious UI skillz

    There you have it! This example can easily be extended with write operations or other stuff you would like to do, as long as you can express the requested permissions using the permission request XML.

    Summary

    In this post you have seen how we can leverage SharePoint data in other applications without using any user credentials and/or saved passwords. It requires minimal effort, especially when you’re using SharePoint Online (or ACS on-premises) since you only require Site Collection permissions to create the App Principal. All this thanks to the App Only policy.

  • Microsoft Azure IAAS and SharePoint 2013 tips and tricks

    Tags: Microsoft Azure, IAAS, SharePoint 2013, SQL Server

    After doing the Microsoft Cloud Show interview with Andrew Connell I thought it might be a good idea to write some of my tips and tricks for running SharePoint 2013 on Azure IAAS. Some of the stuff in this post are discussed in more depth in the interview and some things we just didn’t have time to talk about (or I forgot). I really recommend you to listen to the podcast as well and not just read this post.

    Disks, disks and disks

    As mentioned on the Microsoft Cloud Show interview more than once, one of the first things you should look into is your disk configuration for your Azure VM’s.

    Use a lot of disks

    One of the first things you must look into is the performance of disks. As you should be aware of SharePoint and SQL requires fast disks to operate with decent performance. When running an on-premises installation (virtual or physical) you have almost full control of the disk performance – you can choose from fast spinning disks to SSDs to different RAID configurations. But you cannot do that in Microsoft Azure. The virtual disks in Azure uses the blob storage for storing the VHD files, as blobs. Disk performance are often measured in IOPS (input/output operations per second) and in with the VHD files in the Azure blob storage we are limited to 500 IOPS per disk. Also worth noticing is that if you run the VM using the Basic tier offering, the limit is 300 IOPS per disk.

    TechNet contains a couple of article about calculating IOPS and the need for IOPS. For instance in the article called “Storage and SQL Server capacity planning and configuration (SharePoint Server 2013)” we can see that a content database requires up to 0.5 IOPS per GB. If we translate that into an Azure VHD we see that we can only have about 5 content databases per disk, assuming that they grow to 200GB each (a total of 1.000 GB). The Search Service Application, specifically the crawl and link databases has high IOPS requirements, so they should be on dedicated disks and so on. This is exactly the same as on any SharePoint installation – in the cloud or on your own metal. But the hard limit on 500 IOPS per VHD makes it even more obvious, given that we do not have a choice.

    So, when deciding which machine you are going to use for SharePoint and SQL, choose one with a lot of disks. For instance choose A4/A7 to get 16 disks or A3/A6 to get 8 disks.

    Read and Write caching

    A very important thing to configure for the Azure IAAS disks are the read and write caching, it is turned off by default on data disk – leave it like that!

    SQL Server Filegroups and multiple data files

    SQL Server has a concept called Filegroups which can be used to increase performance of databases. You can for a database in the primary file group add multiple data files and have them reside on different disks. (Thanks to Trevor Seward for the tip).

    Disk striping

    The maximum size of a VHD in Azure is 1TB (1.000GB), and the limitation is in the Azure blob storage. If you need larger volumes/disks in your Azure VMs then you need to stripe multiple disks into a single volume. Just add the VHDs and then use the Disk Manager to configure the disk striping.

    Disk size

    Create large disk files! You only pay for the space you actually use. If you store 1MB on a 500GB disk, you pay for 1MB. Reconfiguring disks cost you more!

    SQL specific stuff

    When running SQL on an Azure VM or any kind of virtual or physical hardware – do not forget to format the disk using 64K allocation unit size and do not forget to give the SQL Server service account “Perform Volume Maintenance Tasks” right. These two things makes a huge difference!

    Examples (and just examples nothing else!)

    For a single SQL Server this could look something like this:

    Disk/Volume Purpose
    1 OS (default)
    2 Temporary (default)
    3 SQL Server binaries
    4 Temp DB
    5 Temp DB log files
    6 Default SQL Server DBs
    7 Default SQL Server DB log files
    8 Search DBs
    9 Search DB log files
    10 Content DBs 1
    11 Content DBs 1 log files
    12 Content DBs 2
    13 Content DBs 2 log files
    14 Service Application DBs
    15 Service Application DB log files
    16 Backup 1

    And for a SharePoint machine it could look like this:

    Disk/Volume Purpose
    1 OS (default)
    2 Temporary (default)
    3 SharePoint binaries
    4 Blob cache
    5 Log files
    6 Index files
    7 ASP.NET Temporary files
    8 Tools
    9 Visual Studio (dev machine)
    10 Project files (dev machine)

    Note: do NOT store anything of value on the Temporary (default) disk, it will be wiped whenever Microsoft decides to. I would not even recommend storing SharePoint log files there (which I’ve heard recommendations of) since you might want to go back in time and search the logs eventually.

    Virtual Machine size

    Choosing the Virtual Machine size is one of the tricky questions; do you need RAM, CPU, Memory etc. “Fortunately” we do not have that much of a choice in Azure IAAS (compared to other vendors). We can choose from A0 to A9 – all the details here: “Virtual Machines Pricing Details”.

    If we’re talking about SharePoint and SQL we’re even further limited – A3 to A7 are the ones with sufficient RAM/CPU for that scenario, but we can almost exclude A5 which only has CPU two cores and only support 4 data disks. There is no “perfect machine” here, it depends. With regards to running SharePoint you might want a lot of cores for search machines etc etc. A4, A5 and A6 are good candidates for SharePoint in my opinion. A3 might do for a simple dev machine.

    Plan for High Availability

    Azure terminologyLet’s assume we’re setting up a SharePoint + SQL production environment in Azure, then you need to start to think about HA (High Availability). Part from the usual ways to do it (see my SPC 2014 presentation on the topic) there are a couple of other things that we must think of in Azure IAAS.

    Location

    Location should be fairly obvious. Do you want your stuff in North America, Asia, Northern Europe etc. It doesn’t have much with HA to do, but more about latencies and costs! Yes, if you want to be cost efficient check out the pricing in the different locations, it’s quite a difference.

    Affinity Groups

    Affinity Groups are a very important construct in Azure. Affinity Groups allows you to make sure that your cloud services, storage, virtual networks etc are placed “together”. Remember the data centers might be several football fields in size and you don’t want your machines scattered across all that space, that would only cause unwanted latencies. An Affinity Group makes sure that all your stuff in Azure are as close to each other as possible. Also this reduced our costs, since we don’t get any cross datacenter communication. An Affinity Group exists in one Location.

    Cloud Services

    “Cloud Services” are used in Azure to “group” instances together. Cloud Services have features such as end-points, load balancing and auto-scale. For SharePoint and SQL don’t even think about auto-scale though.

    It is very important to note that if you are using SQL Server Always-On Availability Groups, this SQL Server setup must be in its own Cloud Service, separate from the clients accessing it.

    Availability Sets

    Availability sets are a logical construct that allows you to specify a group of instances/roles. Whenever Microsoft must reboot one of your VMs for maintenance or other purposes, they will respect the Availability Sets and make sure that only one of the machines within an Availability Set are down at a time. For instance grouping all SharePoint Web Servers into one Availability Set, makes sense.

    An example

    Here is one example that I’ve shown when presenting this that gives you an idea of how an Azure IAAS SharePoint and SQL infrastructure might look like, once again just an example!

    SharePoint and SQL Azure IAAS example

    SQL Server optimizations

    I will not dig too deep into tips and tricks with SQL Server specifically but instead urge you to read the really good article “Performance Best Practices for SQL Server in Azure Virtual Machines”. That article gives you all the details and a nifty check list.

    Short notes

    Here are some other small things for you to remember:

    • Office Web Apps 2013/Office Online/WAC are NOT supported on Azure IAAS at the moment – hybrid with a site-to-site VPN is your way if you want WAC.
    • Always set your machine in High Performance mode .
    • If you’re using the public IP of the Cloud Service, remember to always have a machine running in that Cloud Service, otherwise the IP will change. Or use the Azure static DNS offering.
    • For development SharePoint machines and SQL – do not use the Azure SQL images, instead use your MSDN SQL Server Development Edition. If you don’t you will be billed for the SQL Server resource usage and that is even more expensive than running the actual VM.

    Summary

    That was a mouthful of tips and tricks and I hope you get something out of this. Of course there are plenty more, don’t be shy and use the comments for your best tips and tricks. I might update the post with your best tips or other things that I find. And also note, that these are in no way the official tips and tricks from Microsoft and the Azure Team, just my experience from working with it.

  • Interviewed on the Microsoft Cloud Show about Azure IAAS

    Tags: Microsoft Azure, SharePoint 2013, Interviews

    A couple of weeks back I was interviewed by Andrew Connell for the Microsoft Cloud Show. The Microsoft Cloud Show is an (almost) weekly podcast where Andrew (AC) and his wingman Chris Johnson (CJ) discusses everything related to Microsoft cloud offerings including benchmarks with other cloud vendors. If you’re not subscribing and listening to the show already then I urge you to do that as soon as possible!

    Microsoft Cloud Show

    Me and AC sat down for almost an hour discussing Microsoft Azure IAAS and specifically when running SharePoint 2013 in that service. We had a great talk, as usual when it comes to AC, and I think we covered a lot of the issues and gotchas and things to think about when building a SharePoint 2013 infrastructure on Azure IAAS.

    You can download and listen to Episode 40 of the Microsoft Cloud Show here. Enjoy!

  • How to check the version of AppFabric 1.1 aka the Distributed Cache

    Tags: AppFabric, SharePoint 2013

    Introduction

    The other day I posted about the patching procedure for the SharePoint 2013 Distributed Cache (Microsoft AppFabric 1.1) and on that post I got a great comment from Riccardo:

    Hi Wictor, is it possible to discover the patch level of the Distributed Cache without looking at control panel? Powershell?

    That is a great question Riccardo! But the answer is not that simple…

    Check the version using Installed Updates

    The easiest way to manually check what version of AppFabric you are using, or rather which CU that is applied to AppFabric 1.1, is to use the Program and Features tool in Windows Server and then click on View installed updates.

    Installed AppFabric updates

    As you can see in the screenshot above, they all have the same version 1.1.2106.32 and that is also the version you get when you query the registry with this:

    Get-ItemProperty HKLM:\SOFTWARE\Microsoft\AppFabric\V1.0\ -Name ProductVersion | select ProductVersion

    That version number is the version number of Microsoft AppFabric 1.1 and it doesn’t change when you are applying Cumulative Updates.

    Check the Distributed Cache version using PowerShell

    If you want to automate checking the patch level of the Distributed Cache then the only way I’ve found (if someone has a better way, please enlighten me) is to actually check the version of one specific file, the Microsoft.ApplicationServer.Caching.Configuration.dll file. This is one of the few files that is updated in all the current CU’s (1-5), but I cannot guarantee that it will be updated in future version. The check is is done using the following PowerShell script (remove the line breaks and paste as a single line):

    (Get-ItemProperty "C:\Program Files\AppFabric 1.1 for Windows Server\
    PowershellModules\DistributedCacheConfiguration
    \Microsoft.ApplicationServer.Caching.Configuration.dll" -Name VersionInfo)
    .VersionInfo.ProductVersion

    This script will give you different version numbers depending on your AppFabric 1.1 CU level as follows:

    CU level Product Version
    CU1 1.0.4639.0
    CU2 1.0.4644.0
    CU3 1.0.4652.2
    CU4 1.0.4653.2
    CU5 1.0.4655.2

    If this method is valid for future CU’s then please remind me to update this table.

    Summary

    So Riccardo, there you have the best answer I can give you. It is not an optimal solution but it works…

  • How to patch the Distributed Cache in SharePoint 2013

    Tags: SharePoint 2013, AppFabric

    Introduction

    In SharePoint 2013 the Distributed Cache plays a very important role, it is a key component for performance and caching. An incorrectly configured or managed Distributed Cache will cause issues, with your farm. I’ve even seen blogs recommending turning it off, most likely due to that they don’t manage the cache properly and get into a situation where it causes even worse performance problems. One of the good things with the Distributed Cache is that is not a SharePoint service, it is a standalone service called AppFabric 1.1 for Windows Server. Initially the guidance from Microsoft was that the Distributed Cache (DC) would be patched together with SharePoint and we should keep our hands off it. But that has over the time changed and allowed us to take advantage of the fixes and improvements that the AppFabric team does to the service. So, it is our responsibility to patch the Distributed Cache. But how do we do it?

    [Update 2014-07-16] Here's a link to the official statement that AppFabric/DC is independently updated from SharePoint: KB2843251

    Proper patch instruction

    First of all there are currently five released Cumulative Updates (CU) for AppFabric 1.1, CU1 to CU5. An AppFabric Cumulative Update is exactly the same as a CU for SharePoint, it contains all previous CUs. So if you install CU5 you also get CU1 to CU4 installed. The important thing with this is that you as an administrator should not just read the latest CU Knowledge base article, but also the previous ones (you will see one reason in a minute).

    Let’s assume that we have a SharePoint farm with more than one servers running the Distributed Cache service instance. To patch these we need to download the AppFabric CU that we would like to use. I’d recommend using the latest (CU5) right now and I’ve not yet seen any issues with it, only positive effects. If you are using Windows Server 2012 you definitely should go with at least CU4.

    Here’s a link to the different CU’s and their KB article respectively

    In this case let’s apply CU5.

    The first thing before patching that is really important to do is to properly shut down the Distributed Service instance. The reason that you would like to do this is that some items in the Distributed Cache is only living in the cache and is not backed to a persistent storage, such as some items in the SharePoint Newsfeed. If you do not properly shut down the service instance you will loose that data. Fortunately we have a great PowerShell cmdlet that does this job for us. The process here is that you need to patch one machine at a time according to these steps:

    1. Shut down the service instance on one machine
    2. Patch AppFabric 1.1
    3. Post-patch operations
    4. Start the service instance
    5. Restart from 1 on the next machine

    Do not do servers in parallel unless you have a massive amount of servers and can handle that extra load/redundancy!

    Step (1) is done in PowerShell using one of the built-in SharePoint cmdlets:

    asnp *sharepoint*
    Stop-SPDistributedCacheServiceInstance -Graceful

    This command will gracefully shut down the service instance on the local machine. A graceful shutdown means that all the cache items will be distributed to the other service instances in the cache cluster. Make sure that you have enough overhead to do this. This is yet another example of my “3 is the new 2” rule is important. When you patch one server you don’t want just one extra machine! Once the service instance is stopped, I normally wait a couple of extra minutes to make sure that all the cached items has properly propagated to the other servers.

    Then it is time to apply the actual AppFabric patch, step (2). Run the patch executable and follow the instructions. It’s basically a next, next, finish procedure.

    Step (3). When the patch is applied you should have read through the KB articles, and if you are applying CU3 or later you should have seen that in order to improve performance you need to modify the Distributed Cache configuration file. The CU3 KB article mentions a new feature added to the AppFabric service that takes advantage of the non-blocking background garbage collection feature in .NET 4.5, which is important for machines with large amounts of RAM – and that is exactly the description of a SharePoint server. So modify the DistributedCacheService.exe.config file as below to enable the background garbage collection:

    <configuration>
      ...
      <appSettings>
        <add key="backgroundGC" value="true"/>
      </appSettings>
    </configuration>

    The final thing we need to do on the machine is to start the service instance again, step (4). The AppFabric Windows Service will be disabled when it is shut down and you should NOT try to start that one manually, you must use the following PowerShell (or corresponding action in Central Administration if you’re a n00b).

    $instance = Get-SPServiceInstance | ? {$_.TypeName -eq "Distributed Cache" -and $_.Server.Name -eq $env:computername}
    $instance.Provision()

    This PowerShell snippet will get the Distributed Cache service instance on the machine where it is executed and provision it and start the AppFabric Windows Service.

    Once this is done and you are ready to move on to the next machine, step (5), give it a couple of extra minutes so that the newly patched (and empty) cache service instance has time to catch up.

    Not patching properly…

    A very common issue is that in order to apply the patch you just run the patch executable. Two things will happen when you do this. First of all the service will be shut down, but not gracefully and you will loose data. Secondly it will not start the service instance properly or at all. The patch contains a script that waits for the service to come online, but since this is not a normal AppFabric cache, it’s controlled by SharePoint, so this script will wait forever until the service comes up. If this happens all you have to do is kill the script window and start the service properly as shown above (warning: this is not a recommendation and there might be side effects)

    Summary

    I hope this short post cleared up some confusion on how to patch the Distributed Cache service in SharePoint 2013 and gave you an example on how to do this in a production environment without loosing any data. Of course, you should always test the procedure in a test/staging environment. Cache on!

  • Using SharePoint 2013 with Thinktecture IdentityServer 2

    Tags: SharePoint 2013, Claims

    Introduction

    SharePoint 2013 (and earlier versions) allows you to use alternative authentication “sources” than Windows. We can part from the different options with Windows login, use Forms Based Authentication (FBA) or use a federated/trusted identity provider. Forms based authentication is a good approach if you don’t want to manage your users in Active Directory or if you don’t want to use Windows Login. The downside with FBA is that you must manually do some web.config modifications, there isn’t any UI for managing the users (yes, I know you can use LDAP or just download something from the tubez, but you get my point). Using a federated approach is more interesting, that allows you to get the identity management and authentication away from your SharePoint farm (and this is a really good thing, SharePoint admins are generally not identity management people!). A trusted identity provider is a service such as Active Directory Federation Services (AD FS), Microsoft Azure Access Control Services (ACS) or any other SAML 1.1 compatible Identity Provider (IdP).

    I generally don’t like using FBA, due to the issues above. In this post I will show you how to do forms based authentication without using FBA, but instead using a trusted identity provider called Thinktecture IdentityServer 2. IdentityServer 2 is an awesome implementation of an identity provider and supports a plethora of standards. And it comes as open source, with a web based UI, is very configurable and is just one of the best options for you when you need to set up your SharePoint with users not in Active Directory.

    Thinktecture IdentityServer 2

    Without describing the Thinktecture IdentityServer 2 in my own words I think the official overview says enough:

    IdentityServer is a light-weight security token service built using .NET 4.5, ASP.NET MVC4, WCF and Web API. It has out-of-the-box support for user management using the ASP.NET membership and roles provider, but provides extensibility points to integrate arbitrary user databases. Futhermore almost any aspect of token issuance and data management can be customized using a plugin model.

    The really cool thing with it is that it supports SAML 1.1 and can be used as a SharePoint 2013 (or 2010) trusted identity provider with a few lines of PowerShell. Secondly if you already has SharePoint hooked up to AD FS (or another similar service) you can add IdentityServer 2 as Claims Provider Trust.

    Installing IdentityServer 2

    The first thing we should do is to get our hands on IdentityServer 2 and set it up in our environment. You have two options; either make a github clone if you need to modify the source (https://github.com/thinktecture/Thinktecture.IdentityServer.v2) or download it directly (http://thinktecture.github.io/Thinktecture.IdentityServer.v2/downloads/). In this post I assume you don’t want to do any development and do it the lazy way and just downloads it. Ok, the official link above will only give you a 404 when you click on the download link at the moment, so use this page instead to download the .zip file for version 2.2: https://github.com/thinktecture/Thinktecture.IdentityServer.v2/releases/v2.2.0.

    Now create a folder called IdSrv (or whatever you prefer) on one of your local disks, for instance d:\IdSrv\, and extract the ZIP file there. Next we need to create an IIS Web Site that uses this directory. Before proceeding with that we need to create a web server certificate – remember, always use certificates for these kind of services! You could use a self signed, but as always if you can use a domain/real CA that is preferred. Also you might need to set up a DNS entry, which is what I will use (idsrv.contoso.com). Once you have the certificate and hostname then you can proceed to the IIS configuring.

    We’ll create a new IIS site pointing at the folder we unpacked IdentityServer to. We also need to specify the binding to use https and select our certificate, image on the left below. When we click OK this will create a new application pool for us in IIS, for that application pool we need to switch the identity to use a domain account or if we’re doing a single server install use the Network Service, image on the right below.

    New IIS web site and bindings Change the app pool identity

    Since we now told IdentityServer to run as Network Service we also need to modify the folder permissions of the App_Data catalog and give the Network Service Modify permissions on that catalog. The App_Data catalog is where IdentityServer by default stores its database.

    Once we have done this we can browse to the binding specified in IIS and start using IdentityServer 2.

    Since this is a fresh installation and IdentityServer has no database we will be presented by a configuration screen. On that screen you should enter a site name, you must specify the Issuer URI (which uniquely identifies this issuer) as well as choose a signing certificate (this could be the one you created for the web site or a different one). Then you also need to specify an administrator username and password.

    Before you click save you must make sure that the account running the application pool (Network Service in this sample) has permissions to read the private key of the certificate that you use as signing certificate.

    Configuring IdentityServer

    Once you have done that you are all set to go!

    Users and roles

    To create users and roles you log in as the administrator and go to the administration menu and select users and/or roles. If you want to extend the user profile with more properties you need to modify the profile.config file under /Configuration in the IdentityServer folder. For this sample I add an additional property called “Title”, which I will use in a bit.

    Before proceeding create at least one user and add that user to the “IdentityServerUsers” role, that is by default the role that is needed to use IdentityServer for authentication federation.

    Final identity server tweaks

    Ok, now have an awesome identity system up and running. We need to tweak a few things in IdentityServer before doing the SharePoint stuff. We need to add a Relying Party (RP) to IdentityServer, and the Relying Party in this case is SharePoint. As the administrator in IdentityServer choose to add a new Relying Party. This RP needs a display name, a realm and a redirect URL. The display name is whatever you prefer, the realm must be a URI and the Redirect URL should be you SharePoint site with “/_trust/” appended, like in the screenshot below.

    Relying party in IdentityServer

    Another tweak we need to do is to tell identity server to use SAML 1.1 instead of SAML 2.0, since SharePoint does not support SAML 2.0. That is done under General Configuration of IdentityServer and the “Default WS* Token” property should be set to “urn:oasis:names:tc:SAML:1.0:assertion”.

    SNAGHTML10f558c

    Adding IdentityServer 2 as a trusted identity provider

    Now it is time to head on over to SharePoint and fire up a PowerShell prompt! The first thing we will do is to define a couple of properties:

    ## PROPERTIES
    $realm = "uri:identityserver:sp03"
    $signinurl = "https://idsrv.contoso.com/issue/wsfed"
    $description ="Provide description here"
    $url = "https://rootsite.sp03.contoso.com"
    $metadataurl = "https://idsrv.contoso.com/FederationMetadata/2007-06/FederationMetadata.xml"

    The realm must be the same realm that we configured in IdentityServer for the Relying Party. The signinurl can be found on the home page > application integration in IdentityServer and it’s the URL called WS-federation. Description is just some description that you provide and url is the URL to the Web Application where you want to add this identity provider. Finally metadataurl is the WS-federation metadata end-point for identity server and that URL can be found on the same page as where we located the sign in URL.

    We’ll use this WS-federation metadata end-point to do some discovery from the SharePoint side:

    ## LOAD FEDERATION METADATA
    $fedmd = Invoke-WebRequest -Uri $metadataurl
    $fedmdXml = New-Object Xml
    $fedmdXml.LoadXml($fedmd.Content)
    $base64 = $fedmdXml.EntityDescriptor.RoleDescriptor.KeyDescriptor.KeyInfo.X509Data.X509Certificate
    $base64| Out-File -FilePath temp.cer -Append:$false
    $cert = Get-PfxCertificate -FilePath temp.cer
    Remove-Item temp.cer
    

    By invoking the end-point we can retrieve the signing certificate, and other metadata. We grab that certificate and temporary stores it as a .cer file before loading it up in a certificate variable which will be used to install it in SharePoint.

    ## ADD TRUST IN SP
    asnp *sh*
    $name = $fedmdXml.EntityDescriptor.RoleDescriptor.GetAttribute("ServiceDescription")
    New-SPTrustedRootAuthority -Name $name -Certificate $cert | Out-Null
    

    We use the certificate variable to create a trust in SharePoint, together with the name of the service, retrieved from the metadata endpoint.

    Next up is to create some claims mappings and the trusted identity token issuer in SharePoint.

    ## CREATE TRUSTED IDENTITY TOKEN ISSUER
    $map1 = New-SPClaimTypeMapping "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress" `
        -IncomingClaimTypeDisplayName "Email" –SameAsIncoming 
    $map2 = New-SPClaimTypeMapping "http://schemas.microsoft.com/ws/2008/06/identity/claims/role" `
        -IncomingClaimTypeDisplayName "Role" -SameAsIncoming 
    $map3 = New-SPClaimTypeMapping "http://identityserver.thinktecture.com/claims/profileclaims/title" `
        -IncomingClaimTypeDisplayName "Title" -SameAsIncoming 
    $tit = New-SPTrustedIdentityTokenIssuer -Name $name -Description $description -Realm $realm `
        -ImportTrustCertificate $cert -ClaimsMappings $map1,$map2,$map3 -SignInUrl $signinurl `
        -IdentifierClaim $map1.InputClaimType
    

    We create three claims mappings in this case; the first one is the e-mail claim, the second one is a role claim and the third one is our custom claim using the user property Title we created. We can get the claims identifiers from the XML in the metadata endpoint. We’ll use the first claim, e-mail, as our identity claim when we finally create the trusted identity token issuer also passing in the realm and sign-in URL.

    Once this is done all we need to do is update our Web Application(s) to use this new identity provider. This can be done in the UI in Central Administration per Web Application or using PowerShell like this:

    ## UPDATE WEB APPLICATION WITH NEW IDP
    $wa = Get-SPWebApplication $url
    $ap = Get-SPAuthenticationProvider -WebApplication $wa -Zone Default
    if($ap.Count -eq 1) {
        $aps = New-Object System.Collections.ArrayList
        $aps.Add($ap) | Out-Null
    } 
    else {
        $aps = $ap
    }
    $ap = New-SPAuthenticationProvider -TrustedIdentityTokenIssuer $name
    $aps.Add($ap) | Out-Null
    Set-SPWebApplication -AuthenticationProvider $aps -Zone Default -Identity $wa
    

    First we retrieve the current authentication providers for the web application, then we create a new one from our just recently created trusted identity token issuer and then we add them all (as an array) to the web application.

    That’s it! Time for a test ride…

    Browse to a site on the web application where you enabled the IdentityServer authentication. You should be presented with the awesome standard SharePoint login selector page. In the drop down you will see two options; Windows Authentication and our IdentityServer option:

    Some sweet UI

    Select to log in using IdentityServer. This will take you to the log in page of identity server, where you present your credentials. If you log in using that user you created in identity server you will after a successful login see an access denied page from SharePoint. That’s because that user doesn’t have permissions yet.

    It is now we can take advantage of that we added the role or title claims. Open up a new browser and log in using Windows Authentication and give permissions to either a role or a title. Unfortunately the default claims people picker in SharePoint is a bit stupid when it comes to working with claims. You need to write the name of the group or title and then pick the correct claim. Hover the cursor over the alternatives to see which identity provider and which claim is which one. In this case we see three (our three different claims).

    Default Claims People Picker

    Once you have given permissions to a role, title or directly to the e-mail of the user, you should be able to log in properly and see the site.

    Summary

    In this post I’ve shown you how to use Thinktecture IdentityServer 2 as an identity provider to SharePoint 2013. IdentityServer is a great alternative to using FBA when you need a custom user and role store and it’s way easier and more manageable to set up, no fiddling with web.configs or installing full trust code you download from the interwebs in your SharePoint farm. Enjoy!

  • SharePoint 2013 with SAML Claims and Provider Hosted Apps

    Tags: SharePoint 2013, AD FS, Claims, Windows Server 2012 R2, Apps

    Introduction

    The other week I posted an article about how to use SharePoint Hosted Apps when using SAML Claims, I did not expect that amount of feedback I had on that blog post, in e-mail, comments, tweets etc. Some of that feedback was how do you do it with Provider Hosted apps. Well you’re about to find out. It took me a while to get it properly done and there are some things that you should be aware of. In this post I will walk you through the simplest scenario and you will notice that there are a couple of moving parts. But, since I am such an influencer I thought I should make it easier for you. I will show you how to do this without the minimal changes to your current provider hosted apps – you only have to add an extension file to your solution, make a small modification to the helper files that Visual Studio gives you and a couple of web.config modifications! All the code you need will be published in a Github repository (https://github.com/wictorwilen/SharePointContextSaml) for you to consume and do all the fancy gitty stuff that you code dweebs out there like.

    Let’s get started, bare with me this is a long post and for some of you the first parts might be just a repeat of what you’ve seen/done previously…

    Creating a Provider Hosted High Trust App

    The first thing we need is to create us a High Trust Provider Hosted App. That is an app that does not run in SharePoint and is using the Server to Server (S2S) protocol for authorization. In my case (and it should be yours as well if you want to use my helper code) is to use Visual Studio 2013 and create a SharePoint App, choose a provider hosted app using the ASP.NET MVC template. To create a High Trust App we then need to specify a certificate for the S2S trust. This is how you can create a cert using makecert.exe and some PowerShell. In the script below all you need to do is modify the four properties; domain, name and password of cert and the location where we will store the certificate files:

    #ALIASES
    set-alias makecert "C:\Program Files\Microsoft Office Servers\15.0\Tools\makecert.exe"
    
    # CREATE SELF SIGNED DEV CERTS
    $domain = "localhost"
    $certName = "HightTrustSAMLDemo" 
    $password = "Pass@word1"
    $dir = "c:\HighTrustCerts\"
    
    New-Item $dir -ItemType Directory -Force -Confirm:$false | Out-Null
    
    $publicPath = $dir + $certName + ".cer"
    $privatePath = $dir + $certName + ".pfx"
    
    makecert -r -pe -n "CN=$domain" -b 01/01/2012 -e 01/01/2022 -eku 1.3.6.1.5.5.7.3.1 -ss my -sr localMachine -sky exchange -sy 12 -sp "Microsoft RSA SChannel Cryptographic Provider" $publicPath 
    
    $publicCert = Get-PfxCertificate -FilePath $publicPath
    $publicThumbprint = $publicCert.Thumbprint
    
    Get-ChildItem Cert:\LocalMachine\My | 
        Where-Object {$_.Thumbprint -eq $publicThumbprint} | 
        ForEach-Object {
            $_.Export("PFX",$password) |Set-Content $privatePath -Encoding Byte        
        }

    Once we have the certificate we need to install the certificate in the SharePoint certificate store as well as register a Trusted Security Token Issuer. And to do this we need a Guid! You can use any preferred tool to generate your Guid or generate and register your very own unique Guid at www.vanityguid.com. Here’s a script the does the trick; give it your very own unique Guid/Issuer Id and a unique display name:

    $issuerId = '17721805-67c2-46ba-92d1-fd1fdf91bb01' # MUST BE UNIQUE
    $displayName = "High Trust SAML Demo" # MUST BE UNIQUE
    
    Add-PSSnapin Microsoft.SharePoint.PowerShell -EA 0
    $certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($publicCert)
    
    New-SPTrustedRootAuthority -Name $certName -Certificate $certificate
    $realm = Get-SPAuthenticationRealm
    $fullIssuerId = $issuerId + "@" + $realm
    
    New-SPTrustedSecurityTokenIssuer -Name $displayName -Certificate $certificate -RegisteredIssuerName $fullIssuerId –IsTrustBroker
    iisreset 

    Yes I trust Wictor!Ok, this was standard High Trust App procedure. Copy and paste the issuer Id into the Visual Studio wizard and enter the certificate details to complete the app creation. What you have right now is a standard High Trust app ready to work with Windows (claims) users. Before you start hacking in some code into this app – just make sure that it properly deploys and executes (ie press F5), so that you haven’t messed up any Guids or anything else. This app should now run using IISExpress and it uses Windows Authentication (remove http(s)://localhost from Local Intranet sites in Internet Explorer if you want to test different users).

    What about them SAML users?

    So let’s see what happens if we log into our SharePoint Developer site, where we deployed our High Trust app, as a SAML user (using for instance AD FS 3.0). Of course you need to setup a trusted identity token issuer before doing that, how that’s done is out of scope of this post. When logged in as a SAML user and I click on the High Trust App I either get logged into the app as the currently logged on Windows user or I get the Windows login dialog (if http(s)://localhost was removed from Local Intranet sites). So, it doesn’t really work! We don’t even get the chance of authentication with our AD FS (or similar federation service).

    Configuring the app for federated Authentication

    ASP.NET devs get all the new shiny stuffWhat we need to do is modify our web application (the provider hosted web) to use federated authentication. Unfortunately we are SharePoint developers, and are not given all the love that ordinary ASP.NET developers are from the Visual Studio team. When creating a standard ASP.NET project we get the really awesome option of specifying which authentication method we would like to use and we can point out our federation service and Visual Studio fixes everything for us.

    We can do this in two different ways. Either we add a new ASP.NET MVC project and remove the default web project and convert that ASP.NET MVC project into a SharePoint app project (which is the easiest way) or we start fiddling with web.config and configure it ourselves – which is exactly what we’re going to do here!

    The first thing we need to do is to use NuGet and add the Microsoft ASP.NET Identity Core package and the Microsoft Token Validation Extension for .NET 4.5. These package will give us all the assemblies and references we need to run the app.

    NuGet FTW!

    In your web application project open up web.config, now we’re going to do some copy-paste work! First of all we need to add a configSections element for the System.IdentityModel. Insert the snippet below directly after the configuration element:

    <configSections>
      <section name="system.identityModel" 
        type="System.IdentityModel.Configuration.SystemIdentityModelSection, System.IdentityModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089" />
      <section name="system.identityModel.services" 
        type="System.IdentityModel.Services.Configuration.SystemIdentityModelServicesSection, System.IdentityModel.Services, Version=4.0.0.0, Culture=neutral, PublicKeyToken=B77A5C561934E089" />
    </configSections>

    Then head on over to the system.web element and below the authorization element add the following.

    <authentication mode="None" />  

    This means that we’re not letting ASP.NET do the Authentication, but instead handling it over to a new module which we’re going to add. Directly after the system.web element copy and paste the following, which will add two new modules to the ASP.NET pipeline:

    <system.webServer>
      <modules>
        <add name="WSFederationAuthenticationModule" type="System.IdentityModel.Services.WSFederationAuthenticationModule, System.IdentityModel.Services, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" preCondition="managedHandler" />
        <add name="SessionAuthenticationModule" type="System.IdentityModel.Services.SessionAuthenticationModule, System.IdentityModel.Services, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" preCondition="managedHandler" />
      </modules>
    </system.webServer>
    These two modules are called WSFAM and SAM (two more acronyms for you JB). WSFAM is responsible for the login/logout process and SAM takes care of session management by using cookies.

    Then we need to add the configuration for our Relying Party. This is done by adding the system.identityModel element after the system.serviceModel element. It’s a handful of configuration and we need to modify it in a few places to match our app and our federation service.

    <system.identityModel>
      <identityConfiguration>
        <audienceUris>
          <add value="uri:SharePoint:App:SAML" />
        </audienceUris>
        <securityTokenHandlers>
          <add type="System.IdentityModel.Services.Tokens.MachineKeySessionSecurityTokenHandler, System.IdentityModel.Services, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
          <remove type="System.IdentityModel.Tokens.SessionSecurityTokenHandler, System.IdentityModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
        </securityTokenHandlers>
        <certificateValidation certificateValidationMode="None" />
        <issuerNameRegistry type="System.IdentityModel.Tokens.ValidatingIssuerNameRegistry, System.IdentityModel.Tokens.ValidatingIssuerNameRegistry">
          <authority name="fs.contoso.com">
            <keys>
              <add thumbprint="EF66CEF27067637D296C403691DBBE2A8B5C7BAA" />
            </keys>
            <validIssuers>
              <add name="http://fs.contoso.com/adfs/services/trust" />
            </validIssuers>
          </authority>
        </issuerNameRegistry>
      </identityConfiguration>
    </system.identityModel>

    First of all on line #4 – that row must contain the relying party identifier for our app registered in AD FS. I prefer using a URI, since that makes it easier when I need to deploy this into production, but you could use the URL of your web application (https://localhost:44305/ for instance). I’ll show you in just a minute how to configure this in AD FS. Line 12 should contain the name of our authority and line 17 must have the Federation Service Identifier of AD FS (found under Federation Service Properties in the AD FS Console). On line 14 we need to specify the thumbprint of the AD FS Token-signing certificate (you’ll find that easy in the AD FS Console under Service/Certificates).

    Ok, only one more thing to fiddle with in the web.config.

    <system.identityModel.services>
      <federationConfiguration>
        <cookieHandler requireSsl="true" />
        <wsFederation passiveRedirectEnabled="true"
          issuer="https://fs.contoso.com/adfs/ls/"  
          realm="uri:SharePoint:App:SAML" 
          requireHttps="true" />
      </federationConfiguration>
    </system.identityModel.services>

    Here there are two things we need to modify. First of all we need to point to the SAM/WS-Federation endpoint of AD FS, the login page, that Url is set in the issuer property of the wsFederation element. Secondly we must add the realm of our app – which should be the same as the audienceUri specified above. As I said earlier, I use a URI instead of the web application Url.

    A quick note here as well, notice that I use HTTPS and SSL all the way. For my SharePoint web application and for my app web application. If you prefer to be sloppy you can still use HTTP but that requires some extra work in SharePoint and the app web application config.

    Let’s take our High Trust App for a ride! Press F5 to get the app up and running. Doesn’t work…well some of it do. You should now be taken to the AD FS login page, but that one will just tell you an error occurred and the details doesn’t give much of a hint. You have to take a look in the AD FS server and the event log for AD FS. The error message is pretty clear on what we haven’t done yet:

    MSIS7007: The requested relying party trust 'uri:SharePoint:App:SAML' is unspecified or unsupported. 
    If a relying party trust was specified, it is possible that you do not have permission to 
    access the trust relying party. 
    Contact your administrator for details.
    

    Yes, we haven’t created our Relying Party Trust yet.

    Creating the Relying Party Trust in AD FS

    Claims rulesNow we need to create our Relying Part Trust in AD FS, it’s pretty straightforward, but comes with a couple of caveats. Add a new Relying Party Trust and choose to enter data manually, then give the trust a display name (I prefer to use the realm here). Continue by choosing to use the AD FS Profile, skip the certificate page in the wizard, and then select to enable support for the WS-Federation Passive protocol and enter the URL of your App. On the next page add a new identifier and now use the realm of your app, then just keep on clicking Next until you’re done. Once you’re done with creating the Relying Party Trust a new dialog will open up in AD FS in which you need to create Claims Rules for this new trust. This is a very important step, you need to get the rules correct and include the correct claims. In this case I will add two claims. One which augments the E-Mail claim from Active Directory (LDAP) for my Active Directory users and one which will just pass through the E-Mail claim from my third party Identity Provider (Thinktecture Identity Server). You have to pass either E-Mail, SIP address or UPN to get this to work. SharePoint needs at least one of those claims to be able to rehydrate the user from the User Profile Service Application.

    Once that is done, go back to your App and start it again using F5. Now logon using one of the Active Directory, use some other account than the one you are currently logged on to your machine as (otherwise you’ll think you are home safe already). You should see the default ASP.NET MVC SharePoint App page which says “Welcome Someone”, where Someone is the name of the logged in user. But what! That’s not the name of the account you logged in using? You can easy check that you are actually properly logged in using AD FS by setting a break point in the HomeController.cs in the Index action and take a look at some watches. You can clearly see that we are authenticated through federation and you’ll see the e-mail claim for that user who authenticated through AD FS, but the identity shown is the wrong one!

    Visual Studio Baby!

    Well, it turns out that the SharePointContext.cs and TokenHelper.cs classes that the Visual Studio and Office Dev team so kindly wrote for us only works with Windows Claims for High Trust apps. Bummer!

    Steve Peschka to the rescue!

    I barely gave up when I hit this, but my Bing-fu found a single blog post by Steve Peschka (who knew!) called “Using SharePoint Apps with SAML and FBA Sites in SharePoint 2013” who solved my problems, almost. This post of Steve was written some time ago when the TokenHelper.cs class looked very different and it forced you to make modifications to your app to support federated users. So I took his example code and modified it to fit the new helper classes provided by Visual Studio 2013 and made them in such way that you do not have to re-write your apps and packaged it into a single .cs file; SharePointContextSaml.cs. I will NOT post the entire thing here, since it is a lot of code. But if you’re like me you should read through it, it gives you tons of knowledge on how the app authorization actually works beneath the hood.

    So you just download the .cs extension file and add to your app web application project and then you have to do a couple of tweaks to your app configuration and to the default TokenHelper.cs and SharePointContext.cs files. This is how you do it.

    1) After you copied the SharePointContextSaml.cs file into your project, modify the namespace of that file to match the namespace of your project.

    2) Modify the default TokenHelper.cs class declaration so that the TokenHelper class is a partial class declaration, we need this to be able to extend the TokenHelper class. (This declaration should have been there by default imho!)

    public static partial class TokenHelper {...}

    3) In the SharePointContext.cs file locate the static constructor for the SharePointContextProvider class and modify it so it looks like below. This is needed so that we can use our custom context provider if the user is authenticated through federation.

    static SharePointContextProvider()
    {
        if (!TokenHelper.IsHighTrustApp())
        {
            SharePointContextProvider.current = new SharePointAcsContextProvider();
        }
        else
        {
            if(HttpContext.Current.User.Identity.AuthenticationType == "Federation") {
                SharePointContextProvider.current = new SharePointHighTrustSamlContextProvider();
            } else {
                SharePointContextProvider.current = new SharePointHighTrustContextProvider();
            }
        }
    }

    4) Finally we need to add three entries to the appSettings in web.config:

    <add key="spsaml:ClaimProviderType" value="SAML"/>
    <add key="spsaml:TrustedProviderName" value="High Trust SAML Demo"/>
    <add key="spsaml:IdentityClaimType" value="SMTP"/>

    The first entry is used to tell our custom context provider that we use SAML claims (there is support for Forms based authentication, value = FBA, in the extensions that Steve originally wrote but I have not yet tested it at all in my edition, and personally I prefer using something like IdentityServer instead of FBA). The second entry must be the name of the Trusted Security Token Issuer that you created using PowerShell for your app. And finally we need to specify what claim we will extract and use as identity claim; possible values are SMTP (e-mail), SIP or UPN.

    Yet another hobbit e-mailLet’s try to run the app once again. Most likely you will be able to logon but then you get an exception in the HomeController, an access denied. Then you need to check two things. First make sure that the user that you’re logging in with have permissions on your site. Since we’re talking about SAML claims here the people picker behaves a bit differently, enter the e-mail address of the user and make sure to choose the e-mail claim name (hover the mouse over the suggested options to see the claim type). Secondly you must make sure that the user has the WorkEmail attribute in the user profile store populated. This can be done either by using the User Profile Synchronization, if you’re authenticating with an Active Directory or LDAP source. But if you’re using some other IdP, for instance IdentityServer that I use here, you must add the WorkEmail manually through PowerShell or some other method. Once that is done you should see a nice and warm welcome to your SAML claims user!

    That’s it! All you need to do to enable SAML Claims user for your SharePoint 2013 High Trust App.

    Show me the codez!

    I’ve created a Github repository to host the SharePointContextSaml.cs file, you can find it here: https://github.com/wictorwilen/SharePointContextSaml

    I’d really like to get feedback in form of features, bugs, fixes etc. Please use the issue feature on Github, instead of e-mailing me…

    Currently it is not documented at all, there are some comments; a mix of mine and Steves original ones.

    I’d like to maintain it as long as possible, but my hope is that the Office Dev Team picks this up and do some proper testing and then merges this with the default Visual Studio helper files. Well, one can at least wish!

    Summary

    Yet another long blog post. I really hope I didn’t miss anything, there’s quite a few step once you do this the first time. I’ve shown you the complete example here; how to create and register a high trust SharePoint 2013 app, how to convert the default MVC project to a Claims based web application, registering a Trusted Relying Party in AD FS and then finally adding the new SharePointContextSaml.cs file and enabling you to use federated users in your Provider hosted app. Good luck to you all and thanks Steve for doing all the hard work!

  • SharePoint 2013 with SAML Claims and SharePoint Hosted Apps

    Tags: SharePoint 2013, AD FS, Claims, Windows Server 2012 R2, Apps

    Introduction

    By now each and every SharePoint developer out there should spend their time building SharePoint Apps instead of the old trusted friend of ours; Full Trust Code. Ok, Apps doesn’t solve the equivalent of world hunger in SharePoint at the moment, but that’s a discussion for another time. I assume you get my point. We have two types of apps (we used to have three little monkeys jumping in the bed, but one just bumped his head); Provider hosted apps and SharePoint hosted apps. Without going into details, Provider hosted apps are the apps that are hosted outside of SharePoint on a specific location (URL) and SharePoint hosted apps are running on top of SharePoint (using JavaScript) on a “random” location. This location is called the App Web and is a SharePoint SPWeb with a specific randomly generated URL. That URL could look something like this:

    https://app-2d4f884a19b8e8.apps.sp03-apps.com/sites/dev/SharePointApp1

    In that URL the first part is something that is randomly generated for each and every instance and deployment of that App. That is why we are required to use a Web Application listening on all incoming host headers, SAN wildcard certificates and users are also required to log in if we’re using Windows Authentication and the base App URL isn’t in the Local Intranet Zone.

    There is also another interesting issue specifically when we are using SAML Claims Authentication. Originally this did not work at all with SharePoint and specifically not with AD FS 2.0. One of the requirements of running SharePoint hosted apps with SAML claims is that the identity provider, IdP. (AD FS for instance) supports a wildcard return URL and that it uses the wreply parameter [Source]. Whenever you are trying to authenticate with the app you have to visit the IdP to get your claims and then be redirected back to the app, which is on a “random” URL. Using AD FS 2.0 you have to add a relying party trust for each and every app instance (something your IT people really enjoy doing).

    Fortunately all this is resolved in AD FS 3.0 that is shipped with Windows Server 2012 R2! Let’s take a look on how to set it up!

    Configure AD FS 3.0

    The first thing we need to do is to configure our AD FS 3.0 service. I assume you already have set up and installed AD FS 3.0 on a Windows Server 2012 R2 and that you have Claims authentication working for normal SharePoint sites. If you haven’t come this far yet please take a look at Steve Peschkas end-to-end guide for this, its for SharePoint 2010 and AD FS 2.0 but everything is almost the same.

    Here’s our scenario: We’re having one Web Application hosting both the Intranet (intranet.sp03.contoso.com) and the My Site host (mysite.sp03.contoso.com). We would like users with Windows Claims and SAML Claims with users from Active Directory and a third party to be able to use SharePoint Hosted Apps.

    The traditional way of setting it up in AD FS is to create two Relying Party Trusts; one for the Intranet and one for the My Site, like this:

    AD FS Relying Party Trusts

    Each Relying Party Trust has its unique realm. In this case urn:sharepoint:contoso:sp03:intranet and urn:sharepoint:contoso:sp03:mysite. And on our Trusted Identity Token Issuer in SharePoint we specify an additional Realm for the My Site, which could look similar to below:

    $tit = Get-SPTrustedIdentityTokenIssuer fs.contoso.com
    $tit.ProviderRealms.Add( (New-Object System.Uri("https://mysite.sp03.contoso.com")), 
      "urn:sharepoint:contoso:sp03:mysite")
    $tit.Update()

    It’s a pretty straightforward approach and works fine.

    But there is a much better way. Let’s get rid of this additional Relying Party Trust in AD FS and remove the additional Realm on the Trusted Identity Token Issuer. Well that doesn’t work! Now we get “An error has occurred” when trying to log in to the My Site. Fortunately it is very easy to fix.

    First head on over to AD FS and your (now single) Relying Party Trust and choose to edit its Properties. In the properties dialog choose the Endpoints tab and then choose to add a new endpoint by clicking on “Add WS-Federation”. As the Trusted URL type in your My Site host URL and add /_trust/ at the end. Once you click OK it should look something similar to this.

    Relying Party Trust Properties

    Finish it all by clicking OK. If you now try to log you will still see the same error, and if you take a look at the URL you’re being redirected back to you will see that you are redirected to the Intranet URL, which is the Default endpoint for the Relying Party Trust. So we have to do one more thing to get it to work, and this time on the SharePoint side. We need to tell SharePoint to use the WS-Fed wreply parameter.

    From the WS-Fed specification: wreply: This optional parameter is the URL to which responses are directed. Note that this serves roughly the same purpose as the WS-Addressing <wsa:ReplyTo> header for the WS-Trust SOAP RST messages

    Use the following PowerShell script to modify the Trusted Identity Token Issuer to use the wreply parameter:

    $tit = Get-SPTrustedIdentityTokenIssuer fs.contoso.com
    $tit.UseWReplyParameter = $true
    $tit.Update()

    Now! Let’s try to log in to the My Site once again using one of the providers we have configured in AD FS. If you have given the correct permissions to the user on the My Site host you will actually see it load and you’re golden.

    What about them Apps?

    Ok, I know I told you that this post was about getting SharePoint hosted apps to work with SAML Claims and I’ve only shown you how to log on to the My Site. Well, you see if you get this to work there is very little extra you have to do to get the SharePoint Hosted Apps to work. If I now was logged in as a SAML Claims user on the My Site and decided to install an app onto My Site (or any site for that matter), once I try to open the app I will get “An error has occurred on the server” and I will be redirected to the Intranet site.

    Just as we did get redirected to the Intranet site when not having the additional Endpoint on the Relying Party trust the same will happen with SharePoint Hosted Apps. The SharePoint Hosted Apps lives in these “random” App URLs and AD FS has no idea on how to and where to redirect the user after trying to authenticate on one of those. Now we have two options; add an endpoint for each and every single instance of our SharePoint Hosted Apps or…in AD FS 3.0 use a wildcard endpoint! Let’s try that!

    Just as we added the My Site endpoint to the Relying Party Trust let’s add our App Domain as en endpoint. The URL we need to add as a passive endpoint is formatted like this (of course you should use the app domain that you have configured in your SharePoint environment):
    https://*.apps.sp03-apps.com/_trust

    Notice that I use a * (star) as the wildcard character at the position where the randomly generated app id is used. It is required that you already have configured the Trusted Identity Token Issuer to use the wreply parameter like we did above, without that you will still get redirected to the default endpoint.

    Once done your Relying Party Trust should look like this:

    The App wildcard Endpoint

    If you now install and/or browse to a SharePoint Hosted App within your environment you will get prompted to authenticate and once that is done you should see your app. The picture below shows a user, stored in a third party IdP federated with AD FS, that has installed the default Visual Studio demo SharePoint Hosted App onto his My Site.

    Just another crappy app

    Note: Not a single iisreset was required to set this up! That doesn’t happen often!

    Summary

    In this post I have shown you how to configure AD FS 3.0 in Windows Server 2012 R2 to use a wildcard redirection endpoint to support Claims users to use SharePoint Hosted Apps in SharePoint 2013. A combination of new features in AD FS 3.0 and correct configuration in SharePoint 2013 was all that was needed.

  • Announcing Azure Commander

    Tags: Windows Azure, Windows Phone 8, Windows 8, Microsoft Azure

    For no one out there, in the SharePoint space or any other space, Microsoft Azure has gone unnoticed. Microsoft Azure is a really great service, or rather set of services, that for a (Microsoft or SharePoint) developer or IT-Pro is something that they should use and embrace. Personally I’ve been using Azure since the dawn of the service and I’ve been using it more and more. I use it to host web sites, host SharePoint and Office Apps, Virtual Machines, Access Control and a lots of other things.

    Specifically I’ve been lately using it for work more and more, and my customers and company are seeing huge benefits from using Microsoft Azure. We’re using it to host our development machines, demo environments, hosting full SharePoint production environments (staging, CI etc etc) and we’re using it to host SharePoint and Office Apps.

    All these services and instances and configurations must be managed somehow. For that we have the Azure portal (new and old version) and PowerShell. None of them are optimal in my opinion and I needed something more agile, something I could use when on the run, something to start, stop or reset my environments before a meeting, on my way home etc. This so I can optimize my resource/cost usage for the services and to save time.

    Introducing Azure Commander

    For all these purposes and reasons I have created a brand new Universal App for Windows 8.1 and Windows Phone 8.1 called – Azure Commander. Using Azure Commander I never forget to shut down a Virtual Machine since I can do it while commuting home, I can easily restart any of my web sites, when something hits the fan and more.

    Azure Commander are in its initial release focused on maintenance tasks for Virtual Machines (IAAS), Web/Worker Roles (PAAS) and Azure Web Sites. But you will see more features being added continuously! I personally like the option of easily firing up the app, choose a virtual machine and start an instant RDP session to it – from my laptop or Surface 2. For full features head on over to the Azure Commander web site.

    The interface on Windows 8.1

    AZ-Screen-W8-1.1.1.2

    The interface on Windows Phone 8.1

    AZ-Screen-WP8-1.1.1.2

    The App is since a couple of hours back available in both the Windows and Windows Phone store. You can get the app for only $3.99 – which is something you save in an instant when you use the app to remember turning of a virtual machine or web site or service over the weekend. A good thing is that this is an Universal Windows App – which means that if you buy them on either platform you will get it for free on the other one.

    Windows 8.1

    Download it from the Windows Store!

    Windows Phone 8.1

    Download it from the Windows Phone Store!

    Summary

    I hope you will enjoy the App as much as I do (both using it and building it). If you like it, please review it and follow Azure Commander on Twitter (@AzureCommander) or like it on Facebook.

About Wictor...

Wictor Wilén is a Director and SharePoint Architect working at Connecta AB. Wictor has achieved the Microsoft Certified Architect (MCA) - SharePoint 2010, Microsoft Certified Solutions Master (MCSM) - SharePoint  and Microsoft Certified Master (MCM) - SharePoint 2010 certifications. He has also been awarded Microsoft Most Valuable Professional (MVP) for four consecutive years.

And a word from our sponsors...

SharePoint 2010 Web Parts in Action