Welcome to the Rencore Tech Talks show. This time I’m catching up with Mikael Svenson. We’re talking about the PnP Provisioning engine and how Mikael has used it in his current projects to help them with provisioning new content to SharePoint. This is an interesting show to hear that the PnP Provisioning Engine works for big and small, and can be customized to fit your needs.
[Recorded: 2016-12-08]
Listen to this show to learn more about things like:
- What benefits does PnP provisioning engine give your project?
- A few examples of how Mikael uses the provisioning engine
- Using Microsoft Flow for approval
- What about using PnP Partner Pack?
- Decoupled templating with PnP Provisioning (no onet.xml, hallelujah!)
- And more
Guest: Mikael Svenson 
Mikael Svenson is a principal consultant for Puzzlepart, an Office 365 consultancy. He has worked in the search field for over 15 years implementing solutions for major international corporations and for several Nordic governmental institutions.
Mikael is a an international speaker as well as an Office Server and Services MVP for the past six years. He is a Microsoft P-TSP, and he is also involved in a lot of SharePoint community work in Norway. Mikael has worked with media monitoring software, developed an Enterprise Search Engine, and developed for Office 365 and SharePoint in general. He has authored “SharePoint Search Queries Explained” and “Working with FAST Search Server 2010 for SharePoint”.
Show notes:
Full transcript
[Tobias] = Tobias Zimmergren
[Mikael] = Mikael Svenson
[Tobias]: Hi! I’m Tobias Zimmergren. Welcome to the Tech Talk Show. Throughout this podcast, you’ll be able to listen to the latest happenings and updates from the community along with interviews with industry experts talking about their favorite technologies.
In this episode, I’m catching up with a good friend and Microsoft MVP Mikael Svenson. We’re going to talk a bit about Mikael’s experiences with the Office 365 PnP Provisioning Engine and how they use that in a few real world projects to help customers more easily request new sites in their organization. So let’s check that out!
Welcome, Mikael to another episode of The Rencore Tech Talks. I had a chat with you at the MVP Summit and we kind of said it would be a nice idea to record a podcast so I guess here we are.
[Mikael]: Yeah, thanks for having me and I guess we also did a podcast then, but the audio was crap. So let’s throw it over the boat. Let’s do another one.
[Tobias]: Exactly! Recording in the common room where there are two hundred people outside was not a good idea, but lessons learned.
[Mikael]: November. So last year.
[Tobias]: All right, so we’re going to talk a little bit about the PnP Provisioning Engine and for anyone listening, the PnP is the Office 365 Patterns and Practices. What can you tell me about obviously what PnP is for anyone who hasn’t heard of that before, but also what is the Provision Engine specifically inside of the PnP?
[Mikael]: Sure! So Office PnP is a lot of packed up functionality which allows you to work specifically with the SharePoint artifacts, makes it easier to do a lot of the tasks using the client-side object model. That’s all behind the scene so they’re creating a lot of helper wrapper functions to make it easier for developers to get up and running.
[Tobias]: OK, is it only the CSOM? Is it only Client Side Object Model?
[Mikael]: No, they’re also doing stuff with the Microsoft Graph. So there’s code for that. And it’s just expanding. And also JavaScript, so there are more products coming. Since like almost every week the Patterns and Practices is opening up, but I think they also added SharePoint Framework functionality in there. So there’s more and more stuff that’s coming into the PnP. I mean, it’s Patterns and Practices working in Office 365, so whatever goes on there will probably go into that GitHub project I guess.
[Tobias]: Right, why is it important for the Provisioning Engine to exist? What does it help me with, if I’m a customer or if I work with some kind Office 365 or SharePoint development?
[Mikael]: Right, so if you look at the Provisioning Engine in specifics, back in the old days, people have probably saved a site as a template. I mean, you can configure up a site with different libraries, different columns and you want to use that as a template.
[Tobias]: Oh, I’m getting goosebumps now, man.
[Mikael]: Yes, we don’t want to do that anymore. So basically what the Provisioning Engine does is it creates or you can create a recipe – right now the scheme is in XML – where you sort of put in all the columns, all the lists, all the libraries, all the custom actions, whatever branding, anything you need on the side. Basically, you create a team site and then you play back this recipe and then make the site look like you want.
So instead of having a site definition you just apply it, I mean, you create a list, you add the columns, you create the view. It’s basically a playback engine on everything you want to configure.
[Tobias]: So you’re not stuck with the old XML afterward?
[Mikael]: Yes, but you can still configure the site manually and then you can extract it as a PnP template. And then you can use that. You can use it just by default or you can start to modify it if you want. And the PnP products also publish something called the PnP Partner Pack, which sort of encapsulates all this in the portal, but then you need to store your templates in the Azure. There are some certificates that need to setup. So to me, that was a little too complicated or too much hassle to set up.
So I’ve actually done two projects recently and they’re both going to an end. And I decided, well, let’s create a simple provisioning script just having a list where you order up your site and then have an Azure web job pull that list for modified entries and then create the sites and do all the logic. And that also makes it easier for the customer to actually modify the scripts because IT Pros love PowerShell.
[Tobias]: Yeah, tell me about it.
[Mikael]: So that sort of what sparked the provisioning part on my end. And then right when I started so in the dev branch of the Provisioning Engine, they started creating something called PnP Packages. It means instead of having a lot of XML files and all the other site artifacts which could be CSS, some JS files or images just in a folder structure, they pack all that into a .PNP file, which is basically a zip file and that suited my products perfectly.
So what we do now we create team site templates and then we just stuff them in a SharePoint library. And we also create smaller modules so you can have a main module for a site and then you can have a small module, which could be like external documents or a contact list. So it’s just a concept on how you tie this together and it makes it very easy for people to add new functionality through the ordering list instead of on the site directly.
[Tobias]: Right, so the user flow is after you created this kind of template using the Provisioning Engine and you put it inside of SharePoint or available through SharePoint. Could you have done some kind of UI through a list in this example maybe?
[Mikael]: Yeah, we just kept it real simple. So we have a project directory site, which basically has a couple of lists. So you have a list where you order your site. That means you fill in the title, which template you want to use, possibly some metadata. I think we have site owner and it will specify who should be in the site owners and members and the visitors list, the business criticality. We have some metadata as well. It’s different in both products.
And then there is a list with the actual template and also the smaller modules and then it’s all just linked up with lookup lists. This is all created in SharePoint. So we could actually save the project as a PNP template as well so it could be like a meta-meta. You could provision new ordering sites as well if you wanted.
[Tobias]: Okay, nice. Yeah, I guess it depends on how far you want to take that.
[Mikael]: And the fun part with doing all this is love to program. I have been coding for years, but less and less. And since I decided to do this in PowerShell, there were some missing commandlets in the PnP PowerShell. So then I had to do pull requests. So I could’ve actually done a fair bit of coding in order not to code.
[Tobias]: Right, it’s metaprogramming – you’re programming in order not to program.
[Mikael]: I think I had like 12 or 15 pull requests done in January to either add new commandlets or add some form of missing functionality in the core product to get this up and running. So this actually benefits everyone.
[Tobias]: OK, so the work you’ve done with the PnP Provisioning Engine in your own project is now going to benefit the community when you publish that then?
[Mikael]: Yes, I think I only have two pull requests now that haven’t been merged yet. So I think everything should be in the Dev branch so that means it will make it to the Release branch for next month or the month after. And that’s pretty cool.
[Tobias]: So I guess this is a common thing with the PnP Provisioning Engine itself, but it also the entire PnP project. That people use it in their real projects, then they figure things out or they improve stuff and then they just put in a pull request back to the repository so everyone using it can benefit, right?
[Mikael]: Yeah, and I think one of the main contributions I’ve done, which might not benefit the US market as much, but more in Europe where we have a different locale. So that means the sites that are created for one customer have to be in English, Norwegian, Swedish and Danish. And in the client-side object model, you have locale support for title and description, for lists and different artifacts but not everything. So I think I have done for web part title, view title just added support for all the missing parts, which you don’t have in CSOM because you still doable to do that.
[Tobias]: So let’s imagine from a customer perspective you have done improvements to the GitHub project and you’re using this now with your customers or whatever. And as an end user you don’t know, if it’s a Provisioning Engine from PnP, you don’t know, if it is SharePoint built in, whatever. It doesn’t really matter what my experience is, I should be able to request or create a site easily. If you use this with the Provisioning Engine and you have this request list that you talk about, do you have an approval flow or how do you approve requests and make sure that the flow for the user keeps going?
[Mikael]: For one, the customers don’t have approval at all. So that means you just request it and then we will just provision it, but we do check that if you’re not a manager, then the site in the directory will be marked as not compliant. So we still create the site but will send an email saying, well, you haven’t actually done the site owner training, so your site is not compliant, so you should do that training.
For the other site, we decided to use Flow, because Flow has a nice approval email where you basically do approve or decline.
[Tobias]: What is Flow for anyone listening and is not seeing it?
[Mikael]: Yes, right. So Microsoft Flow is a new workflow engine, which is building on logic apps or logic flow in Azure. So it’s using the Azure Workflow Engine. So that will probably take over as the workflow engine in SharePoint Online instead of using the old SharePoint 2010 or 2013 workflows.
[Tobias]: Oh my God, finally.
[Mikael]: Yeah and it’s also heavily being used within PowerApps. That means if you want to do some data processing in a PowerApp, that means not just to receive or store data, you have to kick off a flow in order to do that.
So it’s still missing some functionality and giving some errors here and there. The funny part is the approval workflow actually says that it has an error when it runs, but it still works. You’re probably familiar with the old “you have too many lookup columns in your list” message. We get that message, but it still works.
So what happens then is you request a site and then by default, they will set the state over one. And then when the Provisioning Engine runs, which is this PowerShell script which runs as a triggered web job in Azure, they’ll say, well, the status is one so let’s kick off the flow and it passes the parameter off the site and then everyone will get the email and they can either approve or decline.
If they approve, the flow sets the state to two and you will get a new modified date on the request entry in the list, which will be picked up on the next triggered run over the Provisioning Engine and then they will publish the site and send an email to everyone well, your site is ready, etc. It’s all very simple. I’m trying to keep it as simple as possible architectural wise.
[Tobias]: Well, less is more oftentimes. Right?
[Mikael]: Yes!
[Tobias]: I like that idea that you mentioned – even if everyone can create new sites, they will be flagged as in-compliant or non-compliant. Essentially you haven’t attended the training, you are not allowed to go live with this site because all of that boils down to the governance of your Internet or whatever kind of application you are running.
[Mikael]: Yes. So this PowerShell script is actually called Mr. Provision and then we have another script called Mr. Governance, which runs on daily, weekly and monthly schedules.
[Tobias]: Oh – nice.
[Mikael]: Yeah. Which checks the site. If it is confidential, then we will turn on IRM on all the document libraries, we will check that whoever is in the access request list matches whoever is in the site director, etc. So we have some checking if people change stuff. And then we’ll change it back if you do stuff you’re not supposed to.
[Tobias]: Right. So one thing that I’m kind of thinking about is you can provision SharePoint sites and that’s one of the big things with a Provisioning Engine. Can you provision other things as well using it? You mentioned something with a graph and other things might be coming.
[Mikael]: Yes. I guess the Provisioning Engine itself in the PnP product is basically taking the template and then applying it to an existing site. So you still have to create the site outside of the actual Provisioning Engine. So right now we’re creating team sites using the commander called New PnP Tenant Site, but you could, in theory, if you create a group, you could apply the same templates to a group, but what’s missing right now there’s no web hooks saying when a new group has been created.
No, it wouldn’t be too hard to say okay if in the ordering list you could say you want a group site or just a team site and then we could provision a group instead and then you would just have to pause and then pull to make sure that the team site is ready before you apply the template. So I don’t see too much programme or adding to the script actually support doing both.
[Tobias]: Right. Otherwise you could have your Mr. Groups Checker PowerShell Script, right?
[Mikael]: We could do that. Right. I mean, you could have two content types I guess on the list. You just pull in the items you want depending on the template.
[Tobias]: Nice! To sum it up, in a Provisioning Engine, the benefit is compared to how you did previously in SharePoint, you get the onet.XML file and you’ve got all that madness of hard coded references and stuff like that. Now you just create a site first, which is not related actually to the provisioning but you create the site and whatever site you create, you can apply a template on top of it, which is essentially running through the schema or like a receipt add that list, add that content type, whatever but then that’s not tied to the templates. You don’t have to save the template file lying around, right?
[Mikael]: Correct! And then we also added, since we do add some metadata when you request a site, we actually add those columns automatically to all document libraries on the site and then set the default values to whatever you had in the site directory. So we don’t show the columns, but that means you can, in theory, create such based experiences where you roll up documents created to a certain project without you actually knowing that it’s Project A or B because that was tagged in the original site. So we’re just using the automatic metadata in folders on all of those libraries without the user even knowing about it – sort of easy auto-tagging.
[Tobias]: Cool! I know one thing that is a popular request in relation to this, which you kind of brushed on at the beginning, which is imagine a user is provisioning a new site and they are applying this template and it’s awesome, but they figure out, I’m going to need additional three lists and five content types and they make some changes to the site they just created and then they say, I want this to be the new template. Can they kind of save this site as a template like you did previously in SharePoint then?
[Mikael]: We don’t support that in the framework we created, but if you installed a partner pack, they actually allow you to point to a site and set that as a template. But in my experience, if you take a site, you probably want to clean up the template somehow anyways. So what we’ve decided on for these projects since, I mean the end users are usually not that technical. So we do some learning on the resources who manage the site request on IT, so if someone wants an image library, they would create a new site and then set up a new image library there and then they would create a template from that library and just keep the image library and save that as PnP package. Just clean up the XML.
[Tobias]: And if you use this partner package that you mentioned? You don’t run that separately? Is that something you have to deploy to your tenant?
[Mikael]: Yes, the partner pack runs as a separate website, ASP.net Website, on the MSDN website. So you need to have that running on some web server or somewhere.
[Tobias]: Is it like a provider hosted app?
[Mikael]: No, it’s just a separate app and you just set up the certificates where you can talk to SharePoint to do the stuff it has to do. So we’re using app tokens in our solution, which works. That’s why we decided not to go with a partner pack. So we just keep everything in SharePoint. Except that one Azure or those two Azure web jobs, which does the provisioning and the governance. We try to do it a little bit simpler than the partner pack.
[Tobias]: Right. So I imagine if you’re a big enterprise client and you have super strict governance and all that, maybe it’s beneficial to use that partner pack, but if you are smaller, you know, less is more again. If you want something done with elegance and simple measures, then this sounds like a perfect fit.
[Mikael]: Yeah and I guess the beauty with this, I mean, we have the ordinates, but it doesn’t mean that you have to use a list directly with the list of new formed orders. I mean you can create whatever app anywhere to have more business logic in the actual form, to have a more beautiful form if you want, which will then just save the data into that list, which would be picked up. So you don’t have to expose the list, if you don’t want to.
[Tobias]: OK, you could make a SharePoint framework web part or whatever as a request form, if you want?
[Mikael]: Yeah, that wouldn’t be a problem at all. So we’re just trying to keep it as simple as possible out of the box. That’s SharePoint, SharePoint, SharePoint: Easy configuration all the way!
[Tobias]: Yeah, I like the idea behind that, because my experience with SharePoint developers and myself being one of them in the past focusing a lot on the development aspects of SharePoint, we kind of miss out on the fact that there’s a lot you can do actually out of the box and the less you customize, the more secure or the more stable environment will be.
[Mikael]: And I think when we’re done doing this, I mean, so I sent over to the IT guy of both customers actually. So we sent them the template for the site catalog and then all the different pre-made modules we had and the scripts. I think it took them like one hour to first provision the site and then set up Azure web job and have everything up and running. Probably an hour or less. So it’s easy stuff.
[Tobias]: Yes and I totally love the fact, I’d brushed on that before as well, I totally love the fact that when you have provisioned a site, it’s not tied to that specific template and you’re not tied to the Onet files or things like that and if you remove the template the site fails to work or load. Everything just works, because it’s hardly coupled to any kind of template.
[Mikael]: So what do we do is when we apply the template, we’re sort of just stamping the property bag, whatever PNP files we have provisioned to the site, so you won’t provision new ones. But that means, if you create a Version 2 of your template and you want to apply that to all the sites, you just save a new file called TeamSite_Version2.pnp and then you could just do a quick edit on the list and then just change that for all the items and save it and then they would be picked up again and it will just reapply the new template to what you already have.
[Tobias]: Nice! Cool! Is there anything else? I think I’m all out of ideas for the Provisioning Engine. That’s as far as my knowledge goes.
[Mikael]: Yeah. So I’m finishing up with these two projects. So the goal is to just take the core parts of this and then applies to publish that as a PnP sample. So anyone can just take this and rework it to their needs as an alternative to using the partner pack, if you need provisioning.
[Tobias]: Super! All right, that’s pretty cool. I’m going to keep my eye out on that. Will you announce something on your blog? Where can we see this stuff when it goes live?
[Mikael]: I will definitely publish it on my blog, when I’m done doing it. It’s a matter of organizing all the different tasks I have to do to have time to do it.
[Tobias]: Time management.
[Mikael]: Maybe if I have time next week. I’m attending a Hackathon next week. So maybe I’ll have time to just finish up and polish it and I can use it as part of the Hackathon entry as well.
[Tobias]: Nice! That’s the one in Oslo, isn’t it?
[Mikael]: Yeah, that’s the one Oslo – The Arctic SharePoint Challenge.
[Tobias]: Yeah, nice! We need to have a chat about that in one episode as well because that’s pretty cool.
[Mikael]: Yeah, I’m really looking forward to it. So we’ll see how that goes.
[Tobias]: All right, cool! Well, I don’t have any additional questions for you about the PnP. Do you have anything else you want to tell us about the PnP?
[Mikael]: No. I mean, if you do it in PowerShell, it’s cool. If you’re missing what you need in PowerShell, do a pull request. The PnP project is more than happy to assess and accept good pull requests.
[Tobias]: So the baseline is, if you want to improve or change something, don’t be afraid to do so?
[Mikael]: Yes, don’t keep the code to yourself. I’m going to get most of the code I’ve been doing here. And committing is actually being paid for by these two customers. So it’s actually the customers paying me to contribute to the community and they are super happy about doing it.
[Tobias]: Yeah, I mean, in the end it benefits them as well, because other people are contributing the same way.
[Mikael]: Because they are accessing stuff that someone else has been paying for, which we also use.
[Tobias]: It makes a lot of sense.
[Mikael]: Supercool!
[Tobias]: All right, thank you for coming out to the show and I hope to invite you to another show talking about the Hackathon there at the Arctic SharePoint Challenge.
[Mikael]: Sure!
[Tobias]: All right, cool! See you next time.
[Mikael]: Yeah, see you next time. Bye, thanks for having me.
[Tobias]: That concludes yet another episode of Rencore Tech Talks. Thank you Mikael for your time and thanks to everyone who tuned in to listen. Find out more about Mikael and his experiences by checking out the links in the show notes.
