Leverage Azure Machine learning in SharePoint

One of the fun things about working with Microsoft products nowadays is the frequency in which they release new features and applications. It’s definitely true when it comes to Azure. Almost every week there’s a new blog or how-to which we can learn something unique from.

One of the new offerings in Azure is Azure Machine Learning. Cloud-based, it lets you build, deploy and share advanced analytics. Which got me thinking (dangerous I know!). Having played around little more with Azure recently I wanted to find a way of leveraging it further in SharePoint or Office 365.

Create your own neural networks with Azure Machine Learning

First of all I thought about machine learning. While computers are undoubtedly complex (any coder or dev will agree), they are also quite basic. They only do what we tell them. Unlike most of us they don’t learn from their mistakes. The idea behind machine learning is that they become more intelligent and learn from previous information by adapting the output, based on data and a configured neural network. For example, machine learning is being used in search engines. The more people click on a certain result, the higher that particular item is ranked in subsequent queries.

Machine learning uses a configured neural network to be able to train itself, learn from previous data, and generate an output. Previously difficult to set up and only achievable if you were highly trained, Azure Machine Learning is available to a much wider audience.

For all of us this is great. As you can now leverage one of the many available Azure Machine Learning API’s in your apps. One of which is the Text Analytics API, which is what I want to look at today.

Putting Text Analytics to work in SharePoint

The Text Analytics API, part of Microsoft Azure Machine Learning platform, allows devs to submit text to the Text Analytics API. The API then analyzes this text, and is able to perform sentiment analysis, or keyword extraction. For example, it is able to determine whether a sentence like “Azure Machine Learning is a very powerful tool” is positive or negative. Pretty cool!

In this post I’m going to look at keyword extraction. I thought it would be really useful to integrate this with SharePoint in the following scenario:

Every time a user saves a content page (for example on the Intranet), the Text Analytics API is used to extract all the keywords from this page, and saves it to a different field in the content page. This information can then be used for automatic tagging, tag clouds, and generally improving the search experience.

This sort of application could be taken even further, especially when the API gets more advanced. Not only could keywords be analyzed, but also sentiment, tone and maybe even quality. The sort of application I will describe in this post could form part of an automatic ‘quality’ assessment for Intranet content. Goodbye to poor content or inappropriate messages, the app could scan for problems in advance. This sort of stuff has previously only been available in very specialist environments or for a lot of money. Now it could be part of your SharePoint environment! Let’s get started.

Sign up for Text Analytics API

To get going, the first step is to sign up for the Text Analytics API via the Azure Marketplace. The API is free for up to 10k transactions / month. Click the “Sign up” button to continue.

Azure Machine learning

Fill in the form and accept the terms & conditions.

Next, probably because of a glitch in my browser (shouldn’t use Edge I guess 😉 ), I had to click “Sign up” again and again accept terms & conditions. Eventually it should show this under My Account -> My Data:

Azure Machine learning

The next step is to copy your primary account key somewhere safe. Click “My account” in the header:

Azure Machine learning

Copy the primary account key. You’ll need it later!

Create the SharePoint Provider Hosted App

Now I am going to create a provider hosted app which registers a remote event receiver in the host web, not in the app web. Registering a remote event receiver in the host web cannot be done declarative, it must be done in code. To get started, I use this excellent OfficeDevPNP sample from Vesa Juvonen and Kirk Evans (both Microsoft).

Azure Machine learning

To be able to debug a remote event receiver, an Azure Service Bus must be registered first. Refer to TechNet for the full explanation how to do this, but essentially you need to do two things:

1. Go to the Azure Management Portal and click on “Service Bus” on the lefthand side.

Azure Machine learning

Click + in the footer to create a new Service Bus.

Azure Machine learning

2. Click “Connection Information” and copy the Connection String.

Azure Machine learning

Paste this in the project properties of the Core.EventReceivers SharePoint add-in project-> SharePoint in Visual Studio.

Azure Machine learning

A service bus is only required for debugging purposes, for the reason that Office 365 cannot access your debugging environment directly, as that most likely will be running on your localhost. So, Microsoft has come up with this smart way of leveraging Azure Service Bus as in fact the man in the middle.

To focus on the Azure Text Analytics code, I leave the code in the OfficeDev.PnP sample to set up a remote event receiver for the ItemAdded event in a newly created list. Though I don’t like the title “Remote Event Receiver Jobs”, so let’s change that to “Text Analytics Demo” first. Change the following line in RemoteEventReceiverManager.cs:

to

I also need to set the URL to my Office365 developer tenant. Select the SharePoint add-in project and configure the Site URL under SharePoint:

Azure Machine learning

At the end I give the app also a nicer name in the AppManifest.xml 🙂

This should now be good to go! Let’s see that it works, hit F5, and wait for the Trust window to pop up.

Azure Machine learning

Click “Trust it” to trust the app and continue. After the app has been installed (this may take a minute or so), open your SharePoint site and verify the list has been created.

Also, make sure you are logged into your developer tenant in your default browser, Chrome in my case. Otherwise it does not work properly either.

Azure Machine learning

The next thing is to change the code in the ItemAdded event receiver.

Go to RemoteEventReceiverManager.cs and add the following usings

Then replace  method ItemAddedToListEventHandler with the following. I’ll add the Text Analytics API magic here.

Make sure to paste your text analytics primary account key that we copied at the very beginning into the authorizationKey variable in line 4.

Then on your dev site, delete the “Text Analytics Demo” list from your site before continuing. When debugging, event receivers are not cleaned up after stopping debugging. An easy fix in our case is to just delete the list. Re-install the app by hitting F5, and go to the list to see Azure Text Analytics API in action!

Add a new item, and enter as Title “SPCAF analyses SharePoint farm solutions, addins, executables and assemblies and checks .NET Code, JavaScript, XML, ASPX, and CSS for correctness, best practices, security, performance etc. (some subliminal product placement here 😉 )

Azure Machine learning

After hitting save, the description field should have the keywords now:

Azure Machine learning

In the same way as I have done here an event receiver could run when a page of content is saved, and the keywords stored in a list somewhere. A ‘tag cloud’ app part could then be configured to display these words on the page somewhere.

You can download the source code of this sample from my GitHub account.

Easily automate text analysis

What I’ve shown today is that with remote event receivers you can enrich your SharePoint data with the Azure Text Analytics API, by extracting keywords. There’s a ton of other ways to play with and maximize the available Azure Machine Learning APIs. One idea might be to use the sentiment analysis, which could analyze whether a user’s input is positive or negative. The API would be leveraged through a workflow instead of a remote event receiver. That’s one I might look at in the future!

Enjoy 🙂

About the author

Matthias is a SharePoint Server MVP and the CEO of RENCORE. Working as a SharePoint developer and architect since 2005 he and his colleagues realized the tooling gaps in the SharePoint Application Lifecycle Management process and created several OpenSource projects eg. the SharePoint Software Factory and the SharePoint Solution Deployer, which eventually led to the creation of SPCop and SPCAF. Matthias is speaking at many conferences and community events and is one of the organizers of SharePoint Saturday Stockholm, SharePoint Saturday Munich and the SP24 online SharePoint conference.