Azure AI Language Service

Azure AI Language Service is a service hosted on the cloud, offering capabilities in Natural Language Processing (NLP) to comprehend and dissect text. It supports summarization, entity linking, key phrase extraction, and many other features. It shares many features with OpenAI, albeit in a much simpler form.

In this post, I’m going to build an Azure Function that wraps the Azure AI Language Service into a nice and simple API. Then we could connect this API to our other LLM capabilities like RAG solutions or maybe even to chatbots and detect what are the user intentions for the input.

Azure AI Language Service

Azure AI Language Service is relatively new service, but it actually combines many old services like Azure Text Analytics into one simple language service. It supports wide range of languages all the way from Afrikaans to Zulu. You can view the full list of supported language from Microsoft Learn. The best part of this service is, that you can try it for free. The free-tier offers 5000 transactions per month (30 days) and it does not have any upfront costs.

There are different ways to utilize this service, but for simplicity I’m going to use Azure.AI.TextAnalytics NuGet package in this sample.

The Azure Function

Let’s start by creating a new Azure Function project. I will use .NET 8.0 in this sample, but you can use any supported version you like. I named my function as “ExtractFunction“.

We need two things to use the language service: URI and credentials. For credentials we are going to use AzureKeyCredential, but you can also use DefaultAzureCredential. For AzureKeyCredential we will need the API key from Azure Portal. You can find the endpoint URI from the same page.

There are different client classes in NuGet package for different Language Service features. For key phrase extraction, we need to use the TextAnalyticsClient. TextAnalyticsClient is simple class, that only requires endpoint URI and credential objects. We can use the values retrieved from Azure Portal to build those entities.

After creating client we can call the ExtractKeyPhrases method with target text and language of our choice. For this Azure Function example, we will read the text from body of the HTTP request and use English as our default language. ExtractKeyPhrases returns collection of words that we will return to caller.

[Function("Extract")]
public async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Function, "get", "post")] HttpRequest req)
{
    _logger.LogInformation($"Extracting words from {req.Body}");
    
    Uri endpoint = new("{uri}");
    AzureKeyCredential credential = new("{credentials}");
    TextAnalyticsClient client = new(endpoint, credential);

    if (req.Body is null && req.Body?.Length <= 0)
    {
        return new BadRequestObjectResult("Please pass a document in the request body");
    }

    var body = await (new StreamReader(req.Body).ReadToEndAsync());
    Response<KeyPhraseCollection> response = await client.ExtractKeyPhrasesAsync(body, "en");
    KeyPhraseCollection result = response.Value;

    return new OkObjectResult(result);
}

HTTP Test

To test the code I added tests.http file into project. We can use this file to easily test our Azure Function. The test will call API with text value of “I have this machine called PM-6, what bearing it has in lower section?” in the body. You can type any string you like and the string can be almost as long as you want to. Just remember to fix the port if you are not running your local azure function in port 7244. You can check the port from Azure Function terminal window.

POST http://localhost:7244/api/Extract
Content-Type: application/json

{
    "I have this machine called PM-6, what bearing it has in lower section?"
}

Now let’s run our function and run the test from .http file. We can do that simply from Visual Studio, by clicking the Send request text above the test.

As we can see on the result window, it returns 200 OK and words “lower section” and “machine“. In this sample I think it missed two important words, which are the bearing and PM-6. Anyway we could now use this key phrase extraction to project actions against the user input. For example we could now raise an adaptive card at Teams, which shows more information about lower section of machines or something like that. With the key phrase extraction, we can try to guess what is the user intention. What the user is interested of and what he or she tries to achieve.

Summary

Azure Language Service offers key phrase extraction as part of its Natural Language Processing (NLP) capabilities. This feature enables the automatic identification and extraction of key phrases or keywords from text, helping in understanding the main points of interest or themes within the text. Service is easy to use through its SDK’s, which are available for C#, Java, JavaScript and Python languages. The service supports multiple different languages and can be tested free of charge.