Microsoft’s Semantic Kernel is currently a focal point in the Microsoft AI landscape. The Semantic Kernel (referred to as SK) serves as an SDK enabling the development of applications utilizing Large Language Models (LLMs). SK facilitates seamless integration of OpenAI or Azure OpenAI into applications, empowering developers to construct AI pipelines capable of executing multiple LLM calls, triggering actions, and executing custom code based on LLM responses. However, the reality is more complex than it may initially appear.
Basics of Semantic Kernel
Semantic Kernel provides tools (well classes and methods…) to interact with OpenAI (or Hugging Face LLM’s). This is done through a class called Kernel, that can be instantiated with class called KernelBuilder. KernelBuilder is responsible for wiring things down and Kernel is used to invoke LLM methods. Kernel has many other abilities, which makes it more than just a invoker of LLM methods. For example we chain multiple LLM calls, react on response messages and even build our own semantic memory. The power to invoke actions and react on LLM responses is the salt of the Semantic Kernel. You don’t have to write if-else statements to do that, which makes it an AI product.
Microsoft has kindly provided this picture of Semantic Kernels Kernel in SK GitHub Page. I don’t find it super intuitive, but it tries to describe, that Kernel can contain a pipeline (or chain) which flows from left to right. Pipeline works with the Kernel context to store and retrieve contextual data. For example we can invoke a OpenAI query, store the result into memory, read PDF file and merge part of that PDF file with OpenAI response. These all can happen when Kernel is invoked once.
Simple Example
To provide a better understanding of Semantic Kernel, I created a small sample code. This very simple example creates a Kernel object with Azure OpenAI capability, makes a query and returns result. You need to provide Azure Open AI endpoint, apikey and model name to run this example.
// Create kernel object
var kernel = Kernel.CreateBuilder()
.AddAzureOpenAIChatCompletion(
deploymentName: "{your Azure OpenAI deployment name}",
new OpenAIClient(new Uri("{AOAI_API_ENDPOINT}"), new AzureKeyCredential("{AOAI_API_KEY}")))
.Build();
var promptGenerator = KernelFunctionFactory.CreateFromPrompt("{Fill your prompt here}");
var result = await promptGenerator.InvokeAsync(kernel);
var resultString = result.GetValue<string>();
As seen in example above the SK does not need much of code to run simple questions against the OpenAI. However this is no magic (or much of an AI) in this sample. The real benefit of SK comes from combining the LLM questions (and answers) into a pipeline, that can invoke actions and hold memories. We could extend this example by using SK’s plugins and planners to orchestrate the pipeline.
If you want to read more about building more complex AI solutions, check out the Planners page from Microsoft Learn. That page explains really well how you can combine multiple LLM queries and actions to build solutions that requires more than one simple LLM query.
Semantic Kernel vs. LangChain
I have spent last few days in learning the SK and principals behind it and I noticed, that tinkering with SK requires a lot of work, because SK samples and documentations are bit scattered currently. You can find some at Microsoft Learn, some at SK GitHub page and even more at Azure GitHub page… This project has evolved quite a bit in short time, which means that not all the documentation or samples are valid anymore. I encountered some code examples that no longer functioned due to changes in the SK SDK over time, or because the models underlying the SK do not support all the features.
When I first started doing a simple SK project, that converts natural language questions into SQL queries and runs them against the DB, it took me approx. three hours to complete. I can do that same thing with LangChain in 30 minutes (after spending 20 minutes of brawling with SQLAlchemy) and I have done C# programming for over 10 years now. Contrast to that, my Python coding experience is less than a year. That is quite a big difference in implementation times IMHO.
When I tried to do a bit more complex stuff with SK, I quickly encountered issues like FunctionCallingStepwisePlanner not working with Azure OpenAI offered model versions (because they lack tools API) and random unhandled HallucinatedHelpers exceptions. SK is still in preview and that is clearly visible when you add the SK nuget package into your project. You will quickly receive a bunch of “evaluation purposes only” errors, that you need to ignore.
I still struggle to understand when I should use SK over LangChain. SK has a great promise with it’s planners and plugin architecture, but still I find the LangChain better, more approachable and well documented. There is this good comparison page at GitHub, that compares SK against the LangChain. Some of it’s points are now old and invalid, but still it gives a good overview how things are now. LangChain has 76 800 stars at GitHub (when this post was written) against the SK 16 800. 11 800 forks against 2 500. I know Semantic Search is a newer project, but still LangChain has quite heavy foothold as LLM orchestrator. One selling point of Semantic Kernel could be that it embraces C#, but who wants to do AI/ML stuff with C#? Python has way better libraries and ecosystem in that area and it also has very good libraries to handle data.
Summary
Semantic Kernel is a fascinating tool. It allows C# developers to orchestrate LLM’s in a very similar way as LangChain allows Python developers. However I don’t know if C# is the right tool for AI/LLM stuff, or should we stick with the Python as it is currently the main language for these things? Time will tell how SK evolves, but at it’s current state it is not suitable for production use (as their nuget packages also clearly states)., but it can be used to build cool demo’s and play with different LLM solutions.
The future of large language models does not solely lie in chat UIs. Instead, I believe we will witness the emergence of solutions that integrate process automation, LLMs, and other AI capabilities to create solutions that makes our lives easier and improves our productivity. For that, we need tools like Microsoft Semantic Kernel.