TL;DR:

  • Semantic Kernel is Microsoft’s SDK for building AI agents and copilot experiences — it runs in C#, Python, and Java, and handles plugin registration, function calling, memory retrieval, and multi-step planning
  • It’s the framework that powers Microsoft’s own Copilot products internally, which means it’s tested at production scale in enterprise environments with Microsoft 365 and Azure integrations baked in
  • If your team is working in .NET or integrating with Azure OpenAI Service, it’s the obvious starting point — for Python-only teams doing data science adjacent work, PydanticAI or LangGraph may be more idiomatic

Most AI agent frameworks share the same basic architecture: a model, some tools, a loop. Where they diverge is in how opinionated they are about structure, how much boilerplate they eliminate, and what ecosystem assumptions they make. Semantic Kernel (SK) sits firmly in the “opinionated and enterprise-oriented” camp, which is exactly right for some teams and wrong for others.

Core Concepts

Semantic Kernel organises everything around three abstractions: the Kernel, Plugins, and Memory.

The Kernel is the central object — it holds your AI model connections, plugin registrations, and configuration. You create one, add your plugins and services, and pass it around. In C#:

var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion(
        deploymentName: "gpt-4o",
        endpoint: Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT"),
        apiKey: Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY"))
    .Build();

Plugins are collections of functions that the AI can call. You define them as C# classes with [KernelFunction] attributes, or Python functions with the @kernel_function decorator:

from semantic_kernel.functions import kernel_function

class OrderPlugin:
    @kernel_function(description="Get order status by order ID")
    async def get_order_status(self, order_id: str) -> str:
        # fetch from your order system
        return f"Order {order_id}: shipped, estimated delivery Thursday"
    
    @kernel_function(description="Cancel an order")
    async def cancel_order(self, order_id: str, reason: str) -> str:
        # cancel in your system
        return f"Order {order_id} cancelled"

The framework serialises these into the tool definitions that OpenAI and Azure OpenAI expect, handles the function call responses, and re-invokes the model with results. You get multi-step tool use without writing the orchestration loop yourself.

Auto Function Calling

SK’s most practically useful feature is auto function calling — you add plugins to the kernel, and when the model decides it needs to call a function, SK handles the entire round-trip automatically. In Python:

from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior

kernel.add_plugin(OrderPlugin(), plugin_name="orders")

response = await kernel.invoke_prompt(
    "What's the status of order #12345 and is it too late to cancel?",
    settings=AzureChatPromptExecutionSettings(
        function_choice_behavior=FunctionChoiceBehavior.Auto()
    )
)

The model calls get_order_status, gets the result, decides it needs more context, potentially calls cancel_order, and returns a coherent response. You didn’t write the loop.

This works well for workflows with well-defined tools and clear stopping conditions. It gets less reliable when you need precise control over execution order or want deterministic behaviour — at which point you want the Handlebars or Stepwise planners, which execute a fixed plan rather than trusting the model to sequence tool calls correctly.

Memory and RAG

SK has a built-in memory abstraction for retrieval-augmented generation. You can connect a vector store (Azure AI Search, Qdrant, Pinecone, Chroma, or an in-memory store for testing), add documents or records, and query semantically:

from semantic_kernel.memory import SemanticTextMemory, VolatileMemoryStore

memory = SemanticTextMemory(storage=VolatileMemoryStore(), embeddings_generator=embedding_service)
await memory.save_information("policies", id="vacation_policy", text="Employees accrue 25 days annual leave...")

results = await memory.search("policies", "how many days off do I get")

The memory module handles chunking, embedding generation, and retrieval. For production use you’d swap VolatileMemoryStore for your actual vector store — the interface is consistent across backends.

Azure and Microsoft 365 Integration

The enterprise differentiation is most obvious here. SK ships with connectors for Azure AI Search, Azure Blob Storage, Microsoft Graph (accessing calendar, email, Teams data), and SharePoint. If you’re building a copilot that needs to answer questions about internal documents in SharePoint or schedule meetings via Graph API, these connectors are already built rather than something you write yourself.

The Microsoft.SemanticKernel.Plugins.MsGraph package gives you Calendar, Email, Files, and Tasks plugins out of the box — the same building blocks Microsoft uses in Copilot for Microsoft 365.

When to Use Semantic Kernel vs Alternatives

Use SK when:

  • Your team is primarily .NET/C# — it’s the best-supported language by a significant margin
  • You’re deploying to Azure and want native integration with Azure OpenAI, AI Search, and Microsoft services
  • You’re building enterprise copilots that need Microsoft 365 access
  • You want a framework that’s tested at Microsoft-scale production workloads

Consider LangGraph when:

  • You need explicit control over agent state and transitions — LangGraph’s graph-based architecture is better for complex multi-agent workflows where you can’t let the model sequence everything
  • Your team is Python-native and already uses the LangChain ecosystem

Consider PydanticAI when:

  • You want a minimal, type-safe Python framework without the abstraction weight of a full platform
  • You’re building simpler agents where SK’s enterprise features are overhead rather than value

Roll your own when:

  • Your agent is simple enough that a framework adds more complexity than it removes — a model plus three tools called in a deterministic order doesn’t need a framework

The honest answer for most enterprise development teams building on Azure: Semantic Kernel is the right default. It’s maintained by Microsoft, it’s what Microsoft’s own products run on, and the Azure integration isn’t something you’d want to build yourself. For Python-only teams doing more research-oriented AI work, the Python ecosystem alternatives may feel more natural.