Showing posts with label Lesson. Show all posts
Showing posts with label Lesson. Show all posts

Monday, October 5, 2026

ASP.NET 10.0 Minimal WebAPI with SQLite & Swagger

This tutorial is about using ASP.NET 10.0 Minimal WebAPI. We will build a simple application with SQLite to save students data. We will then consume the API from an HTML page and add a Swagger interface to test the API endpoints.

Requirements:

  • .NET 10.0
  • C# Dev Kit VS Code Extenstion

In a suitable working directory, create a Web API web application in the terminal window with:

dotnet new webapi -o StudentsMinApi 
cd StudentsMinApi
mkdir Data
mkdir Models
mkdir wwwroot code . dotnet watch

This will appear in your terminal window:

Point your browser to the URL displayed in the terminal window, followed by /weatherforecast.

In the above case the address would be http://localhost:5130/weatherforecast. This would display the following in your browser:

Look at the API code in Program.cs:

app.MapGet("/weatherforecast", () => {
    var forecast =  Enumerable.Range(1, 5).Select(index =>
        new WeatherForecast (
            DateOnly.FromDateTime(DateTime.Now.AddDays(index)),
            Random.Shared.Next(-20, 55),
            summaries[Random.Shared.Next(summaries.Length)]
        ))
        .ToArray();
    return forecast;
})
.WithName("GetWeatherForecast");

Add this Entity Framework tool if you do not already have it: 

dotnet tool install --global dotnet-ef 

Let us add these packages that provide support for SQLite and Swagger: 

dotnet add package Microsoft.EntityFrameworkCore.Design
dotnet add package Microsoft.EntityFrameworkCore.Tools
dotnet add package Microsoft.EntityFrameworkCore
dotnet add package Microsoft.EntityFrameworkCore.SQLite
dotnet add package Microsoft.EntityFrameworkCore.SQLite.Design
dotnet add package Swashbuckle.AspNetCore

Inside the Models folder, add the following Student class: 

public class Student {
    public int StudentId { get; set; }
    public string? LastName { get; set; }
    public string? FirstName { get; set; }
    public string? School { get; set; }
    public string? Gender { get; set; }
    public DateTime? DateOfBirth { get; set; }
}

Developers prefer having sample data when building data driven applications. Therefore, we will create some sample data to ensure that our application behaves as expected. Copy the following data from https://gist.github.com/medhatelmasry/2ae7a36d5392f862b0d1039b2a52b9b4 and save it into a text file wwwroot/students.csv.

Add the following connection string to your  appsettings.json file:

"ConnectionStrings": {
  "DefaultConnection": "DataSource=school.db;cache=shared;"
},

Next, we need to add an Entity Framework context class. Inside the Data folder, add a class file named ApplicationDbContext with the following content: 

public class ApplicationDbContext : DbContext {

    public DbSet<tudent> Students => Set<Student>();

    public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
            : base(options) { }

    protected override void OnModelCreating(ModelBuilder modelBuilder) {
        base.OnModelCreating(modelBuilder);
        modelBuilder.Entity<Student>().HasData(GetStudents());
    }

    private static IEnumerable<Student> GetStudents() {
        string[] p = { Directory.GetCurrentDirectory(), "wwwroot", "students.json" };
        var filePath = Path.Combine(p);

        var json = File.ReadAllText(filePath);
        var data = System.Text.Json.JsonSerializer.Deserialize<List<Student>>(json) ?? new List<Student>();

        return data;
    }

}

In the above code, student data is being seeded in the OnModelCreating() method by reading contents of students.json file.

We need to register the context class (ApplicationDbContext) with dependency injection in Program.cs. Add the following code right before “var app = builder.Build();” in Program.cs: 

var connStr = builder.Configuration.GetConnectionString("DefaultConnection")
    ?? throw new InvalidOperationException("Connection string 'DefaultConnection' not found.");

builder.Services.AddDbContext<ApplicationDbContext>(option => option.UseSqlite(connStr));

Let us add a migration and subsequently update the database. Execute the following CLI commands in a terminal window:

dotnet ef migrations add M1 -o Data/Migrations
dotnet ef database update

At this point the database and tables are created.

Students API

Let us add API endpoints that:

  • Read all the students
  • Read student data by id
  • Add student data
  • Update student data
  • Delete student data

This can easily be done using the scaffold tool. Make sure you install the tool globally with this terminal window command:

dotnet tool install --global Microsoft.dotnet-scaffold

Once you have installed the scaffold tool, enter the following command to start the tool at the root of your project:

dotnet scaffold

Make the choices as shown in the various steps below:

 

A file named StudentsApi.cs gets created in the root folder of your project with this code:

using Microsoft.AspNetCore.Http.HttpResults;
using Microsoft.EntityFrameworkCore;
using StudentsMinApi.Data;
using StudentsMinApi.Models;

public static class StudentsApi {
    public static void MapStudentEndpoints(this IEndpointRouteBuilder routes) {
        var group = routes.MapGroup("/api/Student").WithTags(nameof(Student));

        group.MapGet("/", async (ApplicationDbContext db) => {
            return await db.Students.ToListAsync();
        })
        .WithName("GetAllStudents");

        group.MapGet("/{id}", async Task<Results<Ok<Student>, NotFound>> (int studentid, ApplicationDbContext db) => {
            return await db.Students.AsNoTracking()
                .FirstOrDefaultAsync(model => model.StudentId == studentid)
                is Student model
                    ? TypedResults.Ok(model)
                    : TypedResults.NotFound();
        })
        .WithName("GetStudentById");

        group.MapPut("/{id}", async Task<Results<Ok, NotFound>> (int studentid, Student student, ApplicationDbContext db) => {
            var affected = await db.Students
                .Where(model => model.StudentId == studentid)
                .ExecuteUpdateAsync(setters => setters
                .SetProperty(m => m.StudentId, student.StudentId)
                .SetProperty(m => m.LastName, student.LastName)
                .SetProperty(m => m.FirstName, student.FirstName)
                .SetProperty(m => m.School, student.School)
                .SetProperty(m => m.Gender, student.Gender)
                .SetProperty(m => m.DateOfBirth, student.DateOfBirth)
        );

            return affected == 1 ? TypedResults.Ok() : TypedResults.NotFound();
        })
        .WithName("UpdateStudent");

        group.MapPost("/", async (Student student, ApplicationDbContext db) => {
            db.Students.Add(student);
            await db.SaveChangesAsync();
            return TypedResults.Created($"/api/Student/{student.StudentId}",student);
        })
        .WithName("CreateStudent");

        group.MapDelete("/{id}", async Task<Results<Ok, NotFound>> (int studentid, ApplicationDbContext db) => {
            var affected = await db.Students
                .Where(model => model.StudentId == studentid)
                .ExecuteDeleteAsync();

            return affected == 1 ? TypedResults.Ok() : TypedResults.NotFound();
        })
        .WithName("DeleteStudent");
    }
}

Note the following mapping code in your Program.cs file:

app.MapStudentEndpoints();

You can start the WebAPI app and view all students at endpoint: /api/student.

OPTIONAL: If you want migrations to be applied automatically, add the following code to Program.cs right before the last “app.Run()” statement

using (var scope = app.Services.CreateScope()) {
    var services = scope.ServiceProvider;

    var context = services.GetRequiredService<ApplicationDbContext>();    
    context.Database.Migrate();
}

Adding Swagger Support

It is very easy to add Swagger support to our application so that we can test the various endpoints that our WebAPI application has to offer. 

Add the following code right under app.MapOpenApi():

app.UseSwaggerUI(options =>
{
        options.SwaggerEndpoint("/openapi/v1.json", "My WebAPI");
});

In file Properties/launchSettings.json, set the value of launchBrowser to true in two places:

"launchBrowser": true

Also, add the following property to both the http and https JSON blocks:

"launchUrl": "swagger"

Restart the app by typing dotnet watch in the terminal window. The app will automatically load into your default browser in a Swagger interfact that looks like this:

Go ahead and test all the available student endpoints.

You may encounter the following error when you try any of the endpoints that require an ID parameter - example: /api/Student/{id} endpoint:

This error can easily be resolved by targeting OpenAPI version 3. In Program.cs, replace "builder.Services.AddOpenApi();" with:

builder.Services.AddOpenApi(options => {
    options.OpenApiVersion = OpenApiSpecVersion.OpenApi3_0;
});

CORS (Cross-Origin Resource Sharing)

In wwwroot folder, create a file named show.html and add to it this HTML/JavaScript code:

<!DOCTYPE html>
<html>
  <html>
    <head>
      <meta charset="utf-8" />
      <title>Test API</title>
    </head>
    <body>
      <h3>Test API</h3>
      <button id="btnGetData">Get Data</button>
      <pre id="preOutput"></pre>
      <script>
        const url = "PUT-API-URL-HERE";

        var showResponse = function (object) {
          document.querySelector("#preOutput").innerHTML = JSON.stringify(
            object,
            null,
            4
          );
        };

        const button = document.querySelector("#btnGetData");
        button.addEventListener("click", (e) => {
          getData();
        });

        var getData = async function () {
          await fetch(url)
            .then((response) => {
              return response.json();
            })
            .then((data) => {
              showResponse(data);
            });

          return false;
        };
      </script>
    </body>
  </html>
</html>

Replace PUT-API-URL-HERE with the URL that gets all the students (Example: http://localhost:5130/api/students). 

From the file system, double-click on the wwwroot/show.html file. You will see the following page:


When you click on the “Get Data” button, nothing will appear because there is a JavaScript error. To understand where this error is coming from, hit F12 in your browser and check the console. This error will appear:

We need to enable CORS in the WebAPI project. This is done by adding the following code in Program.cs just before “var app = builder.Build();”:
 
// Add Cors
builder.Services.AddCors(o => o.AddPolicy("Policy", builder => {
  builder.AllowAnyOrigin()
    .AllowAnyMethod()
    .AllowAnyHeader();
}));

Also, in the same Program.cs file, add this code just after “var app = builder.Build();”: 

app.UseCors("Policy");

Save your code then make a new request for data from show.html. This time you should be successful:


The application should work as expected.

Congrats for coming this far.

Sunday, October 19, 2025

Small Language Models with AI Toolkit Extension in VS Code

In this article, we will see how we can work with small language models (SLM) from the AI Toolkit extension in VS Code. Though the toolkit can do other things, our focus is to consume an ONNX SLM hosted on Visual Studio Code from a C# application. We will first look at an example that is based on OpenAI packages. We will later use a similar example based on the Sematic Kernal approach.

Companion Video: https://youtu.be/V_eWAM2fxJg

Prerequisites

You will need:

  • The latest version of VS Code
  • .NET version 9.0 or higher

What are small language models (SLMs)?

Small Language Models (SLMs) are compact versions of large language models (LLMs), designed to deliver strong performance in natural language tasks while using significantly fewer computational resources.

What is the AI Toolkit Extension in VS Code?

The AI Toolkit Extension for Visual Studio Code is a powerful, all-in-one environment for building, testing, and deploying generative AI applications—especially useful for developers working with small language models (SLMs).

Getting Started

Install the following Visual Studio Code extension:


Click on the three dots (...) in the left navigation of VS Code, and choose "AI Toolkit".

Click on "Model Catalog".

Scroll down down the list until you find “Local Models” >> ONNX >> Minstral 7B – (CPU – Small, Standard) >> + Add Model.

Once the model is fully downloaded, it will appear under Models >> ONNX.

Right-click on the model and select “Copy Model Name”.

I copied the following name for the "Minstral 7B" model: 

mistral-7b-v02-int4-cpu

Using OpenAI packages

Create a C# console application named AIToolkitOpenAI and add to it required packages with the following terminal window commands:

dotnet new console -n AIToolkitOpenAI
cd AIToolkitOpenAI
dotnet add package OpenAI

Start VS Code with:

code .

Click on the "AI Toolkit" tab in VS Code and make sure that the "Minstral 7B" model is running.

Replace content of Program.cs with this code:

using OpenAI;
using OpenAI.Chat;
using System.ClientModel;
using System.Text;

var model = "mistral-7b-v02-int4-cpu";
var baseUrl = "http://localhost:5272/v1/"; // root URL for local OpenAI-like server
var apikey = "unused";

OpenAIClientOptions options = new OpenAIClientOptions();
options.Endpoint = new Uri(baseUrl);
ApiKeyCredential credential = new ApiKeyCredential(apikey);
ChatClient client = new OpenAIClient(credential, options).GetChatClient(model);

// Build the prompt
StringBuilder prompt = new StringBuilder();
prompt.AppendLine("You will analyze the sentiment of the following product reviews.");
prompt.AppendLine("Each line is its own review. Output the sentiment of each review in");
prompt.AppendLine("a bulleted list and then provide a general sentiment of all reviews.");
prompt.AppendLine();
prompt.AppendLine("I bought this product and it's amazing. I love it!");
prompt.AppendLine("This product is terrible. I hate it.");
prompt.AppendLine("I'm not sure about this product. It's okay.");
prompt.AppendLine("I found this product based on the other reviews. It worked");

// send the prompt to the model and wait for the text completion
var response = await client.CompleteChatAsync(prompt.ToString());
// display the response
Console.WriteLine(response.Value.Content[0].Text);

Run the application with:

dotnet run

The application does sentiment analysis on what customers think of the product.

This is a sample of the output:

* I bought this product and it's amazing. I love it!: Positive sentiment
* This product is terrible. I hate it.: Negative sentiment
* I'm not sure about this product. It's okay.: Neutral sentiment
* I found this product based on the other reviews. It worked for me.: Positive sentiment

General sentiment: The reviews contain both positive and negative sentiments. Some customers expressed their love for the product, while others expressed their dislike. Neutral sentiment was also expressed by one customer. Overall, the reviews suggest that the product has the potential to elicit strong feelings from customers, both positive and negative.

Sematic Kernel packages

Create a C# console application named AIToolkitSK and add to it required packages with the following terminal window commands:

dotnet new console -n AIToolkitSK
cd AIToolkitSK
dotnet add package Microsoft.SemanticKernel

Start VS Code with:

code .

Click on the "AI Toolkit" tab in VS Code and make sure that the "Minstral 7B" model is running.

Replace content of Program.cs with this code:

using System.Text;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI;

var model = "mistral-7b-v02-int4-cpu";
var baseUrl = "http://localhost:5272/v1/";
var apikey = "unused";

// Create a chat completion service
var kernel = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion(modelId: model, apiKey: apikey, endpoint: new Uri(baseUrl))
    .Build();
var chat = kernel.GetRequiredService<IChatCompletionService>();
var history = new ChatHistory();
history.AddSystemMessage("You are a useful chatbot. Always reply in a funny way with short answers.");
var settings = new OpenAIPromptExecutionSettings
{
    MaxTokens = 500,
    Temperature = 1,
};

while (true)
{
    Console.Write("\nUser: ");
    var userInput = Console.ReadLine();
    if (string.IsNullOrWhiteSpace(userInput)) break;

    history.AddUserMessage(userInput);

    var responseBuilder = new StringBuilder();
    Console.Write("\nAI: ");
    await foreach (var message in chat.GetStreamingChatMessageContentsAsync(userInput, settings, kernel))
    {
        responseBuilder.Append(message);
        Console.Write(message);
    }
}

This is a simple chat completion app.

Run the application with:

dotnet run

My prompt was:

Red or white wine with beef steak?

The response was:

AI:  Both red and white wines can pair well with beef steak, but a red wine is generally the more traditional choice. Red wines, such as Cabernet Sauvignon, Merlot, or Pinot Noir, have flavors that complement the rich and savory flavors of beef. However, if you prefer a lighter taste, a white wine such as Pinot Noir or Chardonnay can also work well with beef steak. Ultimately, it comes down to personal preference.

Conclusion

We have seen how to use SLMs hosted by VS Code through the AI Toolkit extension. We were able to communicate with the model from these two C# applications: (1) a app the uses OpenAI packages, and (2) an app that uses Sematic Kernel.

Sunday, February 2, 2025

Using PHP with AI models hosted on GitHub

Overview

In this article I will show you how you can experiment with AI models hosted on GitHub in a simple PHP web app. GitHub AI Models are intended for learning, experimentation and proof-of-concept activities. The feature is subject to various limits (including requests per minute, requests per day, tokens per request, and concurrent requests) and is not designed for production use cases.

Prerequisites

To proceed, you will need the following:

  • PHP - You need to have PHP version 8.3 (or higher) installed on your computer. You can download the latest version from https://www.php.net/downloads.php.
  • Composer – If you do not have Composer yet, download and install it for your operating system from https://getcomposer.org/download/.

Getting Started

There are many AI models from a variety of vendors that you can choose from. The starting point is to visit https://github.com/marketplace/models. At the time of writing, these are a subset of the models available:


For this article, I will use the "DeepSeek-R1" beside the red arrow above. If you click on that model, you will be taken to the model's landing page:

Click on the green "Get API key" button.


The first thing we need to do is get a 'personal access token' by clicking on the “Get developer key” button.

Choose 'Generate new token', which happens to be in beta at the time of writing.


Give your token a name, set the expiration, and optionally describe the purpose of the token. Thereafter, click on the green 'Generate token' button at the bottom of the page.


Copy the newly generated token and place it is a safe place because you cannot view this token again once you leave the above page.

Let's do some PHP coding

In a working directory, create a sub-directory named PHP-GitHub-AI inside a terminal window with the following command:

mkdir PHP-GitHub-AI

Change into the newly created directory named PHP-GitHub-AI with:

cd PHP-GitHub-AI

In the PHP-GitHub-AI folder, create a file named index.php and add to it the following code:

<?php

// Set your Azure API key and endpoint
$apiKey = 'PUT-YOUR-PERSONAL-ACCESS-TOKEN-FROM-GITHUB-HERE';
$endpoint = 'https://models.inference.ai.azure.com';

// Define the API endpoint
$url = $endpoint . '/chat/completions';

// Set up the data for the API request
$data = [
    'messages' => [
        [
            'role' => 'system',
            'content' => 'you are an expert in astronomy'
        ],
        [
            'role' => 'user',
            'content' => 'which is the furthest planet to earth?'
        ]
    ],
    'model' => 'DeepSeek-R1',    // gpt-4o   DeepSeek-R1
    'temperature' => 1,
    'max_tokens' => 4096,
    'top_p' => 1
];
// Initialize cURL
$ch = curl_init($url);

// Set cURL options
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, [
    'Content-Type: application/json',
    'Authorization: ' . $apiKey,
]);

curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));

// Execute the API request
$response = curl_exec($ch);

// Check for errors
if ($response === false) {
    echo 'Error: ' . curl_error($ch);
} else {

    // Decode the response
    $result = json_decode($response, true);

    // Print the entire response for debugging
    /*
    echo '<pre>';
    print_r($result);
    echo '</pre>';
    */
    // Check if the 'choices' key exists in the response
    if (isset($result['choices'][0]['message']['content'])) {
        echo '<h3>Generated Text by ' . $result['model'] .':</h3>';
        // Print the generated text
        echo "<p>" . $result['choices'][0]['message']['content'] . "</p>";
    } else {
        if (isset($result['error'])) {
            echo 'Error: ' . $result['error']['message'];
        } else {
            echo 'Error: Unable to retrieve generated text.';
        }
    }
}

// Close cURL

curl_close($ch);
?>

In the above code:

  • Set the value of $apiKey to be the personal access token from GitHub
  • The system prompt is: 'you are an expert in astronomy'.
  • The user prompt is: 'which is the furthest planet to earth?'
  • We will be using the ‘DeepSeek-R1’ model

You can start the PHP web server in the PHP-GitHub-AI folder with this terminal window command:

php -S localhost:8888

Point your browser to http://localhost:8888. The output would look like this:


The output is in markdown format. We will need a library that converts from markdown to HTML. To that end, stop the web server and install the erusev/parsedown package into your application with this terminal window command:

composer require erusev/parsedown

Back in the code, make the following changes:
  • Add this code to the first line of your PHP code:
// composer require erusev/parsedown
require_once 'vendor/autoload.php';

  • Replace the following statement:

echo "<p>" . $result['choices'][0]['message']['content'] . "</p>";

WITH

error_reporting(E_ALL ^ E_DEPRECATED);
$Parsedown = new Parsedown();
$text =  $Parsedown->text($result['choices'][0]['message']['content']);
echo "<p>$text</p>";

Restart the web server with “php -S localhost:8888”. The page now shows a much better looking output:

You can change the AI model from DeepSeek-R1 to any other model (like gpt-4o) and will get similar results.

Monday, September 30, 2024

Using Sematic Kernel with SLM AI models downloaded to your computer

In this walkthrough, I will demonstrate how you can download the Phi-3 AI SLM (small language model) from Hugging Face and use it in a C# application. 

What is Hugging Face?

Hugging Face (https://huggingface.co/) provides AI/ML researchers & developers with access to thousands of curated datasets, machine learning models, and AI-powered demo apps. We will download he Phi-3 SLM model in ONNX format onto our computers from https://huggingface.co/models.

What is ONNX?

ONNX is an open format built to represent machine learning models. Visit https://onnx.ai/ for more information.

Getting Started

We will download the Phi-3 Mini SLM for the ONNX runtime from Hugging Face. Run the following command from within a terminal window so that the destination is a location of your choice. In the below example the destination is a folder named phi-3-mini on a Windows C: drive.

git clone https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx C:/phi-3-mini


   NOTE:

   This example only works on the Windows Operating System.

Be patient as the download could take some time. On my Windows computer the size of the download is 30.1 GB comprising 97 files and 48 folders.

We will be using the files in the cpu_and_mobile folder. Inside that folder, navigate into the cpu-int4-rtn-block-32 folder where you will find this pair of files that contain the AI ONNX model:

phi3-mini-4k-instruct-cpu-int4-rtn-block-32.onnx

phi3-mini-4k-instruct-cpu-int4-rtn-block-32.onnx.data

Getting Started

In a working directory, create a C# console app named LocalAiModelSK inside a terminal window with the following command:

dotnet new console -n LocalAiModelSK 

Change into the newly created directory LocalAiModelSK with:

cd LocalAiModelSK

Next, let's add two packages to our console application with:

dotnet add package Microsoft.SemanticKernel -v 1.16.2

dotnet add package Microsoft.SemanticKernel.Connectors.Onnx -v 1.16.2-alpha

Open the project in VS Code and add this directive to the .csproj file right below: <Nullable>enable</Nullable>:

<NoWarn>SKEXP0070</NoWarn>

Replace the contents of Program.cs with the following C# code:

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI; 
 
// PHI-3 local model location 
var modelPath = @"C:\phi-3-mini\cpu_and_mobile\cpu-int4-rtn-block-32"; 
 
// Load the model and services
var builder = Kernel.CreateBuilder();
builder.AddOnnxRuntimeGenAIChatCompletion("phi-3", modelPath); 
 
// Build Kernel
var kernel = builder.Build(); 
 
// Create services such as chatCompletionService and embeddingGeneration
var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>(); 
 
// Start the conversation
while (true) {
    // Get user input
    Console.ForegroundColor = ConsoleColor.Yellow;
    Console.Write("User : ");
    var question = Console.ReadLine()!; 
    
    OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new() {
        MaxTokens = 200
    }; 
 
    var response = kernel.InvokePromptStreamingAsync(
        promptTemplate: @"{{$input}}",
        arguments: new KernelArguments(openAIPromptExecutionSettings){
            { "input", question }
        });
    Console.ForegroundColor = ConsoleColor.Green;
    Console.Write("\nAssistant : ");
    string combinedResponse = string.Empty;
    await foreach (var message in response) {
        // Write the response to the console
        Console.Write(message);
        combinedResponse += message;
    }
    Console.WriteLine();
}

In the above code, make sure that modelPath points to the proper location of the model on your computer.

I asked the question: How long do mosquito live?

This is the response I received:


Conclusion

You can choose from a variety of  SLMs at Hugging Face. Of course, the penalty is that the actual ONNX model sizes are significant making it, in some circumstances, more desirable to use a model that resides online.


Saturday, September 28, 2024

Using Sematic Kernel with AI models hosted on GitHub

Overview

In this article I will show you how you can experiment with AI models hosted on GitHub. GitHub AI Models are intended for learning, experimentation and proof-of-concept activities. The feature is subject to various limits (including requests per minute, requests per day, tokens per request, and concurrent requests) and is not designed for production use cases.

Companion Video: https://youtu.be/jMQ_1eDKPlo

Getting Started

There are many AI models from a variety of vendors that you can choose from. The starting point is to visit https://github.com/marketplace/models. At the time of writing, these are a subset of the models available:


For this article, I will use the "OpenAI GPT-4o mini" model highlighted above. If you click on that model you will be taken to the model's landing page:

Click on the green "Greate Personal Access Token" button.

The first thing we need to do is get a 'personal access token' by clicking on the indicated button above.

Choose 'Generate new token', which happens to be in beta at the time of writing.


Give your token a name, set the expiration, and optionally describe the purpose of the token. Thereafter, click on the green 'Generate token' button at the bottom of the page.


Copy the newly generated token and place it is a safe place because you cannot view this token again once you leave the above page. 

Let's use Semantic Kernel

In a working directory, create a C# console app named GitHubAiModelSK inside a terminal window with the following command:

dotnet new console -n GitHubAiModelSK

Change into the newly created directory GitHubAiModelSK with:

cd GitHubAiModelSK

Next, let's add two packages to our console application with:

dotnet add package Microsoft.SemanticKernel

dotnet add package Microsoft.Extensions.Configuration.Json

Create a file named appsettings.json. Add this to appsettings.json:

{
    "AI": {
      "Endpoint": "https://models.github.ai/inference",
      "Model": "openai/gpt-4o-mini",
      "PAT": "fake-token"
    }
}

Replace "fake-token" with the personal access token that you got from GitHub.

Next, open Program.cs in an editor and delete all contents of the file. Add this code to Program.cs:

using Microsoft.SemanticKernel;
using System.Text;
using Microsoft.SemanticKernel.ChatCompletion;
using OpenAI;
using System.ClientModel;
using Microsoft.Extensions.Configuration;

var config = new ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
    .Build();

var modelId = config["AI:Model"]!;
var uri = config["AI:Endpoint"]!;
var githubPAT = config["AI:PAT"]!;

var client = new OpenAIClient(new ApiKeyCredential(githubPAT), new OpenAIClientOptions { Endpoint = new Uri(uri) });

// Initialize the Semantic kernel
var builder = Kernel.CreateBuilder();

builder.AddOpenAIChatCompletion(modelId, client);
var kernel = builder.Build();

// get a chat completion service
var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

// Create a new chat by specifying the assistant
ChatHistory chat = new(@"
    You are an AI assistant that helps people find information. 
    The response must be brief and should not exceed one paragraph.
    If you do not know the answer then simply say 'I do not know the answer'."
);

// Instantiate a StringBuilder
StringBuilder strBuilder = new();

// User question & answer loop
while (true) {
    // Get the user's question
    Console.Write("Q: ");
    chat.AddUserMessage(Console.ReadLine()!);

    // Clear contents of the StringBuilder
    strBuilder.Clear();

    // Get the AI response streamed back to the console
    await foreach (var message in chatCompletionService.GetStreamingChatMessageContentsAsync(chat, kernel: kernel)) {
        Console.Write(message);
        strBuilder.Append(message.Content);
    }
    Console.WriteLine();
    chat.AddAssistantMessage(strBuilder.ToString());

    Console.WriteLine();
}

Run the application:


I asked the question "How many pyramids are there in Egypt?" and the AI answered as shown above. 

Using a different model

How about we use a different AI model? For example, I will try the 'Phi-4' model. Click on the model on the https://github.com/marketplace/models page.

We need to get the model ID. 

Change Model in appsettings.json to "microsoft/Phi-4".

Run the application again. This is what I experienced with the AI model Phi-4:

Conclusion

GitHub AI models are easy to access. I hope you come up with great AI driven applications that make a difference to our world.