GitHub Launches AI Models Sandbox
Written by Kay Ewbank   
Monday, 05 August 2024

GitHub has launched a playground for developers to try AI models. GitHub Models can be used to test and compare different models.

GitHub says the tool has been developed because

"a vast segment of developers still lack easy access to open and closed models".

githubdeklogo

The models available in the sandbox include Llama 3.1, GPT-4o and GPT-4o mini, Phi 3 and Mistral Large 2. The tool lets developers test different prompts and model parameters for free in GitHub. There's then a "glide path" to bring the models to developer environments in Codespaces and VS Code, with a final route through to Azure AI for production where developers can switch to the free GitHub token to a token from a paid Azure account.

The Codespaces option comes with sample code for a variety of languages and frameworks, with the aim of letting developers try out various scenarios. This can then be used to set up a prototype or proof-of-concept within developers' own applications.

The models are available in GitHub Marketplace, where developers can adjust model parameters and submit prompts to see how the model responds, and can see the code that corresponds to the parameters that have been selected. The playground is rate limited.

Alternatively, developers can run prompt evals in GitHub Actions with a series of JSON files by piping in the GitHub Models command within the GitHub CLI.

The sandbox also offers free API usage so that developers can experiment with AI models in their own applications, choosing the language they want to use, and which SDK to use. All the models can be used with the Azure AI Inference SDK, and some models support additional SDKs. The free API usage is also rate limited.

The playground and free API usage are rate limited by requests per minute, requests per day, tokens per request, and concurrent requests. If you get rate limited, you will need to wait for the rate limit that you hit to reset before you can make more requests. Low, high, and embedding models have different rate limits.

GitHub Models is available in limited beta now.

 

githubdeklogo

 

More Information

GitHub Models Sign Up Page

Related Articles

GitHub Adds Required Workflows For Actions

GitHub Copilot Provides Productivity Boost 

GitHub Desktop Adds Squashing

GitHub Desktop 2.0 Introduces Stashing and Rebasing

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Microsoft Open Sources 6502 Basic
05/09/2025

Microsoft has released the source code of Basic 1.1 for the 6502 microprocessor on GitHub. The page with the code says the assembly language source code represents one of the most historically signifi [ ... ]



Google Gets To Keep Chrome
03/09/2025

Despite realizing and ruling that Google was and is a monopoly in search and advertising, Judge Amit Mehta has decided that Google can keep control of one of the biggest tools in maintaining that [ ... ]


More News

pico book

 

Comments




or email your comment to: comments@i-programmer.info