We wanted to have clustering shipped this week, but unfortunately we ran into some issues, namely managing GPU resources. In general, managing our own GPUs has been more work than its worth so we have decided to a managed service. This will allow us to focus on the core features of the platform and not get bogged down in the details of managing a cluster. So a lot of the work we did this week was in the background, switching over to a managed service.

In the meantime, we are shipping notifications and a revamped logging API.

Notifications

A lot of the tasks you perform with Glaive are asynchronous and can take a while to complete. These tasks include generating a new dataset version, training a model, or clustering a version. We have built a system that should let us send you an email when these tasks are completed, making it easier for you to keep track of what's happening in the platform.

Currently, we are only sending notifications for dataset version generation in your personal account, but we will be expanding this to other areas of the platform soon.

Logging API

Previously, we had a series of API endpoints called outputs, intended to let you measure the quality of your models by uploading prompt-response pairs with a score. While we still believe the idea is good, the implementation was flawed. Not only was the name terrible, but the design was also bad. So, we have reworked this into the following endpoints:

• /logs/create - Allows for the logging of many prompt, response pairs. Each pair must have a score.
• /logs/list - Allows for the listing of logs.
• /logs/dump - Allows for the dumping of your logs to a file. (Not yet implemented).

Once we build out the evaluation/measurement features of the platform, we will be using logs to help you understand how your models are performing.

Changelog

• Fixed a lot of bugs related to loading datasets into the viewer.
• Added notifications for dataset version generation.
• Standardized naming from "input" -> "prompt" and "output" -> "response".
• Removed the outputs API and replaced them with the logs API.