Welcome to TopChef’s documentation!

TopChef solves the problem of getting a computer “here” to do something on a computer over “there”. It provides an abstraction layer over the networking, so that the computer “here” doesn’t need to known how to connect to the computer “there”, as long as it can connect to the TopChef server. It also wraps all the logic behind an interface that runs through HTTP, so that any of the multitude of HTTP clients out there can work together to solve a problem. TopChef wants to make HTTP networking as easy as making an HTTP request.

TopChef was originally developed at the Institute for Quantum Computing, in order to enable online experiment design, connecting machine learning algorithms to the control computers running experiments. To run TopChef, you will need a server capable of connecting to both “here” and “there”. The recommended approach is to use the docker container. For more information about installation and running the application, consult the project’s README.

The project source code is hosted on GitHub. Report bugs, feature requests, support requests, and other issues to the project’s issue tracker.

Features

  • JSON Schema checks to make sure all messages are valid
  • Job result storage on
  • HTTP REST API for sending and receiving messages
  • HTTP method override

HTTP API Guide

This section contains a detailed description of the HTTP endpoints exposed by this application. For more details about about how HTTP works, see the section Anatomy of an HTTP Request.

Resource Operation Description
Job GET /services/(service_id)/queue Get the next few jobs
  GET /jobs/(job_id) Get Job Information
  PATCH /jobs/(job_id) Modify job status or results
Job List GET /jobs Get all the jobs in the API
Metadata GET / Get API Metadata
Service GET /services/(service_id)/jobs/next get the next job
  GET /services/(service_id)/jobs Get jobs for the service
  POST /services/(service_id)/jobs create a new job
  GET /services/(service_id) Get service data
  PATCH /services/(service_id) Change service parameters or check in
Service List GET /services Get all the services in the API
  POST /services Create a new service
Validator GET /validator Describe how to validate
  POST /validator Check an instance against a schema

The User Guide

This section contains tutorials on using the server, and on server maintenance. It also provides introductory guides to some of the technologies that TopChef needs to make it tick. If you want to know how to do something in this server, or why some code was implemented in a particular way, this is the section for you!

API Reference

This section contains the nuts-and-bolts, nitty-gritty description of TopChef. This section is composed primarily of auto-generated documentation built from the source code that runs the server.

Overview

The aim of this project is to have one TopChef server running per group, providing a broker between multiple experiments and clients. To do this, we need to consider the idea of a Service, and a Job.

A Service represents some entity that can listen for jobs, and that does “one thing”. Each Service has one JSON Schema for posting new jobs, and one JSON Schema for posting results. These two schemas specify the contract for the service.

A Practical Example:

Suppose we want a service that adds one to a given integer. The JSON that will create our service is given below. Let’s assign this to a Python variable for use later. All our HTTP requests will be done using Python’s requests library.

job_data = {
   "name": "Add One",
   "description": "Adds one to a given number",
   "job_registration_schema": {
      "title": "The schema for the service",
      "description": "Adds one to a given number",
      "$schema": "http://json-schema.org/schema#",
      "type": "object",
      "properties": {
         "value": {
            "title": "value",
            "description": "The number to add",
            "type": "integer"
         }
      },
      "required": ["value"]
   },
   "job_result_schema": {
      "title": "Results",
      "description": "The schema for valid job results",
      "$schema": "http://json-schema.org/schema#",
      "type": "object",
      "properties": {
         "result": {
            "title": "result",
            "description": "The result",
            "type": "integer"
         }
      },
      "required": ["value"]
   }
}

This may look like a mouthful, but it is in fact quite simple. The name and description keys at the top level of the job_data variable provide human-readable information to describe what our service will do. The job_registration_schema describes what the job parameters must look like. In our case, the service will take an object that will have a key value, and the value of this key must be an integer. For instance, the object

{"value": 1}

is a valid parameter, but the object

{"value": "1"}

is not.

Let’s say a TopChef server was already started, and is running on address http://localhost:5000. Using requests, the code to do this will look something like

import requests
import json

url_to_post_to = 'http://localhost:5000/services'
request_headers = {'Content-Type': 'application/json'}

response = requests.post(
   url_to_post_to, headers=request_headers,
   json=job_data
)
assert response.status_code == 201

The assert statement at the bottom checks that the response returns the status code 201, indicating that our service was successfully created. As part of the process, TopChef will assign a universally-unique identifier (UUID) to our service. Let’s say that ID is 66ca5284-ba62-4307-8739-4a09466a924f.

In order to send jobs to this service, we can run code that looks something like this

import requests
import json

data = {'parameters': {'value': '1'}}
request_headers = {'Content-Type': 'application/json'}
url_to_post_to =
   'http://localhost:5000/services/66ca5284-ba62-4307-8739-4a09466a924f/jobs`

response = requests.post(
   url_to_post_to, headers=request_headers,
   data=data
)
assert response.status_code == 201

In a similar way to services, each Job also gets a job UUID. Let’s say that the job ID is d476cf16-356e-4828-bcb4-81c39f0c1aeb.

If we were to send a GET request to /jobs/d476cf16-356e-4828-bcb4-81c39f0c1aeb, we would find that the request body looks like

{
   "id": "d476cf16-356e-4828-bcb4-81c39f0c1aeb",
   "status": "REGISTERED",
   "parameters": {
      "value": 1
   },
   "results": null
}

We wait for the service to finish this job, and after some time, we get the result

{
   "id": "d476cf16-356e-4828-bcb4-81c39f0c1aeb",
   "status": "COMPLETED",
   "parameters": {
      "value": 1
   },
   "results": {
      "result": 2
   }
}

Our job is now complete.

Indices and tables