Skip to content

Latest commit

 

History

History
73 lines (44 loc) · 2.91 KB

anyscale.md

File metadata and controls

73 lines (44 loc) · 2.91 KB
Raw

Anyscale

Anyscale

Anyscale is a leading AI platform that enables developers and AI teams to build, deploy, and scale AI applications with unmatched efficiency. Built on the Ray open-source framework, Anyscale offers a fully managed platform with capabilities like orchestration, experiment management, and hyperparameter tuning. Anyscale is used by thousands of organizations to accelerate their AI development, providing a seamless experience from laptop to production across diverse AI workloads.

Interface Name

  • anyscale

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'anyscale': process.env.ANYSCALE_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('anyscale', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

  • default: mistralai/Mixtral-8x22B-Instruct-v0.1
  • large: meta-llama/Llama-3-70b-chat-hf
  • small: mistralai/Mistral-7B-Instruct-v0.1
  • agent: mistralai/Mixtral-8x22B-Instruct-v0.1

Embeddings Model Aliases

  • default: thenlper/gte-large
  • large: thenlper/gte-large
  • small: BAAI/bge-large-en-v1.5

Options

The following parameters can be passed through options.

  • frequency_penalty: Penalizes new tokens based on their existing frequency in the text so far, reducing the likelihood of repeating the same line. Positive values reduce the frequency of tokens appearing in the generated text.
  • max_tokens: The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
  • temperature: Controls the randomness of the AI's responses. A higher temperature results in more random outputs, while a lower temperature makes the output more focused and deterministic. Generally, it is recommended to alter this or top_p, but not both.
  • top_p: Controls the cumulative probability of token selections for nucleus sampling. It limits the tokens to the smallest set whose cumulative probability exceeds the threshold. It is recommended to alter this or temperature, but not both.

Features

  • Embeddings

Getting an API Key

Commercial with Free Trial: The Anyscale API does not require a credit card and comes with $10 credit to get started.

To get an API key, first create an Anyscale account, then visit the link below.

Anyscale Documentation

Anyscale documentation is available here.