Class BedrockChat

A type of Large Language Model (LLM) that interacts with the Bedrock service. It extends the base LLM class and implements the BaseBedrockInput interface. The class is designed to authenticate and interact with the Bedrock service, which is a part of Amazon Web Services (AWS). It uses AWS credentials for authentication and can be configured with various parameters such as the model to use, the AWS region, and the maximum number of tokens to generate.

The BedrockChat class supports both synchronous and asynchronous interactions with the model, allowing for streaming responses and handling new token callbacks. It can be configured with optional parameters like temperature, stop sequences, and guardrail settings for enhanced control over the generated responses.

Example

import { BedrockChat } from 'path-to-your-bedrock-chat-module';
import { HumanMessage } from '@langchain/core/messages';

async function run() {
  // Instantiate the BedrockChat model with the desired configuration
  const model = new BedrockChat({
    model: "anthropic.claude-v2",
    region: "us-east-1",
    credentials: {
      accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
      secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
    },
    maxTokens: 150,
    temperature: 0.7,
    stopSequences: ["\n", " Human:", " Assistant:"],
    streaming: false,
    trace: "ENABLED",
    guardrailIdentifier: "your-guardrail-id",
    guardrailVersion: "1.0",
    guardrailConfig: {
      tagSuffix: "example",
      streamProcessingMode: "SYNCHRONOUS",
    },
  });

  // Prepare the message to be sent to the model
  const message = new HumanMessage("Tell me a joke");

  // Invoke the model with the message
  const res = await model.invoke([message]);

  // Output the response from the model
  console.log(res);
}

run().catch(console.error);
Copy

For streaming responses, use the following example:

Example

import { BedrockChat } from 'path-to-your-bedrock-chat-module';
import { HumanMessage } from '@langchain/core/messages';

async function runStreaming() {
  // Instantiate the BedrockChat model with the desired configuration
  const model = new BedrockChat({
    model: "anthropic.claude-3-sonnet-20240229-v1:0",
    region: "us-east-1",
    credentials: {
      accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
      secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
    },
    maxTokens: 150,
    temperature: 0.7,
    stopSequences: ["\n", " Human:", " Assistant:"],
    streaming: true,
    trace: "ENABLED",
    guardrailIdentifier: "your-guardrail-id",
    guardrailVersion: "1.0",
    guardrailConfig: {
      tagSuffix: "example",
      streamProcessingMode: "SYNCHRONOUS",
    },
  });

  // Prepare the message to be sent to the model
  const message = new HumanMessage("Tell me a joke");

  // Stream the response from the model
  const stream = await model.stream([message]);
  for await (const chunk of stream) {
    // Output each chunk of the response
    console.log(chunk);
  }
}

runStreaming().catch(console.error);
Copy