How to access Amazon Bedrock Foundational Models using Boto3 SDK

Accessing Amazon Bedrock models using AWS Boto3 client

Amazon Bedrock is a fully managed service for generative AI-powered applications.

It allows you to build and scale generative AI applications using foundation models. With the Boto3 SDK for Python, you can easily interact with Amazon Bedrock and invoke models for running inference.

Below parameters are required to invoke the model:

• contentType (string): The MIME type of the input data in the request. The default value is application/json.
• body (bytes or seekable file-like object): Input data in the format specified in the content-type request header. Your body consist of the prompt that you pass to the foundation models for prediction.
• modelId (string): Identifier of the model. Ex. anthropic.claude-v1 model id can bed found from the AWS Bedrock console.

Here is an example of how to use invoke_model with Boto3:

import boto3
# Create a Boto3 client for Bedrock Runtime

client = boto3.client('bedrock-runtime')

# Specify the input data and model ID
input_data = "{\"prompt\":\"\\n\\nHuman: Hello Claude How are you?\\n\\nAssistant:\",\"max_tokens_to_sample\":300,\"temperature\":0.5,\"top_k\":250,\"top_p\":1,\"stop_sequences\":[\"\\\\n\\\\nHuman:\"]}"

model_id = 'anthropic.claude-v2'

# Invoke the model for inference
response = client.invoke_model(contentType='application/json', body=input_data, modelId=model_id)

# Retrieve the inference response
inference_result = response['body'].read().decode('utf-8')

# Process the inference result
print(inference_result)

output: {“completion”:” I’m doing well, thanks for asking!”,”stop_reason”:”stop_sequence”,”stop”:”\n\nHuman:”}

You would need latest boto3 SDK for using bedrock client. The supported version is shown as below.

boto3                         1.28.57

You can upgrade the boto3 client version using below command.

pip install -U boto3

Once you upgrade the boto3 client you will be able to use the bedrock client using boto3.

If you are using an older version of boto3 you might be receiving the below error message. To fix this error message upgrade the boto3 client using above command. 🤗

UnknownServiceError: Unknown service: 'bedrock-runtime'. Valid service names are: