Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Bedrock support for Anthropic models #131

Open
blvp opened this issue Jan 30, 2025 · 1 comment
Open

AWS Bedrock support for Anthropic models #131

blvp opened this issue Jan 30, 2025 · 1 comment

Comments

@blvp
Copy link

blvp commented Jan 30, 2025

Since this tool already employs anthropic python sdk it is possible to add AWS Bedrock support out of the box for already supported models.

Right now I'm seeking opinion around what would be the ideal integration pattern:

  1. Adding a separate provider e.g anthropic_bedrock and related configuration/cli options/handlers. and specifically on handler side it is a complete copy (you don't need to modify it besides the client initialization)
  2. extending Handler behaviour to resolve AWS Bedrock client init by model id. Here idea is that the model name/ model id has different values than Anthropic API model name and we need to make that distinction only at create time
  3. Adding a flag for anthropic api | bedrock in the CLI and pass it along in Settings and use this to create client, forcing user to know how to execute
  4. combination of 2/3 where (2) is a default behaviour where nothing is specified and 3 if it is passed along

My own preference is 4, however looking for any advice and I can submit a MR right after.

@blvp
Copy link
Author

blvp commented Jan 30, 2025

Also, I noticed that Bedrock model names is simply prefixing it with "anthropic." and if you decide to use an cross region inference it could be appended with "(us|eu).anthropic." , here is the fork with the implementation of (1) but you can see how easy it is to add AWS Bedrock support:
https://github.com/eli64s/readme-ai/compare/main...blvp:readme-ai:feature/aws_bedrock_anthropic_support?expand=1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant