Add Amazon Bedrock Anthropic Claude v1 and v2 assistants #173
Conversation
|
This PR is for #171 |
There was a problem hiding this comment.
@094459 it seems there was a misunderstanding. To quote myself from #171 (comment)
we won't accept any components that require additional infrastructure to run.
AWS Bedrock certainly fits that bill. We had some initial ideas to provide a ragna.contrib namespace, but scrapped that idea again in favor of community maintained third-party packages.
Seeing that commit is authored by @ricsue-aws a potential solution would be to create a ragna-aws package that holds configuration for Bedrock assistants. Plus, it could also include something like the S3 document that we have in https://github.com/Quansight/ragna/blob/main/examples/s3_documents/ragna_s3_document.py.
That being said, regardless of where the implementation is ultimately put, it needs to somehow find its way into a configuration file. @094459 could you explain your reasoning why this should be part of the Ragna core library rather than just a local extension that you wrote?
| @@ -0,0 +1,79 @@ | |||
| from typing import cast | |||
| import boto3 | |||
There was a problem hiding this comment.
Ignore this if you go for a third party package that can hard-depend on boto3. Otherwise, this should be a requirement on the class below and imported locally to avoid introducing a hard runtime dependency.
|
|
||
|
|
||
| class AmazonBedrockAssistant(ApiAssistant): | ||
| _API_KEY_ENV_VAR = "AWS_ACCESS_KEY_ID" |
There was a problem hiding this comment.
Although we set it here, it is never used anywhere. That hints to the fact that ApiAssistant is probably not the right abstraction and you are better off subclassing from ragna.core.Assistant directly.
| def _instructize_prompt(self, prompt: str, sources: list[Source]) -> str: | ||
| instruction = ( | ||
| "\n\nHuman: " | ||
| "Use the following pieces of context to answer the question at the end. " | ||
| "If you don't know the answer, just say so. Don't try to make up an answer.\n" | ||
| ) | ||
|
|
||
| instruction += "\n\n".join(source.content for source in sources) | ||
| return f"{instruction}\n\nQuestion: {prompt}\n\nAssistant:" |
There was a problem hiding this comment.
This prompt is special for Anthropic, although it should also be working for other assistants.
|
|
||
|
|
||
| class AmazonBedRockClaude(AmazonBedrockAssistant): | ||
| """[Amazon Bedrock Claud v2](https://eu-central-1.console.aws.amazon.com/bedrock/home?region=eu-central-1#/providers?model=anthropic.claude-v2) |
There was a problem hiding this comment.
That is an internal link that requires an AWS account.
|
Hi there. I have no strong opinions where this goes, and only wanted to contribute if you found it useful. I guess I was confused by the term 'additional infrastructure' given that Bedrock acts kind of like Anthropic, OpenAI, so apologies for the misunderstanding. I put this together for folks who might want to try this out but be using Claude via Bedrock rather than directly via Anthropic. Hope this makes sense. |
|
I'm not sure what a local extension means but if you mean a local fork then if that is the recommendation of the project then I have no issues with that. |
No worries here.
Yeah, and that is fine. Like I stated above, maybe we can have a
What I mean is that you can use the implementation you have with Ragna without it being part of the core package. Just throw your implementation in a [core]
# ...
assistants = [
# ...
"ragna_bedrock.AmazonBedRockClaude",
"ragna_bedrock.AmazonBedRockClaudev1",
]If you start the UI now, you should be able to select and use them. If not, please open an issue. |
Additional assistant if you want to use Claudev1 or v2 hosted via Amazon Bedrock.
You need to set AWS environment variables for your AWS credentials (i.e export AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
Code is currently region specific to where Amazon Bedrock is hosted.