I don't have experience w/ Lambda myself, but I have ample experience in Azure Functions specifically with LlamaIndex. Azure Functions is the Azure equivalent to Lambda, so I can probably talk shop about the architecture.
Happy to chat about it. I have a lot of cloud experience myself (mostly in Azure) but the concepts are the same among all the major platforms.
I'm trying to decide the best way to host an app with LlamaIndex
usually AWS Lambda is straightfoward but since LlamaIndex is quite new I doubt that AWS Lamda machines will have all needed packages
Have you used Lambda before?
I haven't but if its anyhting like Azure Funcs you should be able to specify the python version and required packages for your application. They do a good job managing dependencies.
And if you have a more obscure setup you can usually load a docker image packaged with your dependencies. Microservices for the Big 3 clouds are pretty flexible.
as I was imagining the packages are too big for a lambda
CREATE_FAILED: RagLambdaFunction (AWS::Lambda::Function)
Resource handler returned message: "Unzipped size must be smaller than 262144000 bytes (Service: Lambda, Status Code: 400, Request ID: bfd6aea2-f11aea1d82d8)" (RequestToken: 22e99992-c827fa5ddb5c, HandlerErrorCode: InvalidRequest)
Yes docker is an option but I think I would prefer to build a lambda layer
Maybe I will consider Azure as you are suggesting but I feel more comfortable with AWS
They can be if you have to include some modes i.e sentence transformers. That broke our Azure Functions instance as they were too large for the microservice. But LlamaIndex alone will not do that, it requires you to have some additional integrations or model support for that to happen.
It also exceeded the size of our docker container so, just an FYI there. You may need to be picky on what you package together for any microservice. But that's kinda the point, these aren't meant to be giant monoliths in terms of software.
Also no - I think Lambda is fine. I don't think there's really any extra benefit in using Azure as much as I prefer it. Its mostly just because I'm more familiar with it. I can't reliably say Azure is better than Lambda for this use case its just what I'm more experienced in.
I also encountering this same issue, it's my first exposure to llama and this limit on lambda too.
Even the supposedly lighter llama-index-core package is around 260mb.
Did you settle on another service or stay on lambda?
Is there a more basic version than the core package? Also cc @no_dice
Are there major package size limits for Lambda? I've not run into this issue and we have a relatively large microservice on our Azure insurance.
for lambda layer it's has a 250mb limit
I've not used lambda but the limit is 5gb in Azure Functions - but that's because thats the limit for a Docker container iirc, not necessarily imposed by Azure.
