can llamaindex be used in a large

It sure can! Costs depend on what you are indexing, and which LLMs you are using.

You can prototype something pretty quickly and get a feel for the token usage per query

@Logan M @Deleted User
Hey guys, we've been quite cost sensitive with my team building LLM apps, until the expenses got us frustrated (especially OpenAI APIs). This made us start building out a solution where the LLM usage for devs can be free (more on how on our home page).

The solution we provide right now is a free LLM API built on top of the latest SOTA open-source models, our Alpha version has Mistral 7B, which is on bar with gpt3.5

As it's totally free to use, I'd recommend you check it out https://www.llmos.dev/ and lmk what you think 🙂

free? why that? sounds like a good alternative to pplx api

cool project though, do you think its suitable for a service receiving high traffic?

128k context length, would be useful to alot of people, thinking of adding yarn models?

there is also one for mistral 7b

we believe that AI should be free like majority of the open internet, where on the one end people are enabled to craft their business models around AI/LLMs and on the other end model providers (OSS) for these businesses earn a share of the pie. we're sill in an early phase of designin the exact platform for this network, and open to any discussions.
in the core we don't think closed AI such as OpenAI APIs is the best path humanity, but currently there doesn't seem to be an alternative for model providers to get revenue.

there are currently limits, but we're open to discuss specific needs - you can DM me 😉

and we're looking to deploy more models, so thank you for the suggestions!

i will be in contact with you after releasing my product (in ca. 2 weeks hopefully)

it's quite exciting what you're doing

i am also interested in how the infrastructure works

hey @Deleted User sorry for the late response, thank you for sharing the excitement. what are you building? what do you think of yesterday's OpenAI announcements?