Find answers from the community

Updated 10 months ago

Hi, this is a more general question -

Hi, this is a more general question - does anyone care about typing in the python/AI world at all? As my app is getting more users, stability is becoming increasingly important to me. Yet, I see that most libraries do not bother to provide explicit type definitions, llama_index among them. This leaves me with the option to either stub the types for each library (plenty of work) or ignore them altogether. Am I alone in this? I come from a Java/Scala background and this lack of type strictness is driving me nuts. Typescript seems miles ahead in terms of enabling maintainable code. I'm looking for general recommendations on how to approach this
Attachment
image.png
s
Ł
L
23 comments
I care. Mostly I just use pydantic and annote the parameter and return types. But I don't want to break my head around types (unlike java where it plain sucks if dealing with non-String non-primitive types).
Yeah I agree it's a double-edges sword - when you need the flexibility Java sucks. Typescript has good balance imo
When you're defining new methods/objects that's all fine, but how do I solve the case I posted above?
Typescript is just an abstraction though. There are no types in JS world.
Afaik you can only define stubs
ofc, but for practical purposes it does get the job done
or send PRs to corresponding libraries. As far as I see, most of the types are annoted in the method definations for langchain and llama-index.
of course old libraries like pandas may have types missing in some places.
Also use some good linter and enable type checking rules. Should capture most issues in the codebase.
Yeah I'm currently trying to navigate a solid mypy/pylance setup
Have you tried ruff yet?
Nope, will check it out
Looks nice, thanks
Holy fuck ruff is 🔥
@Łukasz as_query_engine is a gigantic catchall and wrapper around every retriever + retriever query engine.

Listing all the args here would a) be like a list of 20+ and b) need to be written for ever index

Is it a great design? Nah not really. But the actual retirvers and query engines themselves are fully typed, as well as the rest of the library
Also yes, ruff is great, we use it in our CI
@Logan M Indeed, I've figured it out for now - part of my the problem is that the tooling doesn't get the job done for me, ex. infer type from usage - as far as I know VsCode is too dumb to achieve this (haven't tried pycharm in a while). Same goes for importing - unless I explicitly inspect the source code & write the import myself, there's plenty of types that aren't "importable" oob. I've no idea why that is but that's a different story
Ex. I've had to manually import both of these at the top:
Plain Text
from llama_index.core.base.response.schema import RESPONSE_TYPE, Response
from utils.promptlayer import PromptName, get_prompt_from_promptlayer

logger = logging.getLogger(__name__)


async def run_prompt(nodes, name: PromptName) -> str:
    logger.info(f"Running prompt {name}")
    start_time = time.time()

    index = VectorStoreIndex(nodes=nodes, show_progress=True)
    query_engine = index.as_query_engine()
    prompt = get_prompt_from_promptlayer(name)
    result: RESPONSE_TYPE = await query_engine.aquery(prompt)
    typed_response = cast(Response, result)
Yes, the imports are kinda messy.
I do like the attempts to clean up the code base in 0.10x releases, but more often than not I use the GitHub or Google search for what I intend.
I do wish copilot would adapt to these changes faster but unless they rearchitect to search from local code in some form rather than pre-training code, these issues will remain for the time being.
I thought it was just me being clunky with python dev but if that's just how things are I can live with that 😄 Thanks for the feedback guys
yea I 100% use ctrl-f with the source code open lol
Glad you like the release! It was a ton of work (and still some kinks). Hoping this will make the codebase more scalable 🙂
Add a reply
Sign up and join the conversation on Discord