Yes! If you use GPTSimpleVectorIndex, you input documents get broken into "chunks", and each chunk is embedded.
Then, at query time, the closest matching chunks are sent to the LLM to help generate a response
I recommend checking out the docs, especially this page:
https://gpt-index.readthedocs.io/en/latest/guides/index_guide.html