You could just parse each row in the csv as a document object, which might also work well with a vector index (just might have to increase the top k if the rows are small, default top k is 2)
doc= Document(text=<text string from row>)
....
index.as_query_engine(similarity_top_k=4)