A common nusance when building AI apps is context embeddings. If you use a traditional stack you generall have to store an embedding, vectorize it, store the vector, then when you query a vector you’ll get another reference to the file which you then have to fetch again. Pinata’s solution is much more elegant. With Pinata’s file vectoring you can upload a file and vector it at the same time.
When it comes time to query a vector, you have the option to either list your query results and judge the matching score, or just return the highest scoring file itself.
During the public beta, Pinata Vector Storage has the following limits:
File Vectorization Limit: You can vectorize up to 10,000 files.
Index Limit: You can create a maximum of 5 indexes, managed using Pinata Groups.
Results Limit: You can query a vector and get a max of 20 files returned in one request.
What this meansYou can organize your vector embeddings into up to 5 searchable indexes using Pinata Groups. Across all these groups, you can store a total of 10,000 vector embeddings corresponding to files stored on Pinata.
A unique feature in the SDK is if you pass in the returnFile flag then you will get the file and it’s contents rather than just the reference to the file.